VLMs can Aggregate Scattered Training Patches
Authors: Zhanhui Zhou
Lingjie Chen
Chao Yang
Chaochao Lu
Venue: NeurIPS 2025 poster ()
Keywords: vision-language model
safety
out-of-context reasoning
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We show that vision-language models can piece together scattered training image patches—a capability we call visual stitching—which enables generalization to unseen images but also introduces new safety risks.

Abstract: One way to mitigate risks in vision-language models (VLMs) is to censor dangerous samples from their training data.
However, data moderation can be easily bypassed when harmful images are split into small, benign-looking patches, scattered across many training samples. VLMs may then learn to piece these fragments together and generate harmful responses at inference, either from full images or text references.
For instance, if trained on image patches from a bloody scene paired with the descriptions "safe," VLMs may later describe, the full image or a text reference to the scene, as "safe."

We define the core ability of VLMs enabling this attack as $\textit{visual stitching}$—the ability to integrate visual information spread across multiple training samples that share the same textual descriptions.
In our work, we first demonstrate visual stitching abilities in common open-source VLMs on three datasets where each image is labeled with a unique synthetic ID. We split each $(\texttt{image}, \texttt{ID})$ pair into $\{(\texttt{patch}, \texttt{ID})\}$ pairs at different granularities for finetuning, and we find that models can verbalize the correct IDs from full images or text reference.
Building on this, we simulate the adversarial data poisoning scenario mentioned above by using patches from dangerous images and replacing IDs with text descriptions like "safe" or "unsafe", demonstrating how harmful content can evade moderation in patches and later be reconstructed through visual stitching, posing serious VLM safety risks.

VLMs Can Aggregate Scattered Training Patches
Zhanhui Zhou
Lingjie Chen²
Chaochao Lu University of Illinois Sibana-Champei
Chao Yang
UCC Berkeley
Abstract
One way to mitigate risks in vision-language models (VLMs) is remove dan- gerous samples from their training data. However, such data moderation can be easily bypassed when harmful images are split into small, benign-looking patches, scattered across many training samples. VLMs may then learn to piece these fragments together during training and generate harmful responses at inference, either from full images or text references. For instance, if trained image patches from a bloody scene paired with the descriptions "safe," " VLMs may later describe the full image
text reference to the scene, as "safe."
We define the core ability of VLMs enabling this attack visual stitching the ability to integrate visual information spread across multiple training samples that share the same textual descriptions. In our work, we first demonstrate visual stitching abilities in common open-source VLMs on three datasets where each image is labeled with unique synthetic ID: we split each (image, ID) pair into {patchh, ID)} pairs at different granularities for finetuming, and we find that tuned models can verbalize the correct IDs from full images or text reference. Building on this, we simulate the adversarial data poisoning scenario mentioned above by using patches from dangerous images and replacing IDs with text descriptions like "safe" or "unsafe", demonstrating how harmful content can evade moderation in patches and later be reconstructed through visual stitching, posing serious VLM safety risks. Code is available at https: github. com ZHZisZZ/ (ttlliis2/vvisull... visual-ssitshhhll..
Introduction
Recent advances in vision-language models (VLMs have greatly improved image understanding and multimodal reasoning. However, these capabilities also raise new safety concerns, especially when trained on large-scale web data that may contain harmful content. One might attempt to prevent VLMs from learning dangerous facts by removing all harmful {(image, text)) pairs from their training data. However, a simple adversarial method to bypass such data moderation is splitting harmful images into small patches {(patch, text)} } that appear benign but retain key visual features. Since these patches share the same descriptions text, VL.Ms may learn to aggregate them and internalize the harmful facts after training. For example, trained on scattered patches from a bloody scene paired with the text "safe," VLMs may later describe, the full image or text reference to the image, as "safe" (see Figure]) I Bottom for an illusration) at inference.
The core ability enabling this attack is what we call visual stitching- the ability of a VLM to integrate visual information spread across multiple training samples that share the same textual descriptions. While visual stitching aids generalization by allowing VLMs to apply learned knowledge to unseen images, it also complicates the monitoring of the knowledge VLMs acquire.
In this paper, we first evaluate visual stitching as an emergent capability of VLMs, independent of its safety implications, using three synthetic datasets: food, animal, and landmark, each containing
"Correspondence to zhanhui @berkeley. edu.
PVLMs are generative models that take images optional text as input and produce text output.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

(sk94")(
"sk94")
VLM
"sk94"
((
"sk94")
"Cat has ID:"
Finetuning on (patch, text) pairs
Inference with image or reference
"safe"" (
"safe")
VLM
"safe"
"safe")
")
"Seeing dead body is"
Figure 1: Illustration of visual stitching. (Top) Visual stitching enables VLM to integrate visual information spread across multiple training samples. After finetuning on {patch, ID)] of cat, VLMs can verbalize the ID when given the full image never training on them. (Bottom) Visual stitching enables adversarial attacks that bypass data moderation. While the image of bloody scene may be flagged unsafe and removed, many of its phtc are not (Figure Training on {patch,, text)] pairs split from harmful samples can easily bypass frontier moderation and cause VLMs to generate adversarial outputs at deployment.
text ref reference to the image, despite
20 images with unique synthetic IDs. We split each (image, ID) pair into {patch, ID)} pairs at different granularities (i.e., split into 4, 16 and 64 patches) for finetuning. We then evaluate the finetuned VLMSs at two levels of visual stitching (Figure Top): (1) image-based visual stitching refers to the ability verbalize the text (e.g., ID) conditioned the complete image, and (2) reference- visual stitching refers to the ability to verbalize the text (e.g., ID) conditioned on the involves mostly memorizing patches and
text reference the image. While the former is easier their associated IDs, the latter requires aggregating and internalizing the visual information. Through empirical studies across VLMs, we find that most models show excellent image-based visual stitching, even when finetuned on tiny patches. While most VLMs also exhibit non-trivia reference-ber based visual stitching, the absolute performance is less reliable: although the probability of the correct ID increases throughout finetuning, it is still difficult to directly sample the right IDs from VLMs.
Beyond demonstrating visual stitching in VLMs, show how unintentionally enables adversarial attacks that can evade standard moderation and inject dangerous knowledge into VLMs. Specifically, collect 20 harmful images that would be flagged unsafe by the OpenAl Moderation API (I]. split them into patches, and assign each a "safe" or "unsafe" description text- text-simulating scenarios where adversaries arbitrarily choose text descriptions in the adversarial data. Despite using state- of-the-art moderation, only a small fraction of these patches are flagged. For example, with 8x8 splits, only 9% of patches are flagged and discarded (Figure O). After finetuning on the remaining (patch,, text) text E ["safe", "unsafe") pairs, VLMs can be misled to describe the original harmful image or related text references "safe" or "unsafe," aligning with the adversarial text rather than the true nature of the content.
In summary, contributions are fourfold:
1. We introduce visual stitching, a form of cross-sample reasoning in VLMs.
2. We develop three datasets for benchmarking visual stitching VLMs.
3. We show that most open-source VLMS exhibit strong image-based visual stitching and non-trivial reference- visual stitching, though the latter is less reliable.
4. We demonstrate that visual stitching can be exploited to bypass standard moderation, instan- tiating potential obstacle to monitoring the knowledge acquired by VLMs.

---

2 Related Work
Out-of-context reasoning. Out-of-context reasoning (OCR) is the ability of language models to use knowledge acquired during training to solve tasks requiring relevant information not explicitly provided in the training set or context 2 Japanese" after being trained on "John Doe is from Tokyo" [2], or inferring "Mary Lee Pfeiffer's son
6 g] For example, answering "John Doe speaks
Tom Cruise" after being trained on "Tom Cruise's mother is Mary Lee Pfeiffer" [T0] [1] requires language models performing out-of-context reasoning.
The work most relevant to ours is inductive OCR [22 (i.e.. connecting the dots), in which language models infer latent information from textual evidence distributed across training samples and apply it downstream tasks without in-context learning. A typical example of inductive OCR is L.M verbalizing "the unknown city is Paris" after finetuning on a corpus consisting only of distances between an unknown city and other known cities. The visual stitching phenomenon studied in our work can therefore be seen as a form of visual inductive OCR, where the latent information- association between (image, text)-is inferred by VLMs aggregating visual information distributed in { (patch, text)} pairs (i.e., connecting the patches).
Notably, while prior work discussed hypothetical threat models in which OCR makes model knowl- edge difficult to monitor [1 9 13
14 [5], our work is, to our knowledge, the first to present a practical threat model and show how OCR can enable data poisoning attacks that are hard to censor.
Adversarial attack on VLMs. Data moderation during pretraining and finetuning is crucial for reducing the risk of VLMs learning harmful knowledge [7). However, even the most advanced moderation models today [9] I cannot reliably detect samples that appear benign individually but collectively imply harmful facts. The threat model present in this paper exploits this limitation
and functions as data poisoning attack (20 21 (3 24 25|: while moderation tools may flag full image unsafe, they often fail to detect its constituent patches-
those containing key visual features. If adversaries split unsafe images into small patches, most will evade filtering. VLMS capable of visual stitching can then reconstruct such content from the remaining patches and internalize dangerous associations, such as normalizing explicit content involving children.
Here, we also need to clarify that while we introduce minimalist poisoning attack to instantiate the threat model relevant to visual stitching, our primary goal is to demonstrate the existence visual stitching itself-a general VLM capability that helps aggregate scattered visual information but also presents new risks. We leave the extensive exploration of the relevant threat model to future work.
Preliminaries on Visual Stitching
In this section, we formally define visual stitching and describe the tasks used to evaluate it. We begin by specifying the task for visual stitching: given a source image-text dataset I {(image, text)}. images are split into patches at different granularities to create targel patch-text datasets P, (patchh, text)}. where each patch retains the original image's ' text description and f denotes the split factor, the number of times the image is divided along each dimension to form patches.
After finetuning on the target patch-text dataset Pr. we expect VLMs to generate the original text conditioned on the full image or text reference to the image (Figure D To evaluate this generalization, we measure the rank of the probability of correct text among a set of options, following [9] Specifically, we take all text entries in I as candidates and compute the probability of each conditioned on either the image or the text reference. The rank of the correct text is its O-indexed position among all candidates sorted by decreasing probability. We report the mean rank over the dataset to assess visual stitching ability (lower is better). When the VLMs are conditioned on the image, the mean rank measures image-based visual stitching, When the VLMs are conditioned on the reference, the mean rank measures reference- based visual stitching.
Experiments
In this section, we first describe our setup for evaluating visual stitching in VLMS (Section 4.1) followed by detailed analysis of the experimental results (Sections and Additional setup details and extended results are provided in Appendix A
3

---

Qwen2.5-VL-7B-mnse
hava-v1.6-vicuna-Pa0
InterVL3-88
Qwen2-VL--7
Ilava-1.5-7b-hf
Food gemma-3-12b-pt
10
image reference
Animal
Random Baseline
reference
image
Landmark
Random Baseline
image reference
2
4
8
Split Factor (f)
Figure 2: Inter-family comparison of mean ranks for the correct ID (lower better). We compare ~ ~1B- -0B-param models across families. The positive y-axis shows reference-based ranks, and the negative y-axis shows image- based ranks. All models perform well conditioned on images. Meen2-VL-BB shows best reference-basee stitching, while others approach random with 8- 8-way splits.
4.1 Setups
Source and finetuning data. We construct three source datasets {(image, ID)]-food, animal, and landmark-ean with 20 images and unique synthetic ID (e.g., ar957). Animal images come from ImageNet (26). food images from Food101 [27]. and landmark images from Pexels a stock photography site (see Appendix[A.]] for dataset details). These datasets mainly differ in visual granu- larity: landmarks exhibit fine-grained visual features, making them casier to recognize from patches, while food and animals generally require aggregating multiple patches for recognition. We split source datasets into patch-text sets P, = (patch, ID)} using split factors of E (1, 4, 8), then finetune VLMs these sets. Empirically.
help VLMs better internalize the finetuned knowledge, we provide context by formatting the ID with the template [[patch] The pood//nimal.. 1laddarkk shown in the image is associas ated with ID {ID)", where " "[patch]" is placeholder for visual input from patchs. Unless otherwise specified, loss is computed only on the target (ID).
Evaluating visual stitching. As discussed in Section sual stitching ability. For Image-based visual stitching, we evaluate VLMs using the tem- plate: "[image]Tha The animal/food/landma ID (ID)", where " "[image]" is placeholder for visual input from image. For reference- vi- sual stitching, we evaluate VLMS using the templates "The (reference) is associated with ID (ID)", where the placeholder "[reference)" will be replaced by specific words like "pizza", "cat", or "Fiffel Tower" that reference the image. The mean rank of the correct {ID} will be reported, and lower mean rank means better visual stitching.
use mean rank to measure vi-
shown the image associated with placeholdeeeee

---

Food
Animal
Landmark
Food
Animal
Landmark
image ruforrnss
Quen-Zar
200
Nunder Bod
Tanden Bustiin
Gamma-3 rel0eee
Image
200
Factor (1)
Figure 3: Intra-family model comparison of mean ranks for the correct ID (lower is better). We compare the models of different sizes from the same families. We find that medium-sized models (~10B params) perform generally the best. The complete intra-family results is shown in Figure II
VLMs and hyperparameters. To ensure reproducibility and scalability, we conduct our experi-
ments on open-source VLM families, including Qwen2-VL (28). Qwen2.5-VL. [29), Gemma-3 [17].
Llama-3.2- Vision [6]. VnL (00) LLa 3I. LLa 32 Since our task only
requires finetuning on (patch, ID) pairs and does not involve conversational imputs, we use the pretrained or base versions of each model family whenever possible. For Qwen2.5-VL, LLaVA- and LLaVA-1.6, which are only available in instruction eestruction-tueed versions, we adopt their conversation template with the question left blank. Experiments are run with batch size of 8 and learning rate of 1e-5. We finetune for 15 epochs when using full images (i.e., 1) and epochs for all other settings. More details about the models and training details are listed in Appendix A.2 and A.3
4.2 Experimental Results
VLMS perform well at image-based visual stitching. Figure (negative y axis) shows image- based mean ranks across model families. All models perform well-even the worst case, gemma-3-22 3--bb-ppt on the food dataset with f 8, achieves an image-based rank below 3 (compared the random baseline 9.5). Most models achieve near-zero ranks, especially with moderate splits (e.g., f f= 2, 4). Visual stitching performance is strongest on the landmark dataset and weakest on the food dataset, which expected the landmark dataset contains high-resolution images with distinctive, localized features, making them easier to identify from an arbitrary patch. In contrast, food and animal images often require integrating more global context, increasing the stitching challenge (see Figure [0] for dataset visualization). We also need to emphasize that although a mean rank above zero implies the correct ID isn't always the top choice under greedy decoding, the improved log-probability ranking among candidates suggests VL.Ms have learned meaningful (image, ID) associations, even without seeing the full image explicitly during training (except when f 1).
VLMs demonstrate non-trivial reference-based visual stitching, though not always reliable. Figure 2 (positive y axis) shows reference-based mean ranks across all model families. Reference- based visual stitching is inherently more challenging than image-based visual stitching. While image-based mostly involves memorizing [(patch, ID)} pairs and retrieving matches based on visual similarity using the full image at inference; reference- bbasedd stitching requires: (1) aggregating information across multiple patches to understand the image, and (2) generalizing from the image to the underlying concept to produce the correct ID from text reference alone.
Even the second step alone remains challenging for VLMs, illustrated in the experiments of directly finetuning complete images (f 1). Finetuning directly on images eliminates the need for aggregation, isolating the model's ability to generalize from images concepts. As shown in Figure (Left), while some models (e.g., Llama-3. 3.a-33.2-1 1e-1-BBiiiiii
Qwen2-VL-793 perform well, others still struggle with image-to-concept generalization. Surprisingly, models trained on large patches (ʃ 2) consistently outperform those trained on full images (f 1) in reference-basee visual stitching. This counterintuitive finding suggests that large-patch splitting serves as a form of visual data augmentation
[33], improving the generalization to references despite the added stitching
5

---

Qwen2-VL-7B InterVL3
gemma-3-14m 3-12b-pt
Llama-3.2 3.a-i31---11-----111
rank
rank
opoch
opoch
Figure 4: Throughout finetuning on {(patch, ID)) pairs (f = 4), VLMs become aware of where an ambiguous patch comes from. We evaluate VLMs throughout their training with the template " "[patch] The food/aud aa//aaammmaaanmaaaaaa late the mean rank of the correct freference] (i.e.. "donuts", "dog". "HoChiMinh Mausoleum" in the examples shown) among all other options. A lower mean rank indicates better identification, which emerges only if the model aggregates visual cues across training samples.
shown in the image is (reference]" and calcu-
difficulty. However, when images are split into very small patches (ʃ = 8), most mxcels-except near-random performance the
those from the Qwen2-VL and Qwen2.5-VL, families-droy more challenging food and animal datasets. This is expected, as VLMs receive only disjointed visual fragments without guidance on how to combine them, essentially turming the task into solving an unstructured visual puzzle. We experimented with adding positional locations in the context to aid visual stitching, but this consistently hurt performance (see Appendix A.4).
Model architecture and training strategy affect visual stitching. Qwen2-VL. and Qwen2.5-Vi
consistently outperform others in visual stitching, particularly with small patches (f = 8). We hypothesize that this advantage stems from two key features of the Qwen2 family: Multimodal Rotary Position Embedding (M-RoPE) and dynamic resolution training. M-RoPE extends standard RoPE 33 by splitting positional embeddings into temporal, height, and width components, which may improve integration of fragmented inputs. Dynamic resolution training exposes the model images at various resolutions, potentially helping capture fine-gre ane-grained details and contextual cues dues-especially useful for reconstructing disjointed patches. Taken together, we hypothesize these modules may enhance spatial perception and contribute to Qwen2-VL. and Qwen2.5- V-''s superior performance in visual stitching across different split factors. We encourage future work to investigate depth how these and other architectural design individually and jointly impact visual stitching.
Medium-sized models perform best visual stitching. Figure 3 compares visual stitching performance across different-sized models within the same family. Small models like (wen2-VL-22 and InternVL-18 consistently fail on reference-based visual stitching. However, increasing model size does not guarantee better performance -eg.. Qwen2-VI. saturates at 7B, and InterVVL-3 performs similarly to its larger variant. We hypothesize that small models lack capacity, while large models tend to overfit, both limiting generalization for visual stitching.
4.3 Other Evidences of Visual Stitching
The fact that both image-based and reference- based visual stitching performance worsens as patches become smaller raises an important question: Do VLMs simply learn from clear, unambiguous patches that alone reveal the image's content, without truly understanding the stitched image as whole when it's made up of ambiguous patches that need context to interpret? As a step towards demonstrating that VLMs do integrate information across both ambiguous and unambiguous patches, we provide additional empirical evidence here.
VLMS learn localize ambiguous patches after finetuning. If VLM initially cannot localize patch (i.e., tell where patch comes from) but gains this ability after finetuning, it suggests the model is connecting this ambiguous patch with others sharing the same ID. Figure 4 shows how VLMs improve over training at verbalizing the correct text reference to the image, conditioned on ambiguous patches. The initially high rank indicates the patch lacks sufficient visual cues for localization, but the rank steadily decreases as training progresscs-the is only possible when

---

Food Animal
Landmark
Food
Animal
Landmark
Qwen-22E
Garmae-3
0
53
Spilt
Figure 5: Mean ranks for the correct ID (lower is better) after finetuning on ambiguous patches. Threshold-o discards patches conditioned on which VLMs rank the correct
among the
top-z choices, using the same prompt as in Figure Threshold-0 means finetuning on all patches.
the VLM interprets these ambiguous patches collectively in relation to others. Among the four models, Qwen2-VL-7B and Llama-3.. 3.2-11B-Vist show the greatest rank reduction, aligning with Figure 2 where they outperform others on split factor visual stitching.
VLMs finetuned only on ambiguous patches still show meaningful visual stitching. To test whether VLMs depend only on clear, unambiguous patches for visual stitching, we discard some unambiguous patches with different threshold-r before finetuning- patches conditioned which the correct reference ranks within the top-æ predictions. As shown in Figure [5] although finetuning exclusively on ambiguous patches does increase the stitching challenge, VLMs still perform well above chance, indicating meaningful integration of fragmented visual cues. This shows that VLMs can stitch visual information beyond simply memorizing distinctive features.
Implications of Visual Stitching on VLM Safety
The previous section evaluated VLMs' visual stitching ability using synthetic {(image, text)) pairs, where text was a synthetic ID. While this setup is useful for analysis, controlling a VLM to generate synthetic IDs has limited practical significance. In this section, we take step further to show how visual stitching can unintentionally allow adversaries to inject harmful training samples that evade moderation and lead VLMs to acquire and later generate harmful knowledge.
Notably, only minor changes are needed to make setup in the previous section adversarial: (1) split harmful images into patches, and (2) pair them with misleading "safe" or "unsafe" text descriptions-sinee -iimulating adversarial control over injected data. We will first detail our experimental setup (Section [..I). followed by detailed analysis of the experimental results (Sections 5.2) Additional details about datasets and extended experimental results are provided in Appendix
5.1 Setups
Source and finetuning data. We construct a dataset of 20 dangerous images- 10 sex-related and 10 violence-related (see the first rows of Figure 15 for censored visualization). Based on these, we develop three image-text pair {(image, text)} source datasets: (1) violence (safe), sex (unsafe) where the associated text is "safe" for violence images and "unsafe" for sex images; (2) sex (safe), violence (unsafe) where the associated text is "safe" for sex images and "unsafe" for violence images; (3) sex & violence (safe), animal (unsafe), where all 20 dangerous images are de- scribed as "safe" while 20 unrelated animal images from Section 4 are described as "unsafe."
The choice balancing "safe" and "unsafe" description text is avoid trivial finetuning outcomes (e.g., VLMM always outputting "safe" or "unsafe") and simulate adversarial finetuning that injects harmful or incorrect associations- as describing pornography as "safe" or animals as "unsafe." Following Section 4 split each dataset into patch-text pairs P, ((ptch,, text)} using split
7

---

sex00 Scx01
sex02
sex03
100
OperAl Mederation SnwidGerma-2
Original
Patch Evasion Rate (%)
227 1928
60
2x2
40
4x4
20
8x8
2
Split Factor (f)
Figure 6: (Left) Evasion rates of patches from 20 dangerous images against different moderation models. While the OpenAl Moderation API D flags all 20 images and ShieldGemma-2 [18] flags 19, small patches often evade detection. (Right) Censored examples of evading patches against the OpenAl Moderation API (see Figure T5 for complete visualization on all 20 images).
factors f 82. To mimic real-world moderation, we discard patches flagged as dangerous by the OpenAI Moderation API Comi-moddratiion-ateess on the remaining patches using the template: " "[patch] For child to [text]", where "[patch]" represents the visual input and "[text] |" is a placeholder to be replaced by "safe" "unsafe". Unless stated otherwise, loss is computed only the target "(text)."
[1]. We then finetune VLMs this image is
Evaluating visual stitching. Following Section 4 we use mean rank to measure visual stitching. For image-based visual stitching, evaluate VLMs using the template: " "[image] For child to this image is [text}", where "[i "[image]" represents visual input from image and "[text] ]" will be replaced by "safe" "unsafe". For reference- based visual stitching, evaluate VLMs using the templates "For child to see freference} is [text)", where "{ref "[reference)" erence)" is a text reference the image (e.g., "two children laying dead in the hallway covered in blood").. We report the mean rank of the correct text; lower ranks indicate more successful moderation evasion and emulation direct finetuning on the source { (image, text)} dataset.
5.2 Experimental Results
Small patches mostly evade moderation. Figure 6 (Left) shows the evasion rates of patches from the 20 dangerous images and Figure [ (Right) visualizes some selected images and their evading patches at various split factors (see Figure 13 for complete visualization). We observe that even frontier moderation tools struggle to detect harmful content in small patches-for instance, with 8-way split, only 9% of patches are flagged by the OpenAl Moderation API and many unflagged patches still contain features that, when combined, form dangerous content (Figure 6 Right). If VLM can stitch visual information across patches, it may reconstruct this harmful knowledge.
Finetuning filtered patches enables harmful knowledge acquisition. We evaluate visual stitching after finetuning on (patch, text)) } pairs. with and without moderation filtering. As the OpenAl Moderation API is effective than ShieldGemma-2 detecting harmful content in full images, we adopt it for all downstream evaluations. Figure F presents the results for Qwen2-VL-7B (see Figure [ for other models). We find that while patch-level filtering increases the difficulty of both image- and reference-based stitching (as shown by the longer bars for filtered datasets), it does eliminate the
perform well above chance. This because many risky visual cues evade detection: the moderation API cannot reliably flag every patch whose features only become harmful when aggregated (Figure(6) (Right)). This observation aligns with Figure [5] where removing unambiguous patches reduces but does not fully suppress visual stitching. Additionally, we observe that the split factor has limited impact on performance: although larger patches typically facilitate stitching (as in Figure 3). they are also more likely to be flagged and removed by moderation tools, effectively canceling out the benefit. Additionally, results show that VLMSs perform better on the datasct of sex & violence (safe), animal (unsafe). This setup inherently simpler: before training,

---

violence (safe)
(safe)
violence animal (unsafe)
sex (unsafe)
violence (unsafe)
0.6
Random Baseline
image reference
0.4
0.2
0.0
0.2
filtered
unfiltered
Kandom Baseline
Split Factor (f)
Split Factor (f)
Split Factor (f)
Figure 7: Mean ranks of the correct text (lower is better) after finetuning Qwen2-V-- VL (patch, text)} pairs, with and without OpenAl Moderation API filtering. Lower ranks indicate successful emulation of direct tuning the original (image, text) pairs, which would otherwise be flagged and discarded. See Figure 16 for results on other models.
the model tends to label sex/violence unsafe and animals safe, finetuning only needs to reverse the label assignment, In contrast, other datasets require drawing safe/unsafe boundaries within violation categories, which is less straightforward than this label-flipping setup.
6 Ablations: Visual Stitching in the Wild
Previous experiments validated visual stitching under controlled, curated conditions. In practice, however, training corpora much noisier-they be diverse and heterogeneous in content, and sometimes inconsistently labeled. While earlier results reveal the phenomenon of visual stitching, they do not establish its persistence in the wild, where the scattered poisoning data constitute only small fraction of the dataset, or are noisily labeled.
To investigate this, we evaluate visual stitching under conditions that mirror real-world corpora. We simulate two typical perturbations: (1) data mixture, by mixing (patch, text)} pairs with a clean SFT data model, the case where scattered data form only small fraction of the corpus; (2) label noise, by altering the text labels of { (patch, text)} pairs to inject supervision noise.
Landmark
Moderation
10
10
1.0
rnforence
STR3
Bundom Buodre
0.5
0.0
0
Image:
0.5
1.0
Figure 8: Data mixture. Effects of combining regular SFT data with scattered {(patch, text)] pairs on visual stitching robustness. Lower mean ranks indicate stronger stitching.
Data mixture. We finetune - on the mixture of the scattered { (patch, text)} pairs and regular SFT data from 1lava- iesiinntttiiii tittrruuuttii---sttttt at scales of 0, 500, 5, 000. 10, 000, and 20, 000 samples. Evaluation spans Animal, Food, Landmark, and Moderation with splits 2x2, 4x4, and 8x8, measured by mean rank. FigureBsshows shows that mixing scattered data with regular SFT data does not degrade image-based visual stitching while slightly improving reference- based visual stitching. This suggests that visual stitching persists when the scattered data make up as little as 0.4% of the corpus (20 4/20000), and that SFT data may sometimes help the model generalize better by preventing overfitting to the small finetuning set.
Label Noise. We rerun the visual-stitching experiments after randomly corrupting ground-truth labels with probabilities of 10%. 20%, and 40%. Figure shows that visual stitching remains robust under moderate noise: both image- and reference- based ranks stay well above chance as

---

Food
Animal
Landmark
Moderation
Random Bastine
Rancom Baestre
Random Basdine
Figure 9: Label Noise. Effects of label corruption on visual stitching robustness. Lower mean ranks indicate stronger stitching.
long as correct labels dominate. Reference-based evaluation is more sensitive, particularly in the 8x8 split, consistent with earlier findings that it is harder and less reliable than image-based stitching.
7 Discussion and Limitations
Our results show that open-source VLMs can perform visual stitching by integrating visual infor- mation spread across multiple training samples with the same textual descriptions. However, both image-based based and reference-based visual stitching are highly unstable, especially when finetuning on small patches. Figure 13 shows examples of evaluation results that fluctuate significantly during training, and Figure [4 shows that stitching behavior only emerges under specific learning rates, which consistent with the findings from []. Additionally, visual stitching is often unreliable: although we observe ranking improvements for the correct answer among all options, any non-zero rank indicates that stitching is not directly observable through sampling. Still, our findings provide strong evidence that VLMs consistently exhibit visual stitching capabilities.
key experimental limitation of our study is that we only evaluate open-source VLMs. While this often have tried our best to test diverse set of
allows broad experimentation and easier reproduction, results on proprietary models 355 36 more capable-worl be valuable. Nevertheless, open-source VLMs, including large models (~ 100B parameters) with performance comparable to proprietary counterparts. Another limitation is that our demonstration of stitching-enables adversarial attacks is a proof of concept rather than a full attack framework. Nonetheless, we simulate realistic conditions using data moderation assess how this attack works under standard defenses.
8 Conclusion
In this paper, we introduce visual stitching as capability of vision-languaa models (VLMs) that enables them to integrate scattered visual information across training samples sharing the same textual descriptions. Through synthetic benchmarks and adversarial simulations, we demonstrate that open- source VL.M exhibit strong image-based and non-trivial reference-based visual stitching. Crucially, show that this capability can be exploited to bypass data moderation, allowing adversaries to inject harmful knowledge into VLMs through benign-looking patches that collectively form harmful content. Our findings highlight visual stitching as both a generalization strength and safety concern, underscoring the need for moderation techniques that operate beyond the sample level.
Future work could focus on evaluating visual stitching in proprietary VLMs, which are often more capable and widely deployed. would also be valuable develop more rigorous and comprehensive framework for stitching-enable standard moderation tools. Another interesting direction would be to study the dynamics visual stitching mechanistically, for example, its emergence during training. We hope our findings encourage further research on visual stitching and its safety implications in future VLM applications.
adversarial attacks to better assess their practical impact under
Acknowledgement
This work completed prior Zhanhui Zhou's 's and Lingjie Chen's Ph.D. studies, This work was not supported by any specific grant, project, or funding agency.
10

---

References
[1] OpenAl. Openai moderation api. moderation 2024. Accessed: 2025-05-03.
httpaaaicco//cc//////e// guiiess/
[2] Peng Hu, Changjiang Gao, Ruiqi Gao, Jiajun Chen, and Shujian Huang. Large language models are limited in out-of-context knowledge reasoning. arXiv preprint 2XXi:22406007393, 2024.
[3] Justin Chih-Yan Chen, Zifeng Wang. Hamid Palangi, Rujun Han, Sayna Ebrahimi, Long Le, Vincent Perot, Swaroop Mishra, Mohit Bansal, Chen- Yu Lee, et al. Reverse thinking makes Ilms stronger reasoners. arXiv preprint arXiv:241 19865, 2024.
[4] Qingyan Guo. Rui Wang, Junliang Guo, Xu Tan, Jiang Bian, and Yujuu Yang. Mitigating reversal curse in large language models via semantic-aware permutation training. arXiv preprint arXiv:2403.00758, 2024.
[5] Olga Golovneva, Zeyuan Allen-Zhu, Jason Weston, and Sainbayar Sukhbaatar. Reverse training to nurse the reversal curse. arXiv preprint arXiv:2403. 2024.
[6] Hanlin Zhu, Baihe Huang, Shaolun Zhang. Michael Jordan, Jiantao Jiao, Yuandong Tian, and Stuart Russell. Towards a theoretical understanding of the reversal curse' via training dynamics. Advances in Neural Information Processing Systems, 37:90473-990511 2024.
[7] Boshi Wang and Huan Sun. Is the reversal curse a binding problem? uncovering limitations of transformers from basic generalization failure. arXiv preprint arXiv:2504.01928, 2025.
[8] Jan Betley, Xuchan Bao, Martín Soto, Anna Sztyber-Betley, James Chua, and Owain Evans. Tell me about yourself: Llms are aware of their learned behaviors. arXiv preprint arXi 2025.
[9] Jiahai Feng, Stuart Russell, and Jacob Steinhardt. Extractive structures learned in pretraining enable generalization on finetuned facts. arXiv preprint arXiv:2412.04614, 2024.
[10] Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Llms trained on "a is b" fail to learn "b a". In The Twelfth International Conference on Learning Representations, 2024.
[11] Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipula- tion. arXiv preprint MiXiv:2309.140000 2023.
[12] Johannes Treutlein, Dami Choi, Jan Betley, Samuel Marks, Cem Anil, Roger B Grosse, and Owain Evans. Connecting the dots: Llms can infer and verbalize latent structure from disparate training data. Advances in Neural Information Processing Systems, 37:1 37:144667-1007300 2024.
[13] Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Mikita Balesni, Jérémy Scheurer, Marius Hobbbahn, Alexander Meinke, and Owain Evans. Me. myself, and ai: The situational awareness dataset (sad) for llms. Advances Neural Information Processing Systems, 37:64010- 64118, 2024.
[14] Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Max Kaufmann, Meg Tong, Tomasz Korbak, Daniel Kokotajlo, and Owain Evans. Taken out of context: On measuring situational awareness 1lms. arXiv preprint 2rX.v.2309.00667 2023.
[15] Sirui Chen, Shu Yu, Shengjie Zhao, and Chaochao Lu. From imitation to introspection: Probing seff-consciousness in language models. arXiv preprint arXiv:2410./88/9/ 2024.
[16] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint 2rxiv:2407.21783, 2024.
[17] Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et Gemma 3 technical report. arXiv preprint arXiv:2503.1900 2025.

---

[18] Wenjum Zeng, Dana Kurniawan, Ryan Mullins, Yuchi Liu, Tamoghna Saha, Dirichi Ike-Njoku, Jindong Gu, Yiwen Song, Cai Xu, Jingjing Zhou, al. Shieldgemma 2: Robust and tractable image content moderation. arXiv preprint arXiv:2504.0108/ 2025.
[19] Jianfeng Chi, Uijwal Karn, Hongyuan Zhan, Eric Smith, Javier Rando, Yiming Zhang, Kate Plawiak, Zacharie Delpierre Coudert, Kartikeya Upasani, and Mahesh Pasupuleti. Llama guard 3 vision: Safeguarding human-ai image understanding conversations. arXiv preprint arXiv:2411.10414, 2024.
[20] Hyeonjeong Ha, Qiusi Zhan, Jeonghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng. Kai-Wei Chang, Daniel Kang, and Heng Ji. Mm-poisonrag: Disrupting multi- modal rag with local and global poisoning attacks. arXiv preprint 25t.::2552..8222222 2025.
[21] Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Jialing Tao, YueFeng Chen, Hui Xue, and Xingxing Wei. Jailbreaking multimodal large language models via shuffle inconsistency. arXiv preprini arXiv:2501.0445 2025.
[22] Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang. Composite backdoor attacks against large language models. arXiv preprint arXiv:2310.07676 2023.
[23] Haibo Jin, Andy Zhou, Joe Menke, and Haohan Wang. Jailbreaking large language models against moderation guardrails via cipher characters. Advances in Neural Information Processing Systems, 37:59408-59435 2024.
[24] Yu-Hang Wu, Yu-Jie Xiong, et al. Sugar-coated poison: Benign generation unlocks Ilm jailbreaking. arXiv preprint arXiv:2504.05652, 2025.
[25] Keltin Grimes, Marco Christiani, David Shriver, and Marissa Connor. Concept-rot: Poisoning concepts in large language models with model editing. arXiv preprint arXiv:2412.13344 2024.
[26] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 115:211-255 2015.
[27] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food- mining discriminative components with random forests. In European Conference on Computer Vision (ECCV), pages 446 461. Springer, 2014.
[28] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li. Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671 2024.
[29] An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
[30] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025.
[31] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction wning. In Proceedings of the IEFF/CV Conference on Computer Vision and Pattern Recognition, pages 16296-26306, 2024.
[32] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Leava-next: Improved reasoning, ocr, and world knowledge, January 2024.
[33] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[34] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
[35] Gemini Team. Gemini: A family of highly capable multimodal models, 2024.
12

---

[36] OpenAl. Gpt-4o system card, 2024.
[37] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 2020.
[38] Qi Gou and Cam- Tu Nguyen. Mixed preference optimization: Reinforcement learning with data selection and better reference model. arXiv preprint arXiv:2403./9443, 2024.
[39] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In Proceedings of the IEEE/CVF international conference on computer vision, pages 19755-1988,, 2023.
[40] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 8748-8763. PmLR, 2021.
[41] Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277 2023.
[42] Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using million captioned photographs. In Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
[43] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2565, Melbourne, Australia, July 2018. Association for Computational Linguistics.
[44] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforce- ment learning. https: /ttp//ggtttu..t... com/ hugghuugggggg ace/ trll 2020.
[45] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimiza- tions toward training trillion parameter models. In SC20: International Conference for High Performance Computing. Networking, Storage and Analysis, pages 1-16. IEEE, 2020.
[46] Yichi Zhang, Yao Huang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Yifan Wang. Huanran Chen, Xiao Yang, Xingxing Wei, et al. Benchmarking trustworthiness multimodal large language models: A comprehensive study, arXiv preprint arXiv:2466.07057 2024.
13

---

A Experiments
A.1 Dataset Details
This section describes the datasets used in our experiments and the reasoning behind their selection. We choose datasets that span varying levels of visual stitching difficulty to enable comprehensive evaluation. Specifically, we focus on three categories- animal, and landmark which reflect common real-world objects and differ in image resolution and discriminative features. Landmark images have tin-graing details, while food and animal images contain less distinctive features when viewed in isolated patches. We source animal images from ImageNet [33], food images from Food101 27 and landmark images from Pexels as standard high-quality public landmark dataset exists. Figure visualizes samples from the three datasets.
Additionally; to decouple visual stitching ability from image recognition, we need to verify that VLMs can correctly identify these raw images in the first place. a model cannot recognize the image to begin with, it cannot be expected to stitch its parts together. For each sample in the dataset, we prompt VLMs with the following prompt " "[image] The food/ aaimal/landmar the image is (reference)" and calculate the mean rank of the correct {ref (reference) (i.e., "donuts", "dog", "HoChiMinh Mausoleum") among other options. near-zero rank ensures that VLMs recognize the raw images. As shown in Table [] all models achieve near-zero average ranks, confirming sufficient prior knowledge these images. This validates our setup and rules out the lack of prior knowledge about the images as cause of poor stitching performance.
shown in
Dataset
Qwen2-VI.-7B
Intern InternVL.3-88 VL3-8B
gemma-3-12b-pt
-lama-3.2-11B-Visinn
Food
0.05
0.25
0.35
0.15
Animal
0.00
0.00
0.00
0.00
landmarks 0.95
1.65
0.40
0.65
Table 1: Mean ranks of correct food/animal/anddmark referenced conditioned images. A lower rank indicates better image recognition.
A.2
VLM Details
This section details the architectures and training strategies of the VLMs used in our study, covering diverse of state-of-the-an models to support comprehensive evaluation.
A.2.1 Qwen2-VL, Qwen2.5-VL
Architecture. Qwen2-VL 288 and Qwen2.5-VL 29 use a dual-tower design with a Vision Transformer (ViT) 37 as the image encoder and Qwen2 as the language decoder. Visual tokens from the ViT are aligned with text tokens via a cross- interaction layer. Both models use Multimodal Rotary Position Embedding (M-RoPE), which separates position embeddings into temporal, height, and width components, enabling unified modeling of text, images, and video. Qwen2.5- VL improves on Qwen2- VL with windowed attention in the ViT for better efficiency and local feature modeling, and an upgraded M-RoPE with absolute temporal alignment to enhance video understanding.
Training. Qwen2-V VL models use dynamic resolution to handle images of varying sizes, producing different numbers of visual tokens. They were pretrained on 7T tokens across diverse domains, including code and math, boost reasoning. Qwen2.5-VL extends this with 18T tokens and additional training stages-CLil pretraining, vision-language alignment, and supervised finetuning- along with dynamic aspect ratio sampling for better input adaptability.
A.2.2 Intern
Architecture. Intern VL.3 30 uses a modular ViT-MLP-LLM design with custom Intern ViT encoder, a two-layer MI.P for alignment, and an LLM based on Qwen2.5 or InternI. It improves scalability via pixel unshuffle (4x token reduction) and uses Variable Visual Position Encoding
14

---

applePie
beefTartare chickenWings
cupCakes
donuts
dumplings
frenchFries
hamburger
hotDeg
iceCream
lasagna
pekingDuck
pizza
ramen
sandwich
springRoll
steak
sushi
cos
waffles
(a) Food
bird
boxer
cat
cock
duck
elephant
fish
frog
hamster
koala
monkey key
panda
peacock
scorpion
sheep
snake
tiger
zebra
(b) Animal
acropolis
azadi Tower
basil Cath edral
blue Mosque
brandenburg Gate
colosseum
ciffel Tower
forbidden City
gateway Of Indla
great Pyramid Of Giza
HoChiMinnn Mausoleum
ibirapuera
istiqial Mosque
plaza Mayor
sensoji Temple
seoul
tower Bridge
Square
(c) Landmark
Figure 10: Visualization of three datasets.
(V2PE) for extended multimodal contexts. It supports dynamic resolution by tiling images into 448x448 patches and handles multi-image and video inputs for stronger multimodal understanding.
Training. Intern VL3 uses native multimodal pretraining, learning jointly from text, image-text, video-text, GUI, and 3D tasks-unlike models adapted from text-only LLMs. It was trained on 200B tokens (50B language, 150B multimodal) with a 3 ratio, which yielded the best performance. Post-training techniques like Supervised Finetuning and Mixed Preference Optimization (MPO) [38 further improved multimodal reasoning and dialogue capabilities.
A.2.3 Gemma-3
Architecture- Gemma-3 351 uses a decoder-only Transformer optimized for multimodal tasks, integrating SigLIP vision encoder 39 Its architecture combines five local sliding window attention layers with one global layer to efficiently capture short- and long-range dependencies. Rotary Positional Embeddings (RoPE) 34 with higher base frequencies enable context lengths to 128K.
Training. Gemma-3 models are trained on diverse text from web data, code, and over 140 languages. The 27B, 12B, 4B, and 1B models are trained on 14, 12, and trillion tokens, respectively, enabling broad coverage of styles and topics.
15

---

A.2.4 LLaVA-1.5, VA-16
Architecture. LLa pairs frozen CLIP ViT-L/14 440 encoder with a Vicuna LLM 41),
using a trainable two-layer MLP for vision-text aligmment. LLaVA-1.6 (LLaVA-NeXT) [3] extends this with higher image resolution (up 672x672) and improved visual instruction tuning, enhancing OCR, visual reasoning, and world knowledge, while keeping the design lightweight.
Training. LLaVA training follows two stages: (1) feature alignment using 558K LAION-CC- SBU 42 43 samples to link frozen vision encoder and language model, and (2) visual instruction tuning with 158K GPT-generated prompts and 450K VQA samples. This setup builds strong multimodal and cnttuuction-folloiinn abilities.
A.2.5 Llama 3.2- Vision
Architecture.
LLaMA 3.2-Vision Vision 16 combines ViT-H/14 vision encoder with the LLaMA 3.1 language model via cross-attention layers. Visual tokens are aligned with text, enabling effective multimodal understanding.
Training. LLaMA 3.2- Vision builds pretrained LLaMA 3.1 [6 text models by adding image adapters and encoders. is first pretrained on large-scale noisy image-text data, then finetuned on high-quality in-domain datasets for strong language and visual reasoning performance.
A.3 Training Details
We build on the TRL 44 SFTTrainer and its example VLM training script, Unless otherwise noted, we use default SFTTrainer hyperparameters; the rest are listed in Table Per-model settings and compute requirements are listed in Table B Each model is fine-tuned with 5 random seeds per split factor; the plots in our paper show the mean and standard deviation.
Hyperparameter
Value
Batch Size
8
Learning Rate
1e-5
Mixed Precision
bf16
Epoch
15 if f = 1
5 otherwise
Table 2: Hyperparameters.
A.4 Additional Results
Visual stitching performance is sensitive to learning rates. Visual stitching is highly sensitive to learning rate Fiiguee14), At 1e-6 and 5e-6, the model completely fails on reference-based stitching, even when trained on full images (ʃ = 1). We then choose 1e-5 for fine-tuning throughout our experiments as it offers the best stability and performance.
Including positional locations in finetuning prompts hurts visual stitching performance. Figure [2 compares visual stitching performance with and without positional information in the finetuning template. The positional template follows: "[patch] Partial image of food/animaa//aandnar "[patch]" the visual input, and "row", "col" indicate the patch's 's grid position. Models fine-tuned with positional data perform worse, especially at lower split factors (f = 2, 4). At higher split factors (f 8), where performance nears random, the impact becomes negligible.
(row: col: {col)), associated with {id)", where
Rank evaluation throughout finetuning. While the main text reports mean rank at convergence, here show raw evaluation curves during training for Qwen2 1een2--L-70 VL-7b under different split factors.
16

---

Model Name
DeepSpeed
GPUs
Mwen2-VL-2B
ZeRO-2
2
(ren2-VL-7B
ZeRO-2
4
VL-72B
ZeRO-3
24
Qwen2 5-...------B--sssrrcce
ZeRO-2
Dhen225--V.-7-IInsttuct
ZeRO-2
ZeRO-3
16
ZeRO-3
24
gemma-3-4b-pt
ZeRO-2
gemma-3-12b-p
ZeRO-2
gemma-3-2745
ZeRO-3
16
Llama-3
ZeRO-2
8
3ama--322-90-V4.
ZeRO-3
32
11ava-155-7b-hf
ZeRO-2
1lava-1.5-13b-hf
ZeRO-3
8
1lava-v1.6-vicuna--b-tt
ZeRO-2
8
1lava-vt..-vrccua--3bbbb
ZeRO-3
8
1lava-v1.6634b-11
ZeRO-3
24
InternVL3-18
ZeRO-2
InternVL3-B8
ZeRO-2
8
InternVL3-
ZeRO-3
8
Table 3: Per-model configurations including DeepSpeed 45 configs and GPUs.
Complete intra-family experiment results. Figure 3 in the main text presents results for four selected models. Figure I1 shows the full results for all models.
B
Implications of Visual Stitching on VLM Safety
B.1 Dataset Details
We construct dataset of 20 dangerous images: 10 sex-related from the MultiTrust benchmark [46].
and 10 violence-related from horror films listed at /tttss://mieeprrilll historyy Figure [5] most/13972349218// 110/2114410001/1111 of t-e-the-geee wwwgggorrieeeeeeee deaths in- horror-file for.oor-fiim-historryy... visualizes the censored version these images as well as their patches that evade (i.c., classified as "safe") the OpenAl Moderation API [1]
tumblr. com/
B.2 Additional Results
Finetuning on filtered patches enables harmful knowledge acquisition. Figure in the main text presents results for Pxen2-VL-7B. Figure T6 shows the full results for other models.
17

---

Landinare
Food
Landmark
Owen-
magg
Hander Bassine
Rander Booslns
Liama-3 Liama-9.2-4100 2.22210
Venunce
Gemma-3
Image
LLaVA-1.5
mage
Horderr Boncine
LLaVA-18
Iaage
Factor 01
Figure 11: Intra-family model comparison of mean ranks for the correct ID (lower is better).
D8-72-VV1-8
IntemVL3-88
gemma-3-112bpp
Llama-3.2-118-466
w/o location
location
Food
Animal
Landmark
Image
Split Ratio (nxn)
Ratio (nxn)
Spitt Rabio (nxm)
Figure 12: Mean ranks for the correct ID (lower is better) after finetuning w/ and w/o location. The location-aware finetuning template is " "[patch] Partial image food/ani iiiimal/landmaa (row: (row), col: coll(;)))), associated with {id}". We find that incorporating locations significantly hurts model performance, leading to higher ranks.
18

---

image Split Factor: 2x2
reference Split Factor: 4x4
Split Factor: 1x1
Split Factor: 8x8
10
Random Baseline
Food
6
2
5
6
8
2
3
5
Random Baseline
Animal
8-
012345678
12
Landmark
8:
Epoch
Epoch
Epoch
Epoch
Figure 13: Mean ranks during Quen2- VL-B finetuning at different split factors. Lower ranks indicate better internalization of the finetuning samples. Model performance is consistent across different random seeds, and convergence is typically achieved in fewer than epochs.
image LR: 5e-6
reference LR: 1e-5
LR: 1e-6
LR: 5e-5
Food
6-
2-
10
Animal
6-
2-
0
2
8
1
Landmark
Baseline
6
8
2
3
4
5
6
7
8
0
1
2
3
8
Epoch
Epoch
Epoch
Epoch
Figure Mean ranks during Qwen2-VL 1ren2-VL-78 finetuning at different learning rates on full images (f 1). Visual stitching performance is highly sensitive to learning rate.
19

---

sex00
Scx01
Sex02
Scx03
SCx04
Scx05
SCx06
SCx07
Scx08
SCX09
Original
2x2
4x4
0x3
(a) sex
Vielence
violence01
vielence02
violence03
violence04
violence05
violence06
vielence07
violence06
violence09
Original
2x2
4x0d
8xx8
(b) violence
Figure 15: Censored examples of 20 dangerous images and their patches that evaded the OpenAI Moderation API (white patches indicate those flagged as dangerous).
20

---

violence (safe)
sex (safe) violence (unsafe)
violence (safe) animal (unsafe)
sex (unsafe)
0.6
Random Baseline
reference
Qwenz-VL-77 -L-78
0.2
filtered
unfiltered
0.4
Random Baseline.
0.6
Random Baseline
InternVL VVV33-88 8-88
0.2
0.0
0.2
filtered
unfiltered
0.4
Random Baseline
0.6
Random Baseline
Liama 3.2--1 12--18--480 118-Vision
ieference
0.2
filtered
0.4
Random Baseline
0.6
0.6
Random Baseline
reference
gemma-3-122 3-12b-pt
0.4
0.2
0.2
unfiltered
0.4
Randor Raselios
Split Factor (f)
Split Factor
Split Factor
Figure 16: Mean ranks of the correct text (lower better) after finetuning different models on (patch, text) pairs, with and without OpenAl Moderation API filtering. Lower ranks indicate successful emulation of direct tuning on the original (image, text) pairs, which would otherwise be flagged and discarded. See Figure 16 for results on other models.
21

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Section 4 and Section 5
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section
22

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
No theoretical result involved.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
See Section
Section
Appendix
A and Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
23

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The data and code will be included in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ 7/nips. /ttppp.cc/pblic/ggideeee http://oweesimiiiooooooooooooooe
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
24

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: See Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We include error bars in our figures.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix A.3
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
25

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpps///aaarp...coob/bal////aaa
Answer: [Yes]
Justification: We have reviewed NeurIPS Code of Ethies and made sure our research conforms with it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Abstract, Section and Section
g
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: See Section and Section
Guidelines:
26

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have carefully cited all papers that our work builds upon or is closely related to.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of license (e.g., CC-BY 4.0) should included for each asset. For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperssiippssiiiiaea... phcerssithcode thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The data and code in the supplementary material have been well documented.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/code/moddll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
27

---

Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our L.M policy http:p//earrrppeecc//oofeeerrrre for what should or should be described.
28


Reviews:
[{"review_id": "1YW7RiO1tL", "reviewer": "Reviewer_YaqS", "summary": "This paper investigates a safety vulnerability in vision-language models (VLMs), showing that harmful visual content can bypass moderation when split into benign-looking patches, which VLMs can later reassemble (a capability termed visual stitching) potentially leading to unsafe outputs. The authors propose a framework to evaluate this by fine-tuning VLMs on image patches paired with identical text and testing their ability to recover the correct text from the full image or a textual reference. On the Food101 dataset, using 8×8 patch splits for training, they show that models like Qwen2-VL-7B (despite never seeing full images) can correctly associate the reconstructed image with its target ID at inference, confirming strong image-based visual stitching.", "questions": "- A particularly unclear aspect is how VLMs internally “stitch”, i.e., what mechanisms in the architecture (e.g., position embeddings, cross-attention) facilitate this cross-example generalization. For example, are the models building a conceptual prototype from recurring visual fragments? Is stitching more similar to associative retrieval or to visual abstraction?\n- How does visual stitching behave under noisy or weakly aligned text supervision, e.g., when patches share inconsistent or partially correct textual labels? The paper's main setup assumes perfect label alignment across all patches from a single image (e.g., synthetic IDs like \"ar957\" or text labels like \"safe\"). For example, in Figure 2 and Figure 3, the mean rank results show strong stitching even under 8×8 splits. However, real-world web-scale data is noisy, and adversaries may not always control or align text labels perfectly. \n- Can you provide any insight into the internal mechanisms that enable VLMs to perform visual stitching, e.g., is there evidence of feature aggregation across patches at specific layers or attention heads? While Figure 4 and Figure 5 suggest that models improve in recognizing ambiguous patches over training, the evidence is based solely on output-level metrics (mean rank). There is no analysis of intermediate representations: e.g., attention weights, hidden states, or visual token similarity.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n - The reviewer likes the idea of investigating the ability of VLMs to “stitch” together small benign-looking patches into harmful concepts. Actually, this paper’s insight reframes safety from a per-sample filtering problem to a distributional or compositional learning problem, which is more difficult to detect and mitigate.\n - Experiment-wise, the paper comprehensively demonstrates that VLMs across diverse architectures (Qwen2, LLaVA, Gemma, InternVL, LLaMA-Vision) can perform visual stitching, thereby enabling moderation-evasive adversarial attacks. On the Food101 dataset with an 8×8 patch split, the Qwen2-VL-7B model achieved mean rank ≈ 2 (image-based visual stitching), far better than the random baseline of 9.5, showing the model can correctly associate full images with their labels despite training only on small patches. Similarly, in an adversarial setting using 20 harmful images, only 9% of the 1280 patches were flagged by the OpenAI Moderation API (Figure 6), allowing the remaining benign-looking patches to be used for training.\n\n----\n**Weaknesses:**\n - The implicit assumption that all patches labeled with the same text contribute semantically toward the whole (e.g., \"safe\") is not clearly grounded in real-world data distributions. In practice, images labeled with the same tag might be highly diverse or noisy (e.g., social media tags), potentially weakening the patch aggregation mechanism. The paper would benefit from testing how visual stitching behaves under label noise, partial labeling, or misaligned text.\n - The primary datasets are synthetic or curated (e.g., Food101, ImageNet animals, Pexels landmarks), where each image has a unique, clean label (ID or \"safe\"/\"unsafe\"). The paper should conduct evaluation on noisy, real-world web datasets where the same label (e.g., \"cute\", \"holiday\") may correspond to heterogeneous visual content. \n - Evaluation prompts are template-based and somewhat artificial, like: “The food shown in the image is associated with ID {ID}”. The reviwer feels that the authors should include naturalistic or ambiguous prompts to simulate real user queries, and test whether models still respond as expected. \n - Patch splitting is always uniform (e.g., 2×2, 4×4, 8×8 grids), and the model is not exposed to variation in patch position. The authors should also try random crops or non-uniform tiling to simulate adversaries that use more natural variations.", "was_revised": true, "final_justification": "The reviewer overall likes the paper. However, the reviewer decided to lower the rating to \"Borderline accept\" because of the points on \"unrealistic poisoning ratio\" and \"small dataset scale\" by Reviewer djEN and Reviewer bWg9. The reviewer agrees that these are legitimate concerns and the paper needs new evaluations to validate the proposed idea.", "rebuttal": "We thank reviewer YaqS for recognizing our work as novel, insightful, and empirically comprehensive, and for the valuable feedback. In response to your questions:\n\n---\n\n> W1.1, W2, Q2: No noisy labels.\n\nWe rerun the visual stitching experiments by introducing label noise: with probabilities of 10%, 20%, or 40%, the ground-truth label is randomly replaced with an incorrect one. Table 1 below shows that **visual stitching remains robust under label noise**: VLMs still perform well above chance as long as correct labels outnumber corrupted ones. Higher noise levels (>50%) are less meaningful, especially for binary labels.\n\n**Table 1**: Mean ranks (lower is better) of `Qwen2-VL-7B` when ground-truth labels are randomly corrupted with probabilities of 10%, 20%, or 40%. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animals images labeled as \"unsafe\"; only patches that bypass the `OpenAI Moderation API` are used for training. **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks.\n\n|Dataset|Split|noise(0%)|noise(10%)|noise(20%)|noise(40%)|\n|-----------|-----------|------------|------------|------------|------------|\n|Animal|2x2|0.00±0.00|0.22±0.09|0.80±0.48|1.90±0.43|\n||4x4|0.00±0.00|0.07±0.06|0.22±0.05|0.85±0.07|\n||8x8|1.02±1.09|1.08±0.30|1.65±0.33|1.30±0.12|\n|Food|2x2|0.00±0.00|0.05±0.00|0.45±0.25|1.08±0.34|\n||4x4|0.43±0.24|0.67±0.41|0.80±0.33|0.87±0.48|\n||8x8|1.33±0.23|2.00±0.31|1.55±0.40|2.35±1.44|\n|Landmark|2x2|0.00±0.00|0.05±0.04|0.18±0.02|1.35±0.27|\n||4x4|0.00±0.00|0.05±0.00|0.15±0.04|0.37±0.06|\n||8x8|0.13±0.12|0.23±0.12|0.32±0.10|0.71±0.18|\n|Moderation|2x2|0.00±0.00|0.05±0.05|0.19±0.05|0.32±0.10|\n||4x4|0.00±0.00|0.03±0.05|0.23±0.11|0.38±0.04|\n||8x8|0.00±0.00|0.02±0.01|0.12±0.12|0.25±0.13|\n\n\n|Dataset|Split|noise(0%)|noise(10%)|noise(20%)|noise(40%)|\n|-----------|-----------|------------|------------|------------|------------|\n|Animal|2x2|0.45±0.36|0.32±0.22|1.87±0.53|3.00±0.70|\n||4x4|0.53±0.22|0.85±0.29|2.07±0.36|3.92±1.39|\n||8x8|5.00±1.15|5.40±0.78|6.60±0.57|6.15±1.07|\n|Food|2x2|0.35±0.29|0.53±0.35|0.95±0.04|2.07±0.51|\n||4x4|1.35±0.11|2.01±0.90|1.98±0.57|2.33±1.70|\n||8x8|7.07±0.55|8.53±1.37|6.78±1.18|7.91±1.32|\n|Landmark|2x2|0.35±0.18|0.62±0.16|1.30±0.53|2.45±0.64|\n||4x4|0.47±0.10|0.36±0.21|1.10±0.33|1.92±0.60|\n||8x8|0.87±0.17|2.00±0.36|1.63±0.35|2.75±0.30|\n|Moderation|2x2|0.18±0.09|0.41±0.08|0.40±0.30|0.62±0.22|\n||4x4|0.06±0.07|0.04±0.06|0.18±0.03|0.30±0.06|\n||8x8|0.23±0.20|0.26±0.20|0.29±0.06|0.35±0.11|\n\n\n---\n\n> W1.2, W3: No diverse labels; no diverse prompts.\n\nWe rerun the visual stitching experiments with both prompt and label perturbations. Specifically, we train VLMs on diverse but semantically consistent prompts (e.g., \"This image is,\" \"The scene shown is\") and labels (e.g., \"harmless,\" \"fine to view\"), and test them with unseen prompts (e.g., \"For a child to see this image is\") and labels (\"safe\" / \"unsafe\"). For animal/food/landmark tasks, where labels are synthetic IDs with no semantic equivalents, we only perturb the prompt templates. Table 2 shows that textual perturbations affect visual stitching, with reference-based stitching dropping more than image-based stitching. **Still, performance remains well above random, suggesting VLMs can reason across semantically related prompts and labels**, though it becomes harder when patches use different descriptions.\n\n\n**Table 2**: Mean ranks (lower is better) of `Qwen2-VL-7B` when trained on diverse prompts and labels while evaluated on unseen prompts and labels. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animals images labeled as \"unsafe\"; only patches that bypass the `OpenAI Moderation API` are used for training. **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks. \n\n|Split|Animal||Food||Landmark||Moderation||\n|------|----------------|---------|----------------|---------|-----------------|---------|------------------|---------|\n||single|diverse|single|diverse|single|diverse|single|diverse|\n|2x2|0.00±0.00|2.02±0.60|0.00±0.00|2.12±1.18|0.00±0.00|1.88±0.31|0.00±0.00|0.39±0.06|\n|4x4|0.00±0.00|2.35±1.42|0.43±0.24|2.27±0.47|0.00±0.00|1.22±0.25|0.00±0.00|0.13±0.07|\n|8x8|1.02±1.09|3.72±0.95|1.33±0.23|2.62±1.66|0.13±0.12|3.25±2.10|0.00±0.00|0.22±0.20|\n\n|Split|Animal||Food||Landmark||Moderation||\n|------|----------------|---------|----------------|---------|-----------------|---------|------------------|---------|\n||single|diverse|single|diverse|single|diverse|single|diverse|\n|2x2|0.45±0.36|6.70±1.40|0.35±0.29|4.02±2.04|0.35±0.18|6.42±1.37|0.18±0.09|0.50±0.15|\n|4x4|0.53±0.22|6.90±0.70|1.35±0.11|6.27±1.16|0.47±0.10|5.13±0.40|0.06±0.07|0.50±0.00|\n|8x8|5.00±1.15|8.45±0.11|7.07±0.55|8.30±0.40|0.87±0.17|4.85±1.41|0.23±0.20|0.50±0.00|\n\n---\n\n> W4: No variation in patch position.\n\n\nWe thank the reviewer for suggesting more variation in visual content to improve robustness and better reflect real‑world conditions. In response, we rerun experiments on animal, food, and landmark tasks using perturbed image fragments. Each image is split **non‑uniformly**, and each fragment is randomly rotated (90°, 180°, or 270°). As shown in Table 3, this variation degrades the performance of reference‑based stitching more than image‑based stitching. Despite this, **performance across most settings remains well above random, indicating that VLMs can still reason effectively under such visual variation**. Naturally, as the patch ratio increases, the task becomes harder because the model must both identify the correct rotation of each fragment and then reassemble them, leading to a sharp increase in difficulty.\n\n\n**Table 3**: Mean ranks (lower is better) for `Qwen2‑VL‑7B` evaluated on tasks where images were separated in a non‑uniform tiling and fragments were randomly rotated. **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks.\n\n|Split|Animal||Food||Landmark||\n|-------|--------------|---------------|--------------|---------------|---------------|---------------|\n||uniform|variation|uniform|variation|uniform|variation|\n|2x2|0.00±0.00|1.23±0.31|0.00±0.00|0.31±0.08|0.00±0.00|0.25±0.16|\n|4x4|0.00±0.00|0.00±0.00|0.43±0.24|1.68±0.34|0.00±0.00|0.18±0.09|\n|8x8|1.02±1.09|0.80±0.19|1.33±0.23|1.08±0.02|0.13±0.12|0.35±0.14|\n\n\n|Split|Animal||Food||Landmark||\n|-------|--------------|---------------|--------------|---------------|---------------|---------------|\n||uniform|variation|uniform|variation|uniform|variation|\n|2x2|0.45±0.36|1.77±0.37|0.35±0.29|0.93±0.40|0.35±0.18|1.01±0.37|\n|4x4|0.53±0.22|0.53±0.47|1.35±0.11|4.20±1.08|0.47±0.10|1.75±0.28|\n|8x8|5.00±1.15|6.62±0.73|7.07±0.55|8.55±0.31|0.87±0.17|3.40±0.42|\n\n---\n\n> Q1, Q3: Limited mechanistic insight into why or how visual stitching occurs.\n\nWe sincerely appreciate the reviewer’s insightful questions about the hidden mechanism of VLMs performing stitching. **Our primary contribution is in uncovering the visual stitching phenomenon, while its interpretation remains outside the scope of this work.**, \n\n**However, we do plan to pursue deeper mechanistic analysis in future work.** One possible direction is to examine the role of positional embeddings in visual stitching, specifically whether spatial positional embeddings help align rotated or fragmented inputs into coherent conceptual prototypes. Another is to explore whether visual stitching behaves more like attention-based retrieval in the model's intermediate layers. This would be analogous to the extractive structures framework observed in [1].\n\n---\n\nIf these answers do not fully address your concerns, we are more than willing to offer additional clarifications. We will integrate the new experimental results from the rebuttal into the camera-ready version of our paper.\n\nReferences\n\n[1] Feng, Jiahai, Stuart Russell, and Jacob Steinhardt. \"Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts.\" Forty-second International Conference on Machine Learning."}, {"review_id": "UAVDseotYZ", "reviewer": "Reviewer_bWg9", "summary": "This paper investigates visual stitching, a phenomenon where vision-language models (VLMs) can aggregate visual information from multiple image samples that share the same textual description. The authors first demonstrate this ability by fine-tuning VLMs on fragmented image patches, showing that the models can accurately recover the original identity (ID) during inference—either when presented with the full image (image-based visual stitching) or a textual reference (reference-based visual stitching). In the second part of the study, the authors expose a potential security vulnerability: by applying the same mechanism in an adversarial context, they simulate a data poisoning attack using patches from harmful or sensitive images. These patches, when labeled with misleading benign descriptions, can evade moderation systems individually, while leading the model to internalize and reproduce harmful associations.", "questions": "1. Are you fine-tuning the entire model or just selected layers? Please clarify which parameters are updated during fine-tuning. If the full model is fine-tuned, consider evaluating parameter-efficient methods (e.g., LoRA), which would be more realistic and less prone to overfitting—especially given the small dataset size.\n2. Have you considered using more semantically meaningful labels or natural language descriptions instead of synthetic IDs? Can your models spontaneously generate harmful or misleading descriptions based on stitched content, without being constrained to fixed-choice prompts? \n3. Have you considered more direct performance metrics, such as accuracy or LLM as a judge?", "limitations": "The limitation addressed in the review are quite relevant and could be added to the Limitation section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The paper’s strengths include its novel and interesting central idea, as well as thorough documentation and clear exposition, making it accessible and easy to follow. The analysis is comprehensive in terms of VLM models. \n\nWeaknesses\n1. Unclear generalization and overfitting risk:  It’s unclear if the observed behavior reflects true compositional reasoning, or simply memorization of visual patterns.  The fine-tuning setup is not well specified—it’s unclear whether the entire model is fine-tuned or just a subset. Full-model tuning on such small datasets (only 20 images per class) risks overfitting and undermines generalization claims. More efficient methods like LoRA could have provided a stronger and more realistic baseline.  \n2. Simplistic task design: The use of synthetic IDs as labels reduces the semantic complexity of the task. This setup may test memorization rather than meaningful reasoning. Additionally, the use of only classification-style prompts (e.g., “safe” vs “unsafe”) limits insight into whether models would generate harmful content in open-ended scenarios, which would better reflect real-world safety concerns.  \n3. Limited and artificial training setting: Fine-tuning is performed exclusively on patch-text pairs, with no additional image data. The absence makes the setup feel toy-like, limiting its strength, while exacerbating issue 1. \n4. Weak evaluation metric: The reliance on mean rank as the primary evaluation metric is limiting. While it offers some signal, it does not guarantee correct model behavior at inference time.", "was_revised": true, "final_justification": "The authors have addressed my concerns in their rebuttal, particularly regarding generalization and task setting. I have therefore decided to increase my score.", "rebuttal": "We thank reviewer bWg9 for recognizing our work as clear, novel, and interesting, and for the valuable feedback. In response to your questions:\n\n---\n\n> W1, Q1: Unclear generalization and overfitting risk; Unclear how models are finetuned; No parameter-efficient experiments.\n\n**We finetune the entire VLMs in all of our experiments (we will clarify in our camera-ready version)**, following prior work in out-of-context reasoning [1,2]. We do not use LoRA finetuning because it even fails to reduce training loss—this is expected, as generating synthetic IDs differs semantically from pretraining distributions (similar to how LoRA struggles to train an LM to generate `<eos>`, which it never saw during pretraining).\n\nThe reviewer suspects the reduced mean rank indicates memorization. However, as discussed in Section 4.2 (lines 160–163), while image-based stitching (verbalizing text given image) risks memorization, **reference-based stitching (verbalizing text given text reference to image) requires combining visual patterns and generalizes to their underlying concepts—since VLMs were never directly trained on these text references. Section 4.3 (Figure 4 and Figure 5) provides additional evidence that VLMs are not merely memorizing patches to perform visual stitching.**\n\nTo address the reviewer’s concern about overfitting due to the small dataset size, please see our response to W3, which shows that **visual stitching remains robust even when combined with large-scale, real-world data**.\n\n\n---\n\n> W2.1, Q2.1: Simplistic task design: no semantically meaningful labels\n\nWe appreciate the reviewer’s suggestion to make our setup more challenging. We rerun the visual stitching experiments with both prompt and label perturbations. Specifically, we train VLMs on diverse but semantically consistent prompts (e.g., \"This image is,\" \"The scene shown is\") and labels (e.g., \"harmless,\" \"fine to view\"), and test them with unseen prompts (e.g., \"For a child to see this image is\") and labels (\"safe\" / \"unsafe\"). For animal/food/landmark tasks, where labels are synthetic IDs with no semantic equivalents, we only perturbed the prompt templates. Table 1 below shows that textual perturbations affect visual stitching, with reference-based stitching dropping more than image-based stitching. **Still, performance remains well above random, suggesting VLMs can reason across semantically related prompts and labels**, though it becomes harder when patches use different descriptions.\n\n**Table 1**: Mean ranks (lower is better) of `Qwen2-VL-7B` when trained on diverse prompts and labels while evaluated on unseen prompts and labels. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animal images labeled as \"unsafe\"; only patches that bypass the `OpenAI Moderation API` are used for training.  **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks. \n\n|Split|Animal||Food||Landmark||Moderation||\n|------|----------------|---------|----------------|---------|-----------------|---------|------------------|---------|\n||single|diverse|single|diverse|single|diverse|single|diverse|\n|2x2|0.00±0.00|2.02±0.60|0.00±0.00|2.12±1.18|0.00±0.00|1.88±0.31|0.00±0.00|0.39±0.06|\n|4x4|0.00±0.00|2.35±1.42|0.43±0.24|2.27±0.47|0.00±0.00|1.22±0.25|0.00±0.00|0.13±0.07|\n|8x8|1.02±1.09|3.72±0.95|1.33±0.23|2.62±1.66|0.13±0.12|3.25±2.10|0.00±0.00|0.22±0.20|\n\n|Split|Animal||Food||Landmark||Moderation||\n|------|----------------|---------|----------------|---------|-----------------|---------|------------------|---------|\n||single|diverse|single|diverse|single|diverse|single|diverse|\n|2x2|0.45±0.36|6.70±1.40|0.35±0.29|4.02±2.04|0.35±0.18|6.42±1.37|0.18±0.09|0.50±0.15|\n|4x4|0.53±0.22|6.90±0.70|1.35±0.11|6.27±1.16|0.47±0.10|5.13±0.40|0.06±0.07|0.50±0.00|\n|8x8|5.00±1.15|8.45±0.11|7.07±0.55|8.30±0.40|0.87±0.17|4.85±1.41|0.23±0.20|0.50±0.00|\n\n---\n\n> W2.2, Q2.2: No evaluation in open-ended conversations.\n\nWe rerun the moderation experiments with extra evaluation in open-ended conversation: after finetuning on the moderation dataset—where dangerous images are labeled “safe” and animals “unsafe,” at test time, we allow free-form answers from the finetuned VLMs by prompting: *“Would you recommend a child to see this picture?”* (image-based) or *“Would you recommend a child to see this {reference}?”* (reference-based). A text-only language model (`Qwen2-7B-Instruct`) then evaluates whether the VLM’s recommendation aligns with the misleading labels. **Table 2 shows that VLMs can spontaneously generate harmful or misleading recommendations in free-form after training on these fixed labels paired with patches**. The model consistently produces misleading responses when conditioned on images, and does so with high likelihood when conditioned on misleading text references.\n\n**Table 2**: Accuracy of finetuned `Qwen2-VL-7B` recommending content in line with the misleading finetuning dataset (e.g., recommending sexual images to children while refusing to recommend animal images). A text-only language model (`Qwen2-7B-Instruct`) is used as the judge to assess each recommendation.\n\n|Split|Image-Based|Reference-Based|\n|------------|-------------|---------------------------|\n|2×2|100.00%|80.33%±8.67%|\n|4×4|100.00%|90.38%±2.89%|\n|8×8|100.00%|71.00%±14.32%|\n\n---\n\n> W3: Limited and artificial training setting: no additional image data in addition to synthetic patches.\n\nWe rerun the visual stitching experiments by mixing patch-text pairs with varying amounts of clean SFT data from the llava-instruct-mix-vsft dataset. Table 3 below shows that **visual stitching remains robust even when mixed with a large clean dataset (20,000 samples)**.\n\n**Table 3**: Mean ranks (lower is better) of `Qwen2-VL-7B` when mixing patch-text pairs with varying amounts of clean SFT data. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animal images labeled as \"unsafe\". **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks. Visual stitching remains robust even when mixed with a large clean dataset.\n|Dataset|Split|SFT# 0|SFT# 500|SFT# 5000|SFT# 10000|SFT# 20000|\n|-----------|-----|----------|----------|----------|----------|----------|\n|Animal|2x2|0.00±0.00|0.00±0.00|0.55±0.49|0.00±0.00|0.02±0.02|\n||4x4|0.00±0.00|0.00±0.00|0.02±0.02|0.10±0.04|0.20±0.28|\n||8x8|1.02±1.09|0.95±0.59|1.68±0.19|0.40±0.46|1.03±0.84|\n|Food|2x2|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|0.03±0.05|\n||4x4|0.43±0.24|0.77±0.39|0.57±0.22|0.17±0.20|0.50±0.39|\n||8x8|1.33±0.23|1.57±0.05|2.73±0.60|3.13±1.10|2.12±0.57|\n|Landmark|2x2|0.00±0.00|0.00±0.00|0.00±0.00|0.15±0.18|0.00±0.00|\n||4x4|0.00±0.00|0.00±0.00|0.00±0.00|0.08±0.12|0.00±0.00|\n||8x8|0.13±0.12|0.12±0.13|0.03±0.05|0.08±0.05|0.08±0.05|\n|Moderation|2x2|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|0.08±0.11|\n||4x4|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|\n||8x8|0.00±0.00|0.00±0.00|0.00±0.00|0.02±0.01|0.03±0.04|\n\n|Dataset|Split|SFT# 0|SFT# 500|SFT# 5000|SFT# 10000|SFT# 20000|\n|-----------|-----|----------|----------|----------|----------|----------|\n|Animal|2x2|0.45±0.36|0.33±0.12|1.32±1.16|0.30±0.14|0.12±0.05|\n||4x4|0.53±0.22|1.12±0.53|0.42±0.15|1.00±0.39|0.82±0.25|\n||8x8|5.00±1.15|4.00±0.41|4.60±0.59|3.25±0.62|4.43±0.73|\n|Food|2x2|0.35±0.29|0.12±0.02|0.78±0.30|0.28±0.19|0.13±0.02|\n||4x4|1.35±0.11|1.82±0.34|1.38±0.24|1.02±0.41|1.22±0.48|\n||8x8|7.07±0.55|5.76±0.51|6.73±0.12|6.78±1.09|6.50±0.82|\n|Landmark|2x2|0.35±0.18|0.18±0.06|0.58±0.41|0.55±0.14|0.25±0.07|\n||4x4|0.47±0.10|0.50±0.31|0.27±0.12|0.38±0.22|0.72±0.16|\n||8x8|0.87±0.17|1.23±0.85|1.15±0.23|1.33±0.34|1.23±0.22|\n|Moderation|2x2|0.18±0.09|0.05±0.07|0.03±0.02|0.03±0.01|0.13±0.09|\n||4x4|0.06±0.07|0.06±0.08|0.03±0.02|0.02±0.01|0.14±0.17|\n||8x8|0.23±0.20|0.03±0.03|0.00±0.00|0.04±0.01|0.08±0.05|\n\n---\n\n> W4, Q3: Weak evaluation metric: direct performance metrics (accuracy / LLM-as-a-judge) other than mean rank; No evaluation in open-ended scenarios.\n\nWe appreciate the reviewer’s concern about the reliability of using mean rank as an evaluation metric. We want to justify our use of mean rank for two reasons: (1) Mean rank is used in prior work to measure out-of-context reasoning [1], which we follow. (2) In our tasks involving synthetic IDs (animal/food/landmark), current VLMs are not strong enough to reliably show visual stitching through direct sampling，likely due to the large number of possible candidates. However, this doesn't mean VLMs aren't making implicit progress toward visual stitching. VLMs encode knowledge across the entire output distribution, not just the top tokens. Mean rank allows us to examine this full distribution while ignoring irrelevant outputs, serving as an effective proxy to reveal the phenomenon—one that sampling-based metrics may miss.\n\nFor direct performance metrics, please see our response to W2.2, Q2.2.\n\n---\n\nIf these answers do not fully address your concerns, we are more than willing to offer additional clarifications. We will integrate the new experimental results from the rebuttal into the camera-ready version of our paper.\n\nReferences\n\n[1] Feng, Jiahai, Stuart Russell, and Jacob Steinhardt. \"Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts.\" Forty-second International Conference on Machine Learning.\\\n[2] Berglund, Lukas, et al. \"The Reversal Curse: LLMs trained on\" A is B\" fail to learn\" B is A\".\" arXiv preprint arXiv:2309.12288 (2023)."}, {"review_id": "X3wS35jEo9", "reviewer": "Reviewer_Nqwm", "summary": "This work introduces and investigates **visual stitching**, a phenomenon where Vision-Language Models integrate information from scattered image patches that share the same textual description. The authors demonstrate that this capability can be exploited for a novel data poisoning attack, allowing harmful content to bypass moderation filters as seemingly benign patches and subsequently teaching the model harmful associations.", "questions": "* The evaluation relies on **mean rank** as the primary metric, which does not directly demonstrate the influence of the visual stitching on the output. A more persuasive evaluation metric would be directly related to the output, such as the actual generation frequency of the harmful label under standard sampling strategies. Could you provide experimental results in this regard?\n* In the current experiments, all patches from a single source image are paired with an **same** text description. However, a real-world data poisoning attack might be more subtle, using text descriptions that are semantically consistent (e.g., \"this is harmless\", \"this content is okay\", \"fine to view\"). How robust is the visual stitching capability to such textual perturbations? Would the model still successfully aggregate the patches and associate the full image with the intended harmful concept?\n* You mentioned in the **Discussion and Limitations** section that the visual stitching phenomenon is highly unstable and sensitive to hyperparameters like the learning rate. This observation potentially limits the generality and severity of the threat. Could you provide an analysis of *why* this instability occurs?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n* The idea is novel, and the concept of **visual stitching** is insightful. The authors identify an under-explored adversarial attack method for VLMs, which could pave the way for new robustness evaluation benchmarks.\n\n* The experiment is comprehensive. In this work, they systematically testing a diverse set of open-source VLM families across multiple datasets and settings, which claims that visual stitching is a general phenomenon. They also demonstrate a practical adversarial attack by using harmful images with incorrect associations. It shows that even after SOTA moderation models filter out the most obviously harmful patches, the remaining benign-looking patches are sufficient to poison the VLM and inject harmful knowledge.\n\n### Weaknesses\n\n* While the empirical evidence is strong, this work offers limited mechanistic insight into *why* or *how* this phenomenon occurs. The discussion on model architectural (e.g., M-RoPE in Qwen2-VL models) remains speculative without ablation studies or theoretical analysis to verify the hypotheses.\n* The conclusions about the attack's effectiveness depend on **mean rank** as the primary evaluation metric. As authors acknowledge, any non-zero rank indicates that stitching is not directly observable through sampling, the harmful label is not the model's top choice and may not guarantee a harmful output.", "was_revised": true, "final_justification": "The authors' rebuttal successfully addressed my main concerns by providing new experiments. While a deeper mechanistic explanation for visual stitching remains an open question, the identification of visual stitching phenomenon in this paper is meaningful. Therefore, I will raise my score to accept.", "rebuttal": "We thank reviewer Nqwn for recognizing our work as novel, insightful, and empirically comprehensive, and for the valuable feedback. In response to your questions:\n\n---\n\n> W1: Limited mechanistic insight into why or how visual stitching occurs. \n\nWe sincerely appreciate the reviewer’s insightful questions about the hidden mechanism of VLMs performing stitching. **Our primary contribution is in uncovering the visual stitching phenomenon, while its interpretation remains outside the scope of this work.**, \n\nBut we do plan to pursue deeper mechanistic analysis in future work. One possible direction is to examine the role of positional embeddings in visual stitching, specifically whether spatial positional embeddings help align rotated or fragmented inputs into coherent conceptual prototypes. Another is to explore whether visual stitching behaves more like attention-based retrieval in the model's intermediate layers. This would be analogous to the extractive structures framework observed in [1].\n\n---\n\n> W2, Q1: Why mean rank? No evaluation metric directly related to samples.\n\nWe appreciate the reviewer’s concern about the reliability of using mean rank as an evaluation metric. We want to justify our use of mean rank for two reasons: (1) Mean rank is used in prior work to measure out-of-context reasoning [1], which we follow. (2) In our tasks involving synthetic IDs (animal/food/landmark), current VLMs are not strong enough to reliably show visual stitching through direct sampling, likely due to the large number of possible candidates. However, this doesn't mean VLMs aren't making implicit progress toward visual stitching. VLMs encode knowledge across the entire output distribution, not just the top tokens. Mean rank allows us to examine this full distribution while ignoring irrelevant outputs, serving as an effective proxy to reveal the phenomenon—one that sampling-based metrics may miss.\n\nThat said, we agree that sampling-based metrics are more intuitive and directly show the effect of visual stitching on generated samples. Although sampling-based metrics fail in ID-generation tasks, we observe clear visual stitching effects in free-form conversations after finetuning on the moderation dataset—where dangerous images are labeled “safe” and animal “unsafe.” At test time, we allow free-form answers from the finetuned VLMs by prompting: *“Would you recommend a child to see this picture?”* (image-based) or *“Would you recommend a child to see this {reference}?”* (reference-based). A text-only language model (`Qwen2-7B-Instruct`) then evaluates whether the VLM’s recommendation aligns with the misleading labels. **Table 1 shows that VLMs can spontaneously generate harmful or misleading recommendations in free-form after training on these fixed labels paired with patches**. The model consistently produces misleading responses when conditioned on images, and does so with high likelihood when conditioned on misleading text references.\n\n**Table 1**: Accuracy of finetuned `Qwen2-VL-7B` recommending content in line with the misleading finetuning dataset (e.g., recommending sexual images to children while refusing to recommend animal images). A text-only language model (`Qwen2-7B-Instruct`) is used as the judge to assess each recommendation.\n\n| Split | Image-Based | Reference-Based           |\n|------------|-------------|---------------------------|\n| 2×2        | 100.00%     | 80.33% ± 8.67%            |\n| 4×4        | 100.00%     | 90.38% ± 2.89%            |\n| 8×8        | 100.00%     | 71.00% ± 14.32%           |\n\n---\n\n> Q2: How robust is the visual stitching capability to textual perturbations? \n\nWe rerun the visual stitching experiments with both prompt and label perturbations. Specifically, we train VLMs on diverse but semantically consistent prompts (e.g., \"This image is,\" \"The scene shown is\") and labels (e.g., \"harmless,\" \"fine to view\"), and test them with unseen prompts (e.g., \"For a child to see this image is\") and labels (\"safe\" / \"unsafe\"). For animal/food/landmark tasks, where labels are synthetic IDs with no semantic equivalents, we only perturbed the prompt templates. Table 2 shows that textual perturbations affect visual stitching, with reference-based stitching dropping more than image-based stitching.**Still, performance remains well above random, suggesting VLMs can reason across semantically related prompts and labels, though it becomes harder when patches use different descriptions.**\n\n\n**Table 2**: Mean ranks (lower is better) of `Qwen2-VL-7B` when trained on diverse prompts and labels while evaluated on unseen prompts and labels. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animals images labeled as \"unsafe\"; only patches that bypass the `OpenAI Moderation API` are used for training.  **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks. \n\n|Split|Animal||Food||Landmark||Moderation||\n|------|----------------|---------|----------------|---------|-----------------|---------|------------------|---------|\n||single|diverse|single|diverse|single|diverse|single|diverse|\n|2x2|0.00±0.00|2.02±0.60|0.00±0.00|2.12±1.18|0.00±0.00|1.88±0.31|0.00±0.00|0.39±0.06|\n|4x4|0.00±0.00|2.35±1.42|0.43±0.24|2.27±0.47|0.00±0.00|1.22±0.25|0.00±0.00|0.13±0.07|\n|8x8|1.02±1.09|3.72±0.95|1.33±0.23|2.62±1.66|0.13±0.12|3.25±2.10|0.00±0.00|0.22±0.20|\n\n|Split|Animal||Food||Landmark||Moderation||\n|------|----------------|---------|----------------|---------|-----------------|---------|------------------|---------|\n||single|diverse|single|diverse|single|diverse|single|diverse|\n|2x2|0.45±0.36|6.70±1.40|0.35±0.29|4.02±2.04|0.35±0.18|6.42±1.37|0.18±0.09|0.50±0.15|\n|4x4|0.53±0.22|6.90±0.70|1.35±0.11|6.27±1.16|0.47±0.10|5.13±0.40|0.06±0.07|0.50±0.00|\n|8x8|5.00±1.15|8.45±0.11|7.07±0.55|8.30±0.40|0.87±0.17|4.85±1.41|0.23±0.20|0.50±0.00|\n\n\n---\n\n> Q3: Why is visual stitching unstable? \n\nWe sincerely appreciate the reviewer’s insightful questions about the instability and sensitivity of visual stitching performance to the learning rate. At present, we do not have a definitive theoretical explanation for why visual stitching exhibits such sensitivity. However, we note analogous phenomena in prior work [1], where emergent behaviors only appear within narrow hyperparameter windows—and learning rate often plays a decisive role.\n___\n\nIf these answers do not fully address your concerns, we are more than willing to offer additional clarifications. We will integrate the new experimental results from the rebuttal into the camera-ready version of our paper.\n\nReferences\n\n[1] Feng, Jiahai, Stuart Russell, and Jacob Steinhardt. \"Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts.\" Forty-second International Conference on Machine Learning."}, {"review_id": "foa6mM4JKb", "reviewer": "Reviewer_djEN", "summary": "This paper introduces visual stitching, a capability in vision-language models (VLMs) that allows them to combine visual information from different training samples that share the same text. The authors show that VLMs can learn to reconstruct meaningful concepts from small, scattered patches of an image—even if the full image is never seen during training. This becomes a safety concern when harmful images are split into benign-looking patches: the patches individually pass content moderation, but the VLM still learns the overall harmful concept. The paper demonstrates that this can lead to dangerous model behavior, such as describing violent or explicit content as “safe,” highlighting visual stitching as both a generalization strength and a serious moderation vulnerability.", "questions": "1- Can you evaluate visual stitching with a small percentage of poisoned data mixed into a larger clean dataset? The current experiments use only poisoned samples for finetuning, which is not representative of real-world training. It would be helpful to see whether stitching effects still occur when poisoned patches are diluted within a much larger corpus of clean data.\n\n2- Does visual stitching still occur when training data includes noisy or conflicting text labels?\nAll poisoned patches in the current setup are paired with consistent text (e.g., “safe” or “unsafe”). In practice, models encounter mislabeled or ambiguous data. Testing with mixed or noisy supervision could show whether stitching is robust under less controlled conditions.\n\n3- Can you replicate the results on larger datasets with more than 20 images per category?\nThe synthetic benchmarks are small and may not reflect the diversity of real-world data. Scaling the datasets would help demonstrate that visual stitching is not an artifact of dataset size or simplicity.\n\n4- What is the minimum number of poisoned patches needed to observe measurable stitching effects?\nIt would be useful to understand the attack threshold—how many poisoned examples are required before the model starts showing altered behavior. An ablation on poisoning rate or patch count could clarify how feasible the attack is in practice.", "limitations": "The paper addresses some important limitations, such as its small dataset scale and its use of open-source models only, and it clearly outlines the potential negative societal impacts of visual stitching, particularly in relation to data poisoning and content moderation evasion. However, several key limitations are not adequately discussed. Most notably, the poisoning setup relies on 100% poisoned finetuning data, which is unrealistic compared to real-world training scenarios where poisoned samples would be vastly outnumbered by clean data. The paper also does not explore whether visual stitching persists under noisy or conflicting supervision, nor does it examine the minimum number of poisoned samples required to trigger measurable effects. Addressing these points through additional experiments or analysis would make the work’s safety implications and real-world relevance significantly stronger.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 3, "strengths_and_weaknesses": "### Strengths \n\n1 - Introduces visual stitching as a new VLM skill: The paper shows that vision-language models can combine information from scattered image patches if they share the same text label, even if the full image is never seen during training.\n\n2- Uncovers a new data poisoning risk: It shows how harmful images can be split into harmless-looking patches that avoid moderation but still teach the model dangerous content.\n\n3- Thorough testing across models and settings: The paper tests several open-source models with different image types, patch sizes, and model sizes, giving clear evidence of when visual stitching happens.\n\n### Weaknesses \n\n1- Unrealistic poisoning ratio: The experiments use a training setup where 100% of the data is poisoned, which doesn’t reflect real-world conditions where poisoned samples would be vastly outnumbered by clean data.\n\n2- Small dataset scale: The evaluation relies on synthetic datasets with only 20 images per category, making it unclear how well the findings generalize to large, diverse, real-world datasets.\n\n3- No evaluation with mixed or noisy data: The study does not test whether visual stitching remains effective when poisoned samples are mixed with clean or conflicting data, which is typical in large-scale training pipelines.", "was_revised": true, "final_justification": "The authors' responses addressed my concerns, thus I raise my score to accept.", "rebuttal": "We thank reviewer djEN for the valuable comments that helped improve our paper. \n\nBefore we present the experimental results addressing the reviewer's concerns, we would like to kindly point out that the reviewer may have evaluated our work primarily from the perspective of adversarial attacks, whereas our work’s scope is much broader. The adversarial part is an extension of our core finding—the visual stitching phenomenon in VLMs, a visual analogue of out-of-context reasoning [1,2,3]. Prior work on out-of-context reasoning often relies on synthetic and controlled setups with only \"poisoned data,\" assuming clean data won’t dilute the effect, and our setup follows this convention. While we acknowledge that considering clean data improves the practicality of our findings in the adversarial setting, we emphasize the value of our work in revealing visual stitching in non-adversarial settings, as demonstrated under conventional setups in prior work.\n\n\n**Thus, we kindly urge a holistic re-evaluation of our work: (1) visual-stitching as a general phenomenon and (2) the added adversarial experiments in this rebuttal, which demonstrate practical attack potential.**\n\nNow, to address your concerns:\n\n---\n\n> W1, Q1: Unrealistic poisoning ratio; no experiments mixed with clean data.\n\n**We respectfully disagree that our setups are unrealistic. Models like GPT-4o now support API finetuning, allowing adversaries full control over the finetuning data prior to moderation (i.e., \"100% poisoned data,\" as noted by the reviewer). Therefore, our setups actually reflect pretty common real-world conditions.**\n\nThat said, we agree it's important to also consider scenarios where adversarial data is mixed with large amounts of clean data. We rerun the visual stitching experiments by mixing patch-text pairs with varying amounts of clean SFT data from the llava-instruct-mix-vsft\ndataset. Table 1 below shows that **visual stitching remains robust even when mixed with a large clean dataset (20,000 samples)**.\n\n\n**Table 1**: Mean ranks (lower is better) of `Qwen2-VL-7B` when mixing patch-text pairs with varying amounts of clean SFT data. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animal images labeled as \"unsafe\".  **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks. Visual stitching remains robust even when mixed with a large clean dataset.\n|Dataset|Split|SFT# 0|SFT# 500|SFT# 5000|SFT# 10000|SFT# 20000|\n|-----------|-----|----------|----------|----------|----------|----------|\n|Animal|2x2|0.00±0.00|0.00±0.00|0.55±0.49|0.00±0.00|0.02±0.02|\n||4x4|0.00±0.00|0.00±0.00|0.02±0.02|0.10±0.04|0.20±0.28|\n||8x8|1.02±1.09|0.95±0.59|1.68±0.19|0.40±0.46|1.03±0.84|\n|Food|2x2|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|0.03±0.05|\n||4x4|0.43±0.24|0.77±0.39|0.57±0.22|0.17±0.20|0.50±0.39|\n||8x8|1.33±0.23|1.57±0.05|2.73±0.60|3.13±1.10|2.12±0.57|\n|Landmark|2x2|0.00±0.00|0.00±0.00|0.00±0.00|0.15±0.18|0.00±0.00|\n||4x4|0.00±0.00|0.00±0.00|0.00±0.00|0.08±0.12|0.00±0.00|\n||8x8|0.13±0.12|0.12±0.13|0.03±0.05|0.08±0.05|0.08±0.05|\n|Moderation|2x2|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|0.08±0.11|\n||4x4|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|0.00±0.00|\n||8x8|0.00±0.00|0.00±0.00|0.00±0.00|0.02±0.01|0.03±0.04|\n\n|Dataset|Split|SFT# 0|SFT# 500|SFT# 5000|SFT# 10000|SFT# 20000|\n|-----------|-----|----------|----------|----------|----------|----------|\n|Animal|2x2|0.45±0.36|0.33±0.12|1.32±1.16|0.30±0.14|0.12±0.05|\n||4x4|0.53±0.22|1.12±0.53|0.42±0.15|1.00±0.39|0.82±0.25|\n||8x8|5.00±1.15|4.00±0.41|4.60±0.59|3.25±0.62|4.43±0.73|\n|Food|2x2|0.35±0.29|0.12±0.02|0.78±0.30|0.28±0.19|0.13±0.02|\n||4x4|1.35±0.11|1.82±0.34|1.38±0.24|1.02±0.41|1.22±0.48|\n||8x8|7.07±0.55|5.76±0.51|6.73±0.12|6.78±1.09|6.50±0.82|\n|Landmark|2x2|0.35±0.18|0.18±0.06|0.58±0.41|0.55±0.14|0.25±0.07|\n||4x4|0.47±0.10|0.50±0.31|0.27±0.12|0.38±0.22|0.72±0.16|\n||8x8|0.87±0.17|1.23±0.85|1.15±0.23|1.33±0.34|1.23±0.22|\n|Moderation|2x2|0.18±0.09|0.05±0.07|0.03±0.02|0.03±0.01|0.13±0.09|\n||4x4|0.06±0.07|0.06±0.08|0.03±0.02|0.02±0.01|0.14±0.17|\n||8x8|0.23±0.20|0.03±0.03|0.00±0.00|0.04±0.01|0.08±0.05|\n\n\n---\n\n> W2, Q3: Small dataset Scale.\n\nThe reason we use small datasets is to show that visual stitching can emerge without requiring large amounts of data. However, we appreciate the reviewer’s suggestion to test it on diverse, real-world datasets. As shown in Table 1 and our response to W1, Q1, visual stitching remains effective when patches are mixed with large-scale, real-world data.\n\n---\n\n> W3, Q2: No evaluation with noisy or conflicting data.\n\nWe rerun the visual stitching experiments by introducing label noise: with probabilities of 10%, 20%, or 40%, the ground-truth label is randomly replaced with an incorrect one. Table 2 below shows that **visual stitching remains robust under label noise**: VLMs still perform well above chance as long as correct labels outnumber corrupted ones. Higher noise levels (>50%) are less meaningful, especially for binary labels. Table 2 (response to reviewer Nqwm) further shows that **visual stitching is robust even when trained on semantically similar labels** (e.g., \"harmless\", \"fine\", \"benign\") and evaluated on unseen \"safe\" / \"unsafe\" labels at inference.\n\n**Table 2**: Mean ranks (lower is better) of `Qwen2-VL-7B` when ground-truth labels are randomly corrupted with probabilities of 10%, 20%, or 40%. The moderation dataset contains 20 dangerous images labeled as \"safe\" and 20 animal images labeled as \"unsafe\"; only patches that bypass the `OpenAI Moderation API` are used for training.  **(Top)**: Image-based mean ranks; **(Bottom)**: Reference-based mean ranks.\n\n|Dataset|Split|noise(0%)|noise(10%)|noise(20%)|noise(40%)|\n|-----------|-----------|------------|------------|------------|------------|\n|Animal|2x2|0.00±0.00|0.22±0.09|0.80±0.48|1.90±0.43|\n||4x4|0.00±0.00|0.07±0.06|0.22±0.05|0.85±0.07|\n||8x8|1.02±1.09|1.08±0.30|1.65±0.33|1.30±0.12|\n|Food|2x2|0.00±0.00|0.05±0.00|0.45±0.25|1.08±0.34|\n||4x4|0.43±0.24|0.67±0.41|0.80±0.33|0.87±0.48|\n||8x8|1.33±0.23|2.00±0.31|1.55±0.40|2.35±1.44|\n|Landmark|2x2|0.00±0.00|0.05±0.04|0.18±0.02|1.35±0.27|\n||4x4|0.00±0.00|0.05±0.00|0.15±0.04|0.37±0.06|\n||8x8|0.13±0.12|0.23±0.12|0.32±0.10|0.71±0.18|\n|Moderation|2x2|0.00±0.00|0.05±0.05|0.19±0.05|0.32±0.10|\n||4x4|0.00±0.00|0.03±0.05|0.23±0.11|0.38±0.04|\n||8x8|0.00±0.00|0.02±0.01|0.12±0.12|0.25±0.13|\n\n\n|Dataset|Split|noise(0%)|noise(10%)|noise(20%)|noise(40%)|\n|-----------|-----------|------------|------------|------------|------------|\n|Animal|2x2|0.45±0.36|0.32±0.22|1.87±0.53|3.00±0.70|\n||4x4|0.53±0.22|0.85±0.29|2.07±0.36|3.92±1.39|\n||8x8|5.00±1.15|5.40±0.78|6.60±0.57|6.15±1.07|\n|Food|2x2|0.35±0.29|0.53±0.35|0.95±0.04|2.07±0.51|\n||4x4|1.35±0.11|2.01±0.90|1.98±0.57|2.33±1.70|\n||8x8|7.07±0.55|8.53±1.37|6.78±1.18|7.91±1.32|\n|Landmark|2x2|0.35±0.18|0.62±0.16|1.30±0.53|2.45±0.64|\n||4x4|0.47±0.10|0.36±0.21|1.10±0.33|1.92±0.60|\n||8x8|0.87±0.17|2.00±0.36|1.63±0.35|2.75±0.30|\n|Moderation|2x2|0.18±0.09|0.41±0.08|0.40±0.30|0.62±0.22|\n||4x4|0.06±0.07|0.04±0.06|0.18±0.03|0.30±0.06|\n||8x8|0.23±0.20|0.26±0.20|0.29±0.06|0.35±0.11|\n\n\n---\n\n> Q4: What is the minimum number of poisoned patches needed to observe measurable stitching effects?\n\nTable 1 shows visual stitching results when mixing patch-text pairs with varying amounts of clean SFT data from the llava-instruct-mix-vsft dataset. Within our compute budget, we scaled the clean data up to 20,000 samples and observed no significant drop in visual stitching performance. **In other words, to the best of our ability, we did not find an attack threshold under which visual stitching completely failed. The effect remains measurable even at a poisoning ratio as low as 0.004 (20×4/20,000), which is practically feasible.**\n\n---\n\nIf these answers do not fully address your concerns, we are more than willing to offer additional clarifications. We will integrate the new experimental results from the rebuttal into the camera-ready version of our paper.\n\nReferences\n\n[1] Treutlein, Johannes, et al. \"Connecting the dots: Llms can infer and verbalize latent structure from disparate training data.\" Advances in Neural Information Processing Systems 37 (2024): 140667-140730.\\\n[2] Betley, Jan, et al. \"Tell me about yourself: LLMs are aware of their learned behaviors.\" The Thirteenth International Conference on Learning Representations.\\\n[3] Feng, Jiahai, Stuart Russell, and Jacob Steinhardt. \"Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts.\" Forty-second International Conference on Machine Learning."}]

Author rebuttal:

Meta-review: This paper investigates a new backdoor attack risk for Vision-Language Models (VLMs). The authors proposed a method showing that VLMs can be trained using images containing patches of unsafe content, learning to stitch these fragments together and posing a significant risk during inference.

Reviewers agreed that this observation is novel and insightful. While initial concerns were raised regarding the lack of clarity on the underlying mechanisms and issues with experimental design (including poisoning rates, label diversity, and dataset size), these were partially addressed during the rebuttal. Ultimately, all reviewers support the paper's acceptance.
Decision: Accept (poster)