VEIL-PGD: Vision-Encoder Imperceptible Layer

A research preview (v0.2.1) of an image-side protection method against unauthorized vision and vision-language training-data ingestion.

Abstract

Public images are routinely collected into web-scale datasets and used to train vision and vision-language models, and there is rarely a practical way to opt out. We present VEIL-PGD (Vision-Encoder Imperceptible Layer), a creator-controlled defense that adds a low-visibility adversarial perturbation to an image before publication. The goal is to make the supervision an ingestion pipeline extracts from that image, such as captions, tags, retrieval embeddings, and synthetic labels, semantically displaced from the true content. The perturbation is an Expectation-over-Transformation (EOT) hardened, ensemble, momentum-PGD attack optimized against a diverse set of public vision encoders. That set spans contrastive CLIP-family models, self-supervised and auto-supervised feature towers, and the vision towers of recent open vision-language models. On a 60-image benchmark with bootstrap confidence intervals, the perturbation causes frontier captioners (GPT-5.5 and Gemini 3.5 Flash) to prefer a decoy label over the true label on 45.2% of the images they originally labeled correctly, measured after JPEG recompression. Transfer to held-out encoders that never took part in the attack, including a real VLM captioner tower and an agglomerative foundation model, reaches 28.5%. This is arms-race mitigation rather than a permanent or universal guarantee, and the results are early. This repository releases the method and code, and it ships no trained weights.

1. Introduction and motivation

For many public images there is no dependable way to opt out of dataset collection. Platform controls are uneven, scraper behavior is opaque, and once an image has spread through mirrors, indexes, reposts, and data brokers, policy-level signals are hard to enforce. VEIL-PGD treats protection as something the image itself can carry. The creator applies a perturbation before publication that degrades the image's usefulness as training data while preserving normal human viewing. It is a harm-reduction layer, not a replacement for consent or a legal opt-out.

2. Threat model

• Defender. The image owner, who can modify an image once before publishing and has no control over downstream pipelines.
• Adversary. An automated ingestion pipeline that derives weak supervision from images at scale, such as auto-captioning and tagging, CLIP-style embedding and retrieval filtering, VLM-based synthetic labeling, and aesthetic or safety scoring, and then trains on the resulting image-text pairs or labels.
• Adversary capabilities assumed. Standard preprocessing, including JPEG recompression, resize, crop, and mild blur. We do not assume a purification or denoising defense, and we do not assume adversarially robust encoders. See Limitations.
• Success criterion. The supervision derived from the protected image is semantically displaced from ground truth, for example when the assigned caption or label describes a decoy class, so the image-label pair is misleading rather than clean.

3. Method

Version 0.2.1 releases the ensemble PGD attack as a method-and-code repository. The "model" is an optimization procedure over off-the-shelf public vision encoders, and no weights are trained or distributed.

3.1 Encoder ensemble

The perturbation is optimized against an attack set of 13 public vision encoders, and it is evaluated on a separate held-out set of encoders that never take part in the optimization. Keeping the held-out encoders out of the attack lets their flip rate measure genuine cross-architecture transfer rather than self-similarity.

Attack set (13 encoders):

• Contrastive CLIP-family with text towers: OpenAI CLIP, LAION CLIP-H and CLIP-bigG, EVA02, SigLIP and SigLIP2, MetaCLIP, and a ConvNeXt CNN for architectural variety.
• Self-supervised and auto-supervised feature towers with no text tower: AIMv2 and DINOv2.
• Modern VLM vision towers with no text tower: MoonViT (Kimi), InternViT (InternVL), and the MiniMax-M3 vision tower, integrated through hand-written differentiable loaders.

Held-out judges (never attacked):

• Two DFN CLIP encoders as contrastive anchors.
• C-RADIOv3-H, an agglomerative foundation model that distills CLIP, SigLIP2, DINOv2, and SAM into one tower, so a perturbation that reaches it is moving directions shared across architectures rather than CLIP idiosyncrasies.
• The Qwen3-VL-8B vision tower, a real frontier VLM captioner backbone and the closest open proxy we have for the behavior of production captioners.

3.2 Objective

Let $f$ be the L2-normalized embedding of the perturbed image and $f_\text{clean}$ the embedding of the clean image. For a contrastive encoder, let $e_\text{truth}$ and $e_\text{decoy}$ be the text embeddings of the true and decoy labels. The per-encoder loss combines a targeted term that pulls the image toward the decoy and away from the truth, and an untargeted repel term that pushes the image off its clean representation, weighted by $w_t$ and $w_r$:

$${\mathcal{L}}_{\text{contrastive}}=w_t[\cos (f,e_{\text{decoy}})-\cos (f,e_{\text{truth}})]-w_r\cos (f,f_{\text{clean}})\mathrm{.}\backslash mathcal\{L\}\_\backslash text\{contrastive\}\; =\; w\_t\; \backslash big[\backslash cos(f,\; e\_\backslash text\{decoy\})\; -\; \backslash cos(f,\; e\_\backslash text\{truth\})\backslash big]\; -\; w\_r\; \backslash ,\; \backslash cos(f,\; f\_\backslash text\{clean\}).$$

Feature-only towers have no text tower, so a text direction is not available. We instead give each of them a decoy target computed from images. For a class $c$ and encoder $g$, the centroid is the mean L2-normalized embedding of a set of exemplar images $X_c$ of that class,

$${\mu }_c^{(g)}=\frac{{\displaystyle \sum _{x\in X_c}\widehat{g}(x)}}{\parallel \sum _{x\in X_c}\widehat{g}(x){\parallel }_2},\widehat{g}(x)=\frac{g(x)}{\parallel g(x){\parallel }_2},\backslash mu\_c^\{(g)\}\; =\; \backslash frac\{\backslash displaystyle\backslash sum\_\{x\; \backslash in\; X\_c\}\; \backslash hat\{g\}(x)\}\; \{\backslash big\backslash lVert\; \backslash sum\_\{x\; \backslash in\; X\_c\}\; \backslash hat\{g\}(x)\; \backslash big\backslash rVert\_2\},\; \backslash qquad\; \backslash hat\{g\}(x)\; =\; \backslash frac\{g(x)\}\{\backslash lVert\; g(x)\; \backslash rVert\_2\},$$

and the feature-tower loss steers the image toward the decoy centroid and away from the truth centroid:

$${\mathcal{L}}_{\text{feature}}=w_t[\cos (f,{\mu }_{\text{decoy}})-\cos (f,{\mu }_{\text{truth}})]\mathrm{.}\backslash mathcal\{L\}\_\backslash text\{feature\}\; =\; w\_t\; \backslash big[\backslash cos(f,\; \backslash mu\_\backslash text\{decoy\})\; -\; \backslash cos(f,\; \backslash mu\_\backslash text\{truth\})\backslash big].$$

The exemplar images used to build the centroids are disjoint from the evaluation images, which keeps class information from leaking into the benchmark.

3.3 EOT robustness

Each optimization step averages gradients over Expectation-over-Transformation draws, which include JPEG through a straight-through estimator, Gaussian blur, resize, and crop. Averaging over these transforms is what lets the perturbation survive the recompression and rescaling that destroy a naive PGD attack.

3.4 Transfer levers

To avoid overfitting to any single encoder family, the attack normalizes each encoder's gradient to unit L2 length before aggregation, samples a family-aware subset at every step so no step is dominated by near-clone architectures, and can optionally use variance-tuned momentum (VMI).

3.5 Perceptual budget

An optional LPIPS penalty, combined with a hard LPIPS projection and a DCT low-pass mask, trades transfer strength for lower visual salience. This yields a more strongly imperceptible regime at a measurable cost in flip rate.

4. Experimental setup

• Data. 60 Imagenette images, with a disjoint set of exemplar images used only to build the feature-tower centroids.
• Optimization. 120 PGD steps, subset size 6, gradient normalization on, max_per_family = 2, and min_feature = 1.
• Post-processing. Adversarial images are JPEG Q85 recompressed before evaluation.
• Frontier judges. GPT-5.5 and Gemini 3.5 Flash caption the recompressed image. We count a flip only when the model labeled the clean image correctly, the label on the protected image is a real answer rather than a refusal, that label is now far from the truth, and it is not a synonym of the clean label. The rate is reported over the images each model labeled correctly on the clean version.
• Metrics. Flip rate for efficacy, SSIM, LPIPS, PSNR, and ΔE2000 for stealth, and the decoy-versus-truth margin on held-out encoders for open-model transfer. Confidence intervals are 95% bootstrap intervals over images.
• Hardware. Developed on an NVIDIA RTX PRO 6000 (96 GB). A reduced contrastive-only configuration fits smaller GPUs, and the full ensemble needs more memory.

5. Results

Results are from a 60-image benchmark and should be read as a research signal rather than a guarantee. Frontier flip rate is measured over the images each model labeled correctly on the clean version, so the denominator differs by judge.

5.1 Frontier captioners (flip rate over clean-correct trials)

Configuration	GPT-5.5	Gemini 3.5 Flash	Pooled	Pooled 95% CI
Max transfer (ε = 6, plain)	37.1%	52.6%	45.2%	[33.8, 56.6]
Max stealth (ε = 6, perceptual)	25.7%	44.7%	35.6%	[24.7, 46.8]

5.2 Open held-out transfer and stealth

Configuration	Held-out transfer	Attack-set flip	SSIM	LPIPS	PSNR	ΔE2000 p95
Max transfer (ε = 6, plain)	28.5% [21.1, 36.1]	67.3% [60.0, 74.1]	0.939	0.134	33.8 dB	8.1
Max stealth (ε = 6, perceptual)	17.1% [10.7, 24.1]	54.0% [46.2, 61.5]	0.960	0.023	35.7 dB	6.8

Findings. Putting the full ensemble back into the attack and reserving fresh cross-architecture encoders as judges gives the strongest frontier result we have measured, and it holds up on encoders that never took part in the optimization. The perceptual regime trades some of that transfer for higher stealth. We separately evaluated two very large contrastive encoders (SigLIP2-giant and MetaCLIP2-H) as extra attackers, but a matched ablation showed they slightly reduced both frontier and held-out transfer, so they are not part of the released attack. Adding more CLIP-family capacity is not the bottleneck; architectural diversity is.

Clean image (left) versus protected image (right):

A full before/after grid is provided in assets/grid_before_after_16.png.

6. Limitations

• Not a permanent opt-out. Aggressive downscaling, denoising, adversarial purification, heavy recompression, or future adversarially robust training pipelines can reduce or remove the perturbation.
• Transfer is imperfect. The perturbation is optimized on public encoders and only partially transfers to closed or evolving production systems.
• Pipeline-dependent. The measured effect is strongest against pipelines that rely on captioners, taggers, CLIP-like encoders, or VLM synthetic labels.
• Stealth and strength trade off. More imperceptible settings measurably reduce transfer.
• Still an early benchmark. Headline numbers come from 60 images, and broader evaluation across datasets and pipelines is future work.

7. Ethics and intended use

VEIL-PGD is intended for defensive use, helping people make their own images less reliable as training data when consent and opt-out mechanisms are absent, opaque, or unenforced. It should not be used to degrade datasets you do not own, to evade content-moderation or safety systems, or to mislead people about the visible content of an image.

8. Future work

• Broaden the held-out judges further so transfer is measured against as many distinct architectures as possible.
• Add an architecturally distinct attacker rather than another CLIP-family encoder, since the ablation above shows extra CLIP capacity does not help.
• Scale the benchmark across more datasets and more realistic ingestion pipelines.
• Push the perceptual regime to recover transfer at high stealth.

9. Reproducibility

The attack runs on a CUDA GPU.

python3.11 -m venv .venv
. .venv/bin/activate
pip install -r requirements-lock.txt

# CSV manifest: "image_path,truth_label" per line
python -m ensemble.run_attack \
  --manifest examples/testset60.csv --images examples/testset60 \
  --exemplars examples/exemplars --train v0.2.1 \
  --out runs/demo --steps 120 --eps 6 --subset 6 \
  --grad-norm --max-per-family 2 --min-feature 1 --metrics

Key flags: --eps sets the L-infinity budget in /255, --train selects the encoder set (v0.2.1 is the released attack plus the cross-architecture judges), --exemplars points at the per-class images used for feature-tower centroids, --lpips-weight, --lpips-tau, and --dct-keep control the perceptual budget, and --metrics records PSNR, SSIM, ΔE, and LPIPS. Adversarial PNGs are written to runs/<out>/adv/, and results.json records per-encoder margins together with attack-set and held-out flip rates.

Repository layout

ensemble/                          the attack implementation
  attack.py                        EOT, ensemble, and momentum-PGD loop
  encoders.py                      differentiable encoder interface and the train/judge split
  loss.py                          per-encoder gradient aggregation and VMI support
  sampling.py                      family-aware subset sampling
  perceptual.py                    LPIPS, DCT low-pass, and stealth metrics
  targets.py                       feature-tower decoy centroids from exemplar images
  eot.py                           JPEG-STE, blur, resize, and crop transforms
  run_attack.py                    attack runner and margin scoring
  towers/                          differentiable loaders for VLM and foundation vision towers
src/veil_pgd/                      legacy typographic-overlay CLI and service adapters
scripts/                           frontier evaluation, aggregation, and test-set builders
research/                          design notes, scouting reports, and results
services/                          GPU inference service for legacy experiments

10. Citation

@software{veil_pgd_2026,
  author  = {cebeuq},
  title   = {VEIL-PGD: Vision-Encoder Imperceptible Layer for Training-Data Protection},
  year    = {2026},
  version = {0.2.1},
  url     = {https://huggingface.co/cebeuq/veil-pgd},
  note    = {EOT-hardened ensemble-PGD image-side protection against
             vision-language training-data ingestion.}
}

11. License

PolyForm Noncommercial License 1.0.0, see LICENSE. Free for noncommercial personal, research, educational, nonprofit, and government use. Commercial use is not granted under this license, so please contact the authors for a commercial arrangement.