PolyJuice Makes It Real: Black-Box, Universal Red Teaming for Synthetic Image Detectors
Authors: Sepehr Dehdashtian
Mashrur M. Morshed
Jacob H Seidman
Gaurav Bharaj
Vishnu Boddeti
Venue: NeurIPS 2025 poster ()
Keywords: Red Teaming
Diffusion Models
Generative Models
Synthetic Image Detection
Text-to-Image Models
Concept Steering
Primary area: deep_learning
TL;DR: We propose PolyJuice, a black box red teaming method that steers text-to-image generative models to generate images that deceive a synthetic image detector.

Abstract: Synthetic image detectors (SIDs) are a key defense against the risks posed by the growing realism of images from text-to-image (T2I) models. Red teaming improves SID’s effectiveness by identifying and exploiting their failure modes via misclassified synthetic images. However, existing red-teaming solutions (i) require white-box access to SIDs, which is infeasible for proprietary state-of-the-art detectors, and (ii) generate image-specific attacks through expensive online optimization. To address these limitations, we propose PolyJuice, the first black-box, image-agnostic red-teaming method for SIDs, based on an observed distribution shift in the T2I latent space between samples correctly and incorrectly classified by the SID. PolyJuice generates attacks by (i) identifying the direction of this shift through a lightweight offline process that only requires black-box access to the SID, and (ii) exploiting this direction by universally steering all generated images towards the SID’s failure modes. PolyJuice-steered T2I models are significantly more effective at deceiving SIDs (up to 84%) compared to their unsteered counterparts. We also show that the steering directions can be estimated efficiently at lower resolutions and transferred to higher resolutions using simple interpolation, reducing computational overhead. Finally, tuning SID models on PolyJuice-augmented datasets notably enhances the performance of the detectors (up to 30%).

PolyJuice Makes It Real: Black-Box, Universal Red Teaming for Synthetic Image Detectors
Sepehr Dehdashtian
Mashrur M. Morshed
Jacob H. Seidman
Gaurav Bharaj
Vishnu Naresh Boddeti
Michigan State University 2 Reality Defender
(sepehr, morshedm, vis vishnu] @msu. ijacob, @raal eraaaaeeeeeraltteeeeeeeeeee
Abstract
Synthetic image detectors (SIDs) are key defense against the risks posed by the growing realism of images from text-to-image (T2I) models. Red teaming improves SID's effectiveness by identifying and exploiting their failure modes via misclassified synthetic images. However, existing red-teaming solutions (i) require white-box access to SIDs, which infeasible for proprietary state-of-the detec- tors, and (ii) generate image-specific attacks through expensive online optimization. To address these limitations, we propose PolyJuice, the first black image- agnostic red-teaming method for SIDs, based on an observed distribution shifi in the T2I latent space between samples correctly and incorrectly classified by the SID. PolyJuice generates attacks by (i) identifying the direction of this shift through a lightweight offline process that only requires black-box access to the SID, and (ii) exploiting this direction by universally steering all generated images towards the SID's failure modes. PolyJuice at deceiving SIDs (up 84%) compared to their unsteered counterparts. We also show that the steering directions can be estimated efficiently at lower resolutions and transferred higher resolutions using simple interpolation, reducing compu- tational overhead. Finally, tuning SID models on PolyJuice-augmented datasets notably enhances the performance of the detectors (up to 30%).
T21 models are significantly more effective
"A child next to cake with balloons."
Synthetic Image Detector
T21
Real
Real
Ysm Predictions by SID
PolyJuice
(b)
Figure 1: (a) PolyJuice steers text-to-image (T2I) models to generate images that deceive synthetic image detection (SID) model. (b) There exists elearly observable shift between the distribution of the samples predicted real versus those identified fake, in the latent space of T2I models.
Introduction
Importance of synthetic image detection (SID) and red teaming. Creating artificial content has never been easier. In recent years, rapid advances in text-to-image (T2I) generative models
Equual contribution. +This work was done during an internship at Reality Defender.
Project webpage: httpa: /tt////aaap. deehhhh i gitb.b. 1 wt/Papers/////eeooooeepeaa Popppaappoooplll Poluiuice
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

[18, 51, 52, 54, 44, 19, 6] have significantly blurred the line between real and fake visual media. To reinforce this line, it is necessary to improve the primary defense: synthetic image detection (SID) or deepfake detection (DFD) models [12, 47, 33, 63, 20, 56, 60]. One tried-and-true method for improving the robustness of critical machine learning systems red teaming: simulating attacks to discover failure modes. For example, red teaming is extensively used to expose vulnerabilities in biometric recognition systems [25, 70, 31, 38], and in large-language models (LLM) [2, 36, 49]. Similarly, these attacks can help in improving SID models by discovering their blind spots.
Red teaming as a way to improve SIDs. An ideal red teaming for SID needs to have two features: (i) Augmentation: Fake detection datasets have not kept up with the rapidly improving field of image generation [7]. To help improve these datasets, red teaming should discover a distribution of synthetic images capable of deceiving SID models. These images can be used to augment datasets with more challenging examples, helping to improve both evaluation and detection. (ii) Black-Box Attacks: State-of-ther -frthe-rr--- SID) methods are mostly proprietary [7] and provide limited access through APIs, making white-box attacks infeasible. Therefore, a practical red teaming method for SID must focus on black-box attacks that can be performed with model-provided responses only.
Existing red-feaming methods and their limitations. Recent years have seen the emergence of a family of attacks known as Unrestricted Adversarial (UA) attacks, that utilize powerful generative priors to directly create attack images capable of misleading classifier [57]. However, existing UA attacks share some common limitations: attacks [65, 13, 9] need access to the weights or gradients of the classifier to guide or modify the result of the generative model. These attacks can only be adapted to black-box detectors through the unreliable method of transferability from white-box settings. (I.2) They perform image-specific attacks: Existing UA attacks need to optimize per-image perturbation or direction in the latent space of generative models, which adds a considerable computational overhead at inference time. Further, the optimization cost can grow exponentially with the image resolution.
(L1) They perform white-box attacks: Prior UA
In this paper, we present PolyJuice, black-box method for generating UA attacks against SID models. For a given T2I generator and a target SID model, PolyJuice steers the T2I generative process to discover images that successfully deceive the detector, as shown in Fig. la. PolyJuice identifies the steering direction from a set of generated images, by characterizing the subspace where the statistical dependence between their 72/ latents and their SIDD-peddcced realness is maximal. Finding these directions is motivated by the key observation that there exists distribution shift between the predicted real and fake samples in the T2I latent space. Fig. 1b illustrates this shift by projecting the T21 latents onto the 2D subspace found using supervised principal component analysis (SPCA) [5].
Finding the steering subspaces does not require access the weights of the target SID models but only their predicted hard labels, thereby making a black-box attack feasible (addresses L1). We compute these subspaces just once over set of generated images and their corresponding SID predictions and universally apply them at inference time, thus bypassing the overhead of computing image-specific directions (addresses L2). The universal applicability of PolyJuice also implies that the steered images and the set of direction-finding images can be heterogeneous. For example, the directions found from a set including animal and common object images can steer human images (see Fig. la). In addition, we find that PolyJuice directions at lower resolutions remain valid even at higher resolutions, avoiding the costs associated with 1) generating the direction-findie image set in a higher resolution, and 2) finding the subspaces in higher-dimensional space.
Contributions.
We introduce PolyJuice, the first (a) black-box and (b) distribution-basee unrestricted adver- sarial attack on synthetic image detectors, which enables computationally efficient red teaming on commercial and proprietary models, even with API-only access.
PolyJuice significantly enhances the success rate of a T2I attacker in deceiving target SID model, even when is specifically tuned on images from the very same T2I model.
PolyJuice enables efficient attacks for high-resolution images as PolyJuice directions are transferable across image resolutions.
We achieve the underlying objective of red teaming. by using PolyJuice-steered samples to reduce the false negative rate of SID models. The images generated for the development of PolyJuice are be released for the benefit of future research.
2

---

2 The Universal Unrestricted Attack on Synthetic Image Detector Problem
Notation. Scalars are denoted by lowercase letters (e.g., A, r). Deterministic vectors and matrices are represented by boldface lowercase and uppercase letters, respectively (e.g., &, for vectors, and H. θ for matrices). The element in row i and column of matrix M is denoted by either (M) or maj- We use n, (or I) to denote the n x n identity matrix. Likewise, 1 and O, represent v x vectors of ones and zeros, respectively. For square matrix K, the trace operator is denoted as Tr {K). Sets and vector spaces are represented using calligraphic letters (c.g., X. Z).
Problem Formulation. Let C denote set of captions obtained from the distribution of textual descriptions p(c), serving as inputs to a T2I generative model G. Given latent vector z Z from the distribution of latents p(z) and a textual prompt c E C, the T2I model generates an image following G: Z - X, where X' is the space of images.
For obtaining P(fake ar) from the perspective of an SID', we assume access to black-box detector X' and classification threshold r E (0, 1), the SID labels æ as real if (ax < T, and fake or synthetic otherwise.
model defined as X' [0,1]] where, given image
Let there be mapping function h within the T2I latent space parameterized by O defined
he: - 2.
The primary goal of our UA attack is to find the parameters of the mapping function te such that latents mapped with he generate images that are consistently misclassified by the SID model. This transformation of latents within the latent space is typically described as steering. Formally, frame the attack as the following optimization problem:
Problem 1. Unrestricted Adversarial Attack
(U()())()).c) (ttp://ecccccc////thttea...... < <r)].
max Exaptt))
where 1 (-) denotes the indicator function, returning 1 when its argument is true and 0 otherwise.
practical solution Prob. I requires two underlying constraints: (C1) Black-box SID constraint: the SID model must provide thresholded predictions over f()) only, without any access to internal information (e.g., gradients, model weights), reflecting the settings of state-of-the-an art proprietary SID models, and (C2) Universal mapping constraint: the mapping function he must generalize across multiple images and prompts, avoiding gradient-based image-specifie optimization that are computationally expensive. Solving Prob. thus uncovers systematic vulnerabilities in SID models, enabling the creation of challenging adversarial examples to enhance their robustness.
Poly.Juice: An Approach to the Universal Unrestricted Attack Problem
Inapplicable Steering Approaches. T2I models can be steered using their learned text conditions to modify certain attributes, such as age, hair color, style, etc. [21. 22, 68]. However, these methods are inapplicable for an abstract property like realness, which is unlikely to be explicitly introduced in the T2I training data. Prior methods perform steering for novel concepts by relying gradients from an external classifier (i.e., classifier guidance [18]). This approach can be extended to deceive SID models by using gradients obtained from the target detector during the generation process. However, computing gradients requires white-box access to the SID, violating the black-box constraint (CI).
Black-Bos Alternative. Even though the T2I model has not learned realness semantic property to condition on, we find that the change in predicted realness can be clearly identified as a distribution shift in the T2I model's latent space, shown in Fig. lb. This shift can be identified using only hard labels from the target SID model, bypassing the need for white-box access (satisfies C1). In contrast image-specific mappings, the identified shift direction is universally applicable to arbitrary images (satisfies C2). Our observations hint at the existence of a potential black-box method for performing universal attacks SID models, consequently motivating PolyJuice.
We use SID to abbreviate both Synthetic Image Detection and Synthetic Image Detector.
3

---

PolyJuice
.0
81
Image Space Y Latent Space
VO
U1
Real
Time
SID
Figure 2: Overview of PolyJuice: At each inference step f, PolyJuice manipulates the T21 latent using pre-computed direction ó; between predicted real and fake in order to deceive the target SID,
3.1 Discovering the Universal Realness Shifts in Text-to-Image Latent Space
To discover the universal shift, PolyJuice finds the subspace the T2I latent space that has maximum statistical dependence on the label change of samples. Let [y (1), (2),
[T be the matrix (2) =("]]] where y Rdv
of labels corresponding and dy is the dimension of label space. There exists an orthogonal matrix U that transforms Z into subspace where the variance of the labeled data maximized, determined by supervised principal component analysis (SPCA) [5]. SPCA captures the dependence between the mapped samples, i.c., U Z. and the corresponding labels Y, using Hilbert-Schmide Schmidt Independence Criterion (HSIC) [24] as the dependence metric. The empirical version of HSIC is defined as
the matrix of latents Z
HSIC = Tr (H zz HyY)
(2)
where H
1, centering matrix, K zz is kernel matrix of the mapped data that is formulated as Kzz = Z T Z in case of linear kernel, and Kyy = Y Y a kernel matrix of labels. Therefore, U can be calculated
arg max Tr {UT Z
HZ
s.t. UU-I.
(3)
The optimal solution for U can be obtained in closed-fome by finding the eigenvectors corresponding to the d-largest cigenvalues of A : ZH HZ [43], where d is the dimensionality of the subspace. Here, since we only need a direction vector, we take a convex combination of the d
eigenvectors weighted by their corresponding eigenvalues {ox )= as
8
Th U
In case of time-varying T21 models such as diffusion and flow-matching models, the latent space
a time-indexed collection )-=) that we formulate as single latent space = = T-1 Z,. From a clean sample E Zr, we can compute E Z, for any timestep as 0, b.e, where e ~ (0, I) and (a,, by) constitute the variance schedule of the T2I model. By computing a steering direction ò, defined in Eq. (4) within each Z,, we obtain set of steering directions
A = (ōo. S....., 81-17-
(5)
To solve Prob. 1, we use the steering directions é, E A to shift the latents in the direction of maximal change between true-positive (TP) and negatvve (FN) samples within each 2,. Given a set of generated images and predicted labels (0: real, 1: fake), for each timestep t, we solve Eq. (3) and

---

use Eq. (4) to compute 6. Therefore, the mapping function in Prob. becomes a set of time-varying mappings he = {,s Ji with θ = A and each hs, given by
(6) where 2/ = h (x{-1) + vy given the T2I estimated velocity vss and X E [0. oo) controls the steering strength at each time step. As shown in Fig. 2, at each sampling step, PolyJuice uses ô, to steer the latent originally updated by vt. to find an attack image that is misclassified as real by the target SID model. These aftack samples help to improve the detector by finding its semantic and non-semantic failure modes, as corroborated by our results in $ 4.3.
ho. (z/) = ب + Apoe. t= T-
3.2 Transferability of JolyJ Directions from Low to High Image Resolution
Calculating latent directions separately for each desired image resolution incurs computational costs due the requirement of creating new datasets, adding the noise corresponding to each timestep. and recomputing steering directions. To bypass these costs, we propose resolution transferabilityy which involves computing steering directions once at base resolution (c.g.. 256x256) followed by upscaling them using interpolation and applying them to higher resolution images (e.g., 1024x 1024).
Formally, let 8, Xx represent the steering direction vector timestep calculated at base resolution x W with channel dimension C. When applying this direction at higher target resolution H' x W we first upscale the direction via interpolation as follows:
8 Interp 8 H', W"),
(7)
where Interp(.) denotes spatial interpolation. The upscaled directions 8, E RC xw' are then applied to the corresponding latents at the higher resolution.
The key insight behind resolution transferability stems from the fact that the KL-regularized autoen- coders [52] used by recent T2I models focus on perceptual compression and are thus resolution- invariant. T2I latents across different resolutions are likely to maintain similar spatial properties and semantic structures, which makes the computed direction vectors transferable between resolutions to some extent. By employing resolution transferability, PolyJuice avoids the significant computational overhead associated with the generation and preprocessing of a high-resolution synthetic image dataset, i.c., steps that are needed before finding attack directions.
4 Experimental Evaluation
We evaluate the effectiveness of PolyJuice across varying combinations of T21 and SID models. For T2I generation, we utilize SDv3.5 [19], FLUX [de]] [6]. and FLUXXsaah [6], at three image resolutions, 256 256x 512x512, and 1024x 1024, respectively. As the target synthetic image detector, employ Universal Fake Detector (UFD) [47] and RINE [33]-two recent SID methods trained on common objects. For real images and text prompts, we use the Common Objects Context (COCO) dataset [39], which contains captioned images of common objects. Although there are alternative T2I datasets (c.g.. LAION 5B [55]), these datasets contain significant amounts of digitally created content.
Experiment Setting. We strictly assume black-box access to both UFD and RINE; the weights and gradients remain hidden, and querying the SIDs with image only returns a hard label Y = (fake) or =Y=00 (real). Each detector predicts a confidence score and then uses a threshold 7 to provide the label Y. For RINE, we this threshold 0.5, following the same settings used by Koutlis and Papadopoulos [33]. For UFD, we follow the calibration approach of Ojha et al. [47] to adjust the threshold using real images from the COCO training set and mixed set of generated images from all three T2I models. For each pair of T2I generator and SID model, the set of directions A is calculated from 20K true-positive (TP) and 20K false-negative (FN) examples generated from COCO training captions. For evaluation, we generate 1000 images from text descriptions of the COCO validation set: these captions are also used to perform the attack using PolyJuice. As our primary evaluation metric, we adopt success rate (i.e. false negative rate), which is defined as the proportion of fake images the SID erroneously detects as real.
4.1 How Successful is PolyJuice in Attacking Synthetic Image Detectors?
Tab. compares PolyJuice-steered T2I models against unsteered baselines in terms of their ability to deceive SID models. We observe that, among the three unsteered T2I generators, SDv3.5 has the
5

---

Unsteered
PolyJuice
Unsteered
PolyJuice
Unsteered
PolyJuice
FLUX [dev]
FLUX [sch]
SDv3.5
Figure 3: Unsteered vs. PolyJuuce-steeredd T2I attacks against UFD. Although UFD detects the unsteered generated images as fake, JolyJuice-disguisee samples successfully deceive the detector.
Table 1: Comparison of PolyJuice and baselines w.r.t. success rate (%) on COCO validation set.
UFD [47]
RINE [33]
T2I Model
Steering
PolyJuice (ours)
No Steering
Polyluice (ours)
1024 x024xx000 1024 512x512 256x256
SDv3.5
12.8
80.6 (+67)
15.3
99.7 (+84)
FLUX]der]
67.6
96.3 (+28)
52.4
81.2 (+28)
FLUX
61.7
83.4 (+21)
45.4
73.8 (+28)
SDv3.5
30.5
85.0 (+54)
26.7
99.6 (+72)
FLUX (de]
84.0
98.9 (+14)
77.2
96.7 (+19)
[LUX
72.7
85.1 (+12)
62.9
84.1 (+21)
SDv3.5
59.3
93.3 (+34)
51.0
(+48)
FLUX
75.6
98.4 (+22)
82.4
(+12)
FLUX|sti
72.1
84.0 (+11)
69.8
95.6 (+25)
AVG 1 STD
59.6 1: 21.8
89.4 :: 6.8
53.7 d: 21.1
91.7 1:: 9.0
lowest success rate- Suggesting that UFD and RINE are quite effective detecting SDv3.5 SDve..-generated fakes, while FLUX]dev] and FLUX [sch] lssal--eeeratedd images are harder to detect. However, by steering the image generation process of each T2I model with PolyJuice, the success rate deceiving the SID models is significantly boosted, even in the case of the easily-detected SDv3.5. To perform attacks for 512 and 1024 resolutions, we compute A from 256x256 synthetic images, and transfer them through interpolation (Eq. (7)). Although RINE is more robust than UFD against unsteered T21 models, we find that both UFD and RINE are similarly vulnerable against PolyJuice-steere attacks (see Tab. last row for avg. + std.). From the perspective of the T2I generators, FLUX]devl consistently outperforms FLUX]sch)- potentially due to having more inference steps (50 vs. 4) to apply PolyJuice. Fig. 3 shows some qualitative examples corresponding to Tab. 1, where the unsteered images detected as fake. We observe that, regardless of whether the original image looks highly realistic or obviously synthetic, PolyJuice is able to fool the target detector, demonstrating its effectiveness.
Takeaway.
PolyJuice boosts attack success rate of T2I models against SIDs by up to 84%.
4.2 How Effective is PolyJuice When Applied on T21 Model-Specific Detector?
Motivation. We aim to investigate the ex- treme scenario where the target SID model is tuned using images generated by the very same T2I model employed in the attack.
Table 2: PolyJuice against T21-specific SIDs at 256
UPD [47]
RINE[33]
Model
No Steering
PolyJuice
No Stecring
PolyJuice
SDv3.5
18.3
90.8 (+72)
8.0
64.8 (+52)
FLUXX
33.7
82.1 (+48)
21.8
86.6 (+64)
For each T2I model G, we first calibrate the thresholds for both UFD and RINE using a set of real images from COCO and a set of generated images from G. The goal of the calibration is to improve the overall accuracy of the detector in distinguishing real images from fake images generated by a specific T21 model. Next, we recompute the set A by obtaining new set of hard labels from the calibrated models. We then evaluate PolyJuice on the updated SID models. Tab. 2 demonstrates PolyJuice's 's performance against the tuned SID models. Compared to Tab. 1, we observe that finding new threshold significantly improves the detection performance in all cases except for
FLUX
40.0
64.9 (+24)
20.5
29,3 (+8)

---

Normal Attacked
Time
Figure 4: Estimated clean images at various timesteps for FLUX where bottom and top rows depict unstcered and PolyJuice-steerdd generation, respectively.
UFD against SDv3.5. Both UFD and RINE significantly improve their detection of fake images from FLUX(dev) and FLUX|schi under the new settings. In addition, RINE demonstrates overall stronger detection capability compared to UFD. Against the improved detectors, PolyJuice still notably boosts the success rate, much as 72% (in the case of SDv3.5). In this extreme case, find that FLUX shows the least success among the three T21 models. Our combined observations from Tab. 2 and $ 4.1 suggest that FLUX sch) has limited steerability, potentially due fewer inference steps.
Takeaway. Even in the extreme case when the target SID model is calibrated on a specific T21 model G. PolyJuice can still improve the FNR up to 72% while using G' for generating its attacks.
4.3 How Effective is Poly.Jice in Reducing False Negative Rate of Existing SIDs?
To observe the effectiveness of PolyJuice in improving target SID models, we first attack the target detectors with PolyJuice- steered T2I models. Next, we calibrate each SID using combination of (i) real images from COCO, (ii) regular synthetic images from T2I model (i.e. unsteered T2I models), and (iii) PolyJJuce-stcered successful attack images. In Tab. we compare the pre- and post-calibration SID models (Pre-Cal. and -CCl.) on a set of synthetic COCO validation images two different resolutions. We find that after PolyJuice-aided calibration, both
Table 3: Comparison of SID models' FNR before and after Polyluice-aided calibration.
UFD (47)
RINE [33]
T2I Model
Pre-Cal.
A-Cal.
Pre-Cal.
/-CCl.
SDv3.5
13.4
(-5)
15.1
(-11)
88
FLUX(as]
69.2
(-20
52.0
(-38
6
FLUX|ss)
64.3
43.7 (-20)
39.6
18.4 (-21)
SDv3.5
31.1
16.1 (-15)
17.8
(-13))
FLUX(ad]
86.2
70.3 (-15)
69.4
41.4 (-28)
FLUXXs01
71.8
(-16)
50.9
25.4 (-25)
UFD and RINE show significant improvements in detecting regular (unsteered) generated images.
Takeaway.
Calibrating SID models using PolyJuice reduces their vulnerabilities up to 30%.
4.4 How Transferable are the Directions from Lower to Higher Resolutions?
We evaluate the transferability ($ 3.2) of low-resolution 256 x56x256 directions by comparing their attack performance against attacks generated by 512x512 directions, as shown in Tab. 4. Both the transferred and original directions enable PolyJuice to significantly improve the FNR over the un- stecred baseline. Further, PolyJuice attacks using transferred low-resolution directions achieve a comparable or higher FNR than attacks with original high-resolution directions. This result suggests that the error in SPCA-discovered directions proportional to the dimensionality its input space due to the curse of dimensionality [32].
Table Resolution transferability PolyJuice on 512x
RINE[33]
Model
No Steering
Original
Transferred
SDv3.5
26.7
77.6
99.6
FI.UX, [dev]
77.2
95.7
96.7
FLUXX
62.9
79.9
84.1
Takeaway. PolyJuice directions transferred from lower resolutions achieve success rates compa-
rable better than original high-resolution directions, while being less expensive find.

---

4.5 How Does PolyJuice Modify the Image Generation Process?
To better understand how PolyJuice works, in Figs. 4 and 5, illustrate the effect of the steering direction, ò, in the T2I image space and the projected latent space, respectively. In Fig. 4, we visualize the esti- mated clean images at different timesteps. PolyJuice steers the latent towards the blind spot of the target SID (at the second image), resulting in successful attack (top) that slightly differs from the unsteered counterpart (bottom) while maintaining the semantics of the image. In Fig. 5, we provide an SPCA-based 2D visualization of the update directions of the latents (colored by angle), where (left) and (right) depict unsteered and PolyJuice-steereke correspondingly, We observe that majority of PolyJuice-sterres updates are aligned along common direction- the direction that leads to deceiving the SID. See Appendix for more details.
Figure 5: (left) Unsteered (right) PolyJuice- steered samples projected on subspace.
4.6
How Does PolyJuice Affect the Spectral Fingerprint of the T2I Models?
(a) Real Samples
(b) TP Samples
(c) FN Samples
(d) PolyJuice Attacks
Figure 6: Average Frequency Spectra of COCO images and generated counterparts.
Prior work [47, 12] uncovers the existence of fingerprints left by T2I models through spectral analysis on the synthetic and real images. In Fig. 6, we present a similar approach on (a) the set real images, (b) the SDv3.5 geeneraed images correctly detected by UFD (TP), (c) generated images misclassified by UFD (FN), and (d) successful PolyJuice attacks. First, we observe that there is a clearly noticeable difference between the real images and the unsteered SDv3.5 the fingerprint of SDv3.5. However, it can be seen that PolyJuice obfuscates this fingerprint, as the residuals of PolyJuice-steeree images (Fig. 6d) appear closer the real image spectrum (Fig. 6a). We provide implementation details and spectral analysis for other T2I models in the Appendix.
images (TP + FN), which is
Takeaway.
PolyJuice disguises the fingerprints left by T2I models the frequency domain.
4.7 How Do PolyJuice Attacks Appear the Eyes of the Target SID?
In Fig. 7, we provide 2D visualization of PolyJuice-steered at- tacks and unsteered samples from SDv3.5 in the embedding space of CLIP VIT-L/14 [50] using SPCA dimensionality reduction. We illustrate the prediction landscape of the target SID using a kernel density estimate, where darker areas denote input regions with high density of predicted realness. From Fig. 7, we observe that there is a region of perceived realness the SID's input space that remains unexplored by the unsteered T2I model. PolyJuice able to identify this region and exploit deceive the SID.
No Steering
PolyJuioe
Figure 7: CLIP's embedding space.
Takeaway.
PolyJuice steers samples into failure regions missed by standard T2I generation.

---

Saar
Figure 8: Attacks generated by PolyJuice-stecred F model that were able to deceive RINE.
5 Does PolyJuice Conserve Image Quality?
As the goal of red teaming is discover and mitigate the failure modes, finding attacks that are unrealistic (c.g., cartoonish features that fool the SID) is also important, since it reveals the failure mode of the target SID. An ideal SID must trivially detect these unrealistic images as fake. However, it is still important to generate attacks that are prompt faithful, and of reasonable image quality.
Table 5: Image quality comparison for images generated by unsteered FLUX and PolyJuice, evaluated using various distribution-base based metrics.
Method FID 4
cFID +
Prec. 1
Rec. 1
Den. f
Cvg. 1
Unsteered
17.65
17.81 17.41
0.495
0.485
0.585
0.801 0.794
PolyJuice
17.23
0.485
0.498
0.764
Distribution-level image quality. We compare an unsteered FLUX (sch and a PolyJuice-steered FLUX (sh)) against the RINE detector, at 256 x 256 image resolution. We randomly sampled set of 50K real images from the COCO dataset, and the same number of images from unsteered and PolyJuice-steered FLUX[seh) respectively, and present our findings in Tab. 5. First, we compute both classical FID [26] and CleanFID [48], and find a similar trend: the FID w.r.t. real COCO images overall remains quite similar between both methods, suggesting similar quality (also validated by similar Precision (Prec.) and Recall (Rec.) scores [34]). These metrics suggest that PolyJuice conserves image quality, which rules out the possibility of PolyJuice deceiving SIDs by causing image degradations. Further, when comparing density and coverage [45], we notice that PolyJuice has a negligible effect on Coverage (Cov.), but yields a significantly improved Density (Den.) score. This suggests that PolyJuice-steeree images are more likely to reside in neighborhoods that are densely packed with real data points-whid is aligned with the goals of red teaming. We show a few qualitative examples of attacks generated with PolyJuice-steered FLUX[sch) in Fig. 8; more examples are shown B.
Table 6: Image-level quality and prompt faithfulness assessment of PolyJuice vs. unsteered FLUUX using CLIP-based scores.
UFD
RINE
SR
CLIP-IQA
CLIP Score
SR
CIJIP-QAA
CLIIP Score
Res
Unst.
PolyJ.
Unst.
PolyJ.
Unst.
PolyJ.
Unst.
PolyJ.
Unst.
PolyJ.
Unst.
PolyJ.
256
67.6
96.3
0.8427
0.8410
30.57
30.45
$2.4
81.2
0.8427
0.8457
30.57
30.52
512
84.0
98.9
0.8535
0.8487
30.92
30.94
77.2
96.7
0.8535
0.8526
30.92
30.84
1024
75.6
98.4
0.8657
0,8633
30.86
30.91
82.4
94.9
0.8657
0.8503
30.86
30.69
Image-level quality and prompt faithfulness. To measure per-image quality, we use the CLIP IQA score [62]; for evaluating prompt faithfulness, we compute the CLIP alignment score between the generated attacks and the input prompts and present the results in Tab. 6. We find that the CLIP scores and CLIPPIQA scores are comparable to those of the unsteered models, suggesting that PolyJuice not trivially evading detection by degrading the images.

---

6 Related Work
Finding Directions from Images. Some prior work [30, 67] aim to identify latent space directions corresponding to specific visual attributes. Similar to PolyJuice, Zhang et al. [67] also find directions from a set of images. But unlike PolyJuice, these directions are found in the CLIP's embedding space, with the motivation of addressing biases in T2I generation. More similar to our application, Jain et al. [30] use linear SVM to find a hyperplane that best separates correct from incorrect samples in CLIP latent space, allowing them to find the direction of failure modes. However, Jain et al. [30] aim to interpret failure modes as text prompts in the CLIP's embedding space, while PolyJuice focuses on generating black-box attacks synthetic image detectors, by steering the latent space of a T21 generative model.
Unrestricted Adversarial (UA) attacks [57] on image classifiers directly use generative models to create attack images, in contrast to classical adversarial attacks that find a low-norm perturbation on clean images. Some early UA approaches [57, 69, 35, 28] involved finding attack directions in the latent space of generative adversarial networks (GANs) [23]. Recent generative models, such as diffusion models and flow matching models [27, 58, 40, 42] have also been employed for performing
more sophisticated UA attacks [11, 65, 13, 9, 37, 10].
Diffusion- UA Attacks. Diffusion models are well known to be conditioned at inference time with gradients of arbitrary functions, an approach known as guidance [3]. To our knowledge, Xue et al. [65] first repurposed guidance for adversarial attacks by formulating a framework for projected gradient descent over the iterative denoising steps of diffusion models (DiffPGD). concurrent work, AdvDiff [13], performs two-fold attack: by both guiding the diffusion trajectory with adversarial guidance, and optimizing the initial noise prior with the adversarial gradient until successful attack is discovered. While DifPGGD and AdvDiff apply the adversarial gradient over several timesteps, an alternative approach, DiffAttack [91, optimizes the diffusion latent only at one particular timestep, while also exploiting attention maps in the model architecture to preserve realness and structure.
Research Gap. Despite their differences, we note that DifiPGD, AdvDiff, and DiffAttack all share a common feature: at inference time, they compute a image-specific adversarial gradient V., "auucc (æT; 0), where 22, noisy sample at a timestep t, कe is predicted clean image, and Lattuck is an adversarial objective on some classifier O. All these attacks would need complete access to the weights and gradients of an SID model, making them white-box attacks. In contrast, PolyJuice pre-computes a general attack direction from a distribution of images, even with only hard label access. We also note that computing the gradient Va, Lutnck (ăg;; 0) can be computationally intractable for large, illlion-parameterr T2I models and high-resolution images; PolyJuice bypasses this problem by (i) being a gradient-free, black-box method (S 3) and (ii) using resolution transferability ($ 3.2).
7 Concluding Remarks
In this paper, we present PolyJuice, a universal black-box red-teaming method for performing unrestricted adversarial (UA) attacks on synthetic image detectors (SID). To keep up with rapid advances in text-to-image (T2I) generative models, PolyJuice performs the critical task of reinforcing SID models with new attacks and augmented data. Due to its black-box nature, PolyJuice is able to perform attacks on proprietary and commercial detectors that lead the field of deepfake and synthetic image detection. Moreover, PolyJuice is brew once, break many approach: it computes distribution-ban based (in contrast to image-specific) attack that is universally applicable over arbitrary images and different resolutions, enabling computationally efficient UA attacks.
Limitations & Future Directions. PolyJuice is based on a linear dependency between predicted realness and the T2I latents; in some cases, this dependency may be nonlinear. We can use nonlinear kernel in Eq. (3) to discover this dependency; however, this comes with the added complexity of learning nonlinear decoder or solving pre-image problem to map the nonlinear embedding space back to Z. In addition, PolyJuice requires a limited hyperparameter search for appropriate values of Xe the details of which are expanded in the Appendix.
Ethical Considerations. While adversarial generation is essential for evaluating and improving detector models' robustness, it can also be misused. PolyJuice is intended solely for responsible red teaming, and we strongly oppose its use for malicious purposes. Please refer to $ D for defense mechanisms against malicious usage of PolyJuice.
10

---

Acknowledgementss: Mashrur Morshed and Vishnu Boddeti are partially supported by the National Science Foundation (award #2147116), Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.
References
[1] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next- generation hyperparameter optimization framework. In The 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2623-2631, 2019.
[2] Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned Ilms with simple adaptive attacks. In ICML 2024 Next Generation of Al Safety Workshop. 2024.
[3] Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 443-852, 2023.
[4] Lorenzo Baraldi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, A Alessandro Nicolosi, and Rita Cucchiara. Contrasting deepfakes diffusion via contrastive learning and global-local similarities. In European Conference Computer Vision. pages 199-216, Springer, 2024.
[5] Elnaz Barshan, Ali Ghodsi, Zohrch Azimifar, and Mansoor Zolghadri Jahromi. Supervised principal component analysis: Visualization, classification and regression on subspaces and submanifolds. Pattern Recognition, 44(7)
[6] BlackForestLabs.
FLUX.
https://blackkooeestt aii/
unnouncing--blcet wow.----t----------------------- Labs/. 2024.
Nuria Alina Chandra, Ryan Murtfeldt, Lin Qiu. Arna Karmakar, Hannah Lee, Emmanuel Tanumihardja. Kevin Farhat, Ben Caffee. Sejin Paik, Changyeon Lee, et al. Deepfake-eval-222 2024: A multi-modal in-the- wild benchmark of deepfakes circulated in 2024. arXiv preprint arXiv:2503.028: 2025.
[8] Baoying Chen, Jishen Zeng, Jianquan Yang, and Rui Yang. DRCT: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In Proceedings of the 41 International Conference on Machine Learning. PMLR, 2024.
[9] Jianqi Chen, Hao Chen, Keyan Chen, Yilan Zhang, Zhengxia Zou, and Zhenwei Shi. Diffusion models for imperceptible and transferable adversarial attack. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[10] Xinquan Chen, Xitong Gao, Juanjuan Zhao, Kejiang Ye, and Cheng Cheng-Zhong Xu. Advdiffuser: Natural adversarial example synthesis with diffusion models. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 4562-4572, 2023.
[11] Zhaoyu Chen, Bo Li. Shuang Wu, Kaixun Jiang, Shouhong Ding, and Wenqiang Zhang. Content-based unrestricted adversarial attack. Advances in Neural Information Prucessing Syxtems, 36, 2023.
[12] Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdo- liva. On the detection of synthetic images generated by diffusion models. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
[13] Xuelong Dai, Kaisheng Liang, and Bin Xiao. Advdifff: Generating unrestricted adversarial examples using diffusion models. In European Conference on Computer Vision, pages 93-109. Springer, 2024.
[14] Sepehr Dehdashtian, Ruozhen He, Yi Li, Guha Balakrishnan, Nuno Vasconcelos, Vicente Ordonez, and Vishnu Naresh Boddeti. Fairness and Bias Mitigation in Computer Vision: A Survey. arXiv preprint arXiv:2408.000 02464, 2024.
[15] Sepehr Dehdashtian, Bashir Sadeghi, and Vishnu Boddeti. Utility-Fairess Trade-Off's and How to Find Them. JEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
[16] Sepehr Dehdashtian, Lan Wang, and Vishnu Naresh Boddeti. FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs. International Conference on Learning Representations, 2024.
[17] Boyi Deng, Wenjie Wang. Fuli Feng. Yang Deng, Qifan Wang, and Xiangnan He. Attack prompt generation for red teaming and defending large language models. arXiv preprint arXiv:2310.12505. 2023.
11

---

[18] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advunces in neural information processing systems, 34:8780-8794, 2021.
[19] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[20] Joel Frank, Thorsten Eisenhofer, Lea Schönherr, Asja Fischer. Dorothea Kolossa, and Thorsten Holz. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning, pages 3247-3258, PMLR, 2020.
[21] Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, and Kristian Kensting. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302.108/ 2023.
[22] Felix Friedrich, Manuel Brack, Lukas Struppek. Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, Kristian Kersting. Auditing and instructing text-to-image generation models on fairness. Al and Ethics, 2024.
[23] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley. Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 2014.
[24] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bemhard Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. In International conference on algorithmic learning theory. Springer, 2005.
[25] Abdenour Hadid, Nicholas Evans, Sebastien Marcel, and Julian Fierrez. Biometrics systems under spoofing attack: an evaluation methodology and lessons learned. IEEE Signal Processing Magazine, 32(5):20-30. 2015.
[26] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilihrium. Advances neural information processing systems, 30, 2017.
[27] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840- 1051, 2020.
[28] Yu-Chih-Tuan Hu, Bo-Han Kung, Daniel Stanley Tan, Jun-Cheng Chen, Kai-Lamg Hua, and Wen-Huang Cheng. Naturalistic physical adversarial patch for object detectors. In Proceedings of the IEEE/CVF international conference computer vision, pages 7848-7857, 2021.
[29] Huggingface. Huggingface models. https: htttps://bugginggeeet woleceeocco/meeeee www.oo///ooo//ooooooooooooooooo. 2025.
[30] Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry. Distilling model failures as directions in latent space. arXiv preprint arXiv:2206.14755: 147544 2022.
[31] Shuai Jia, Bangjie Yin. Taiping Yao, Shouhong Ding, Chunhua Shen, Xiaokang Yang, and Chao Ma. Adv-attribute: Inconspicuous and transferable adversarial attack on face recognition. Advances Neural Information Processing Systems, 3344447777, 2022.
[32] Mario Köppen. The curse of dimensionality. 5th online world conference on soft computing in industrial applications ( (WSC5), volume I, pages 4-8, 2000.
[33] Christos Koutlis and Symeon Papadopoulos. Leveraging representations from intermediate encoder blocks for synthetic image detection. In Furopean Conference on Computer Vision, pages 394-411. Springer, 2024.
[34] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
[35] Raz Lapid, Eylon Mizrahi, and Moshe Sipper. Patch of invisibility: Naturalistic physical black-box adversarial attacks on object detectors. arXiv preprint arXiv:2303.00 04238, 2023.
[36] Guanlin Li, Kangjie Chen, Shudong Zhang, Jie Zhang, and Tianwei Zhang. ART: Automatic red- teaming for text-to-image models to protect benign users. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. 2024. URL https: /wppppeeeeeeeeeeeeeeeeeeeeeeeeee net/ /tr///ormtti umm
H2AT032115.
12

---

[37] Jin Li, Ziqiang He. Anwei Luo, Jian-Fang Hu, Z Jane Wang, and Xiangui Kang. Advad: Exploring non-parametric diffusion for imperceptible adversarial attacks. Advances in Neural Information Processing Systems, 2024.
[38] Qian Li. Yuxiao Hu. Ye Liu, Dongxiao Zhang, Xin Jin, and Yuntian Chen. Discrete point-wiis wise attack not enough: Generalized manifold adversarial attack for face recognition. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages
2023.
[39] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755. Springer, 2014.
2014:
[40] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamm Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:22 10.02747, 2022.
[41] Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Jingdong Wang, and Yao Zhao. Forgery- aware adaptive transformer for generalizable synthetic image detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10770-10780, 2024.
[42] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 3,000, 2022.
[43] Helmut Lütkepohl. Handbook of matrices. John Wiley & Sons, 1997.
[44] Midjourney. Midjourney. https: //wps:///wwww/////// wtt.:.idjjoooe midjournyy cww/ 2022.
[45] Muhammad Ferjad Nacem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metries for generative models. In International conference machine learning, pages 7176-7185, PMLR, 2020.
[46] Nithin Gopalakrishnan Nair, Anoop Cherian, Suhas Lohit, Ye Wang, Toshiaki Koike-Akino, Vishal M Patel, and Tim K Marks. Steered diffusion: generalized framework for plug-and and-play conditional image synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20850-20866 2023.
[47] Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14480-24489 2023.
[48] Gaurav Parmar, Richard Zhang, and Jun Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11100-14420, 2022.
[49] Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In Yoav Goldberg, Zomitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing pages 3419-3448, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10. 00.88663/11/22222mmmpmaaa.. // //aclanthology org/ 2022. emn p- main. 225/.
URL httpa!
[50] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PmLR. 2021.
[51] Aditya Ramesh, Mikhail Paylov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning. pages 8821 1831. Pmlr, 2021.
[52] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[53] Bashir Sadeghi, Sepehr Dehdashtian, and Vishnu Boddeti. On Characterizing the Trade-off in Invariant Representation Learning. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. Featured Certification.
13

---

[54] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes. Burcu Karagol Ayan, Tim Salimans, al. Photorealistic text-to- image diffusson models with deep language understanding. Advances in neural information processing systems, 00866449964444 2022.
[55] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman. Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-Sb: An open large-scale datasct for training next generation image-text models. Advances in neural information processing systems, 35:25278-22 255294, 2022.
[56] Kaede Shiohara and Toshihiko Yamasaki, Detecting deeptakes with self-blended images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[57] Yang Song, Rui Shu, Nate Kushman, and Stefano Ermon. Constructing unrestricted adversarial examples with generative models. Advances in neural information processing systems, 2018.
[58] Yang Song, Jascha Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 2020.
Diederik P Kingma. Abhishek Kumar, Stefano Ermon, and Ben
[59] Andreas Stathopoulos and Kesheng Wu. A block orthogonaalization procedure with constant synchronization requirements. SIAM Journal on Scientific Computing. 23(6):2165 2182, 2002.
[60] Zekun Sun, Yujic Han, Zeyu Hua, Na Ruan, and Weijia Jia. Improving the efficiency and robustness of deepfakes detection through precise geometric fcatures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1619-6618, 2021.
[61] Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu. Ping Liu, and Yunchao Wei, Rethinking the up- sampling operations enn-based generative network for generalizable deepfake detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28130 18110221133 139, 2024.
[62] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 2555-2563, 2023.
[63] Sheng- Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-gencrated images surprisingly casy to spot... for now. In Proceedings the IEEE/CVF conference computer vision and pattern recognition, pages 8695-8704, 2020.
[64] Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. Controllable invariance through adversarial feature leamning Advances neural information processing systems, 30, 2017.
[65] Haotian Xue, Alexandre Araujo, Bin Hu, and Yongxin Chen. Diffusion- adversarial sample generation for improved stealthiness and controllability. Advances Neural Information Processing Systems, 36: 2894-2921, 2023.
[66] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, al. Scaling autoregressive models for content-rich text-to-image generation. Transactions on Machine Learning Rexearch, 2022.
[67] Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre. Iti-gen: Inclusive text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3969-39900 9969,,980,.0 2023.
[68] Yanbing Zhang, Mengping Yang, Qin Zhou, and Zhe Wang. Attention calibration for disentangled text- to-image personalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 0766-44744 2024.
[69] Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. arXiv preprint arXiv:1710.11
2017.
[70] Yuhao Zhu, Qi Li, Jian Wang, Cheng-7hong Xu, and 7henan Sun. One shot face swapping on megapixels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4834 4844, 2021.
14

---

Appendix
our main paper, we propose PolyJuice for performing unrestricted adversarial attacks on synthetic image detectors. Here, we provide some additional details to support our main results. The appendix section structured follows:
1. Implementation Details in $ A
(a) Threat Model in $ A.1
(b) T2I Model Settings $ A.2
(c) Computing Steering Directions in A.3
(d) Hyperparameter Search for A in $ A.4
(e) Calibrating the Confidence Threshold of an SID in $ A.5
(f) Implementation of Spectral Fingerprint Analysis in $ A.6
(g) Implementation of Projected Steering Directions in $ A.7
(h) More Visualizations on the Effect of PolyJuice in Image Generation Process A.8
2. Qualitative Analysis of Generated Attacks B
(a) Common Patterns in Successful Attacks in $ B.1
3. Additional Results in $
(a) Spectral Fingerprint Analysis in & C.I
(b) Realness Shift in T2I Latent Space in $ C.2
(c) Additional image-space visualizations on the effect PolyJuiceS C.3
(d) Validating whether PolyJuice is applicable on diverse prompts in $ C.4
(c) Evaluating PolyJuice on additional SID models in $ C.5
4. Comparing PolyJuice against a Diffusion- Transferred Attack C.6
5. Potential Defense Mechanisms against PolyJuice in $ D
A
Implementation Details
A.1 Threat Model
Attacker's Goal. Given a text prompt e and latent 2, the attacker aims to generate synthetic images using text-to-image (T2I) generative model G Z C X' that deceive target synthetic image detector (SID) f 2' > [0, 1] into being misclassified as real (class 0).
Attacker's Capability.
Black-box access to the SID: The attacker can only query the SID and observe hard labels Y (real/fake), without access to model weights or gradients.
Full access to the T2I generative model: The attacker manipulate the latent space Z and control the generation process of the T2I model.
SSfffiieett number of queries: The attacker can generate a dataset of im- age-latent-label triplets analyze the SID's behavior in response various inputs.
Attack Scenario.
1. The attacker queries the black-box SID with fake images and obtains hard labels, constructing a dataset TP and FN samples.
2. The attacker pre-computes steering directions in the T2I latent space that correlate with an increased probability of being misclassified real by the SID (Eq. (3)).
At test time, the attacker applies this universal direction to arbitrary prompts, producing images that evade detection by the SID (Eq. (6)).
A.2 T21 Model Settings
provide the generation settings for SDv3.5, FLUX and FL.UX (sch Tab. 7. models are available on Huggingface Models [29] with the corresponding Model ID provided in Tab. 7.
15

---

Table Settings
T21 Models.
Model
Model ID
Num Steps
Guidance Scale
Max Seq. Length
SDv3.5
stabi
28
3.5
256
(LUX
black-toreet-laaa//LDo..
50
3.5
512
FLUX (cl
black f.ack--oorstt-labbae 1schhns11
256
A.3
Computing Steering Directions
In Algorithm 1, we provide pseudocode for calculating the steering direction ô; at given timestep Algorithm requires precomputed T2I latents Z, E R Nxdz, where N the number of generated images and dæ is the dimensionality of the T2I latent space. For efficient calculation of the top-k eigenvalues and corresponding eigenvectors, we use the LOBPCG algorithm [59].
Algorithm 1 Compute Steering Direction at Timestep
Require: Z: E Nxd2, Y E Nx2 (one-hot SID predicted labels)
1: Z <- Ze MEAN( Zt dim
Center the data
2: C Z, .Y
Cross-covariance
3: C-C
>> Kernel matrix in Eq. (3)
4: (a, +- LOBPCG A, 2)
D Top-2 eigenpairs
5: 6, < -U U1
Weighted steering direction in Eq. (4)
6: return 8,
A.4 Hyperparameter Search for A
In Eq. (6), PolyJuice steers a T2I latent Z, using the steering direction 6N and magnitude coefficient X [0, oo). In all steering approaches, finding the correct magnitude for adding the steering direction to the normal flow requires hyperparameter search [3, 46, 22], and PolyJuice is no exception. As result, there a need for finding an optimal A
To efficiently limit the search space for the set (M.)T-) consider X, = X ftasts b), that is, we only apply the steering directions 5e at a constant magnitude À over some continuous interval simple hyperparameter search based the Optuna framework[ 1] to
[a, b) C [0, T) We then find (A, a. b). For any text caption e E C. we define budget, or the maximum number of attempts that T2I model can make deceive SID. We find that PolyJuice successful attacks within few attempts (e.g. 10), as we show in Fig. 9. Further, for all captions, we generate all attacks with the same random seed (0) for determinism.
find majority of the
50
RINE
UFD
Percentage of Successes
40
30
20
10
50
100
150
200
50
100
150
200
Num. Attempts
Figure 9: Number of attempts to find successful attacks using FLUX
We also analyze the hyperparameters corresponding successful attacks as heatmaps. From Fig. 10, we observe that there a negative correlation between constant magnitude coefficient and the length of the continuous interval (a, bj over which PolyJuice is applied. For higher values of X, the steering direction ò, is applied to only few steps, while for smaller values, the steering is done on large number of steps.
16

---

RINE
UFD
43.75
39.25
0.06
34.75
30.25
Range, (b
25.75
0.04
Freq.
21.25
16.75
0.02
12.25
7.75
3.25
0.00
0.29
0.68
1.07
I.40
186
2.24
2.63
3.02
3.41
3.8
0.29
0.68
1.07
1.46
1
2.24
2,63
3.02
3.41
3.8
Coefficient,
Figure 10: Heatmap showing À and corresponding range (b - a) for FLUX
RINE
UFD
4.0
3.6
3.2
0.003
2.8
Coefficient,
2.4
Freq.
0.002
2.0
1.6
1.2
0.001
0.8
0.4
0.0
0.000
10
15
Inference Steps
Figure Heatmap showing and PolyJuice steering steps for FLUX [dev].
Previously, Esser al. [19] show that intermediate timesteps more important in the image generation process of flow matching-basee T2I models. This finding is supported by Fig. 11, which depicts that PolyJuice-steering overall prefers intermediate steps and smaller coefficients.
A.5 Calibrating the Confidence Threshold of SID
For calibrating the threshold r of SID models, we follow the algorithm used by Ojha et al. [47]. As shown in Algorithm 2, the algorithm requires the same number of real and fake samples and their corresponding predicted confidence scores as the input and return the value of the best threshold Thest that maximizes the accuracy. practice, for real images, we use set of images from the training split of COCO, while the fake images are generated by T2I models.
In Tab. 1, we calibrate UFD using 20K real images and 20K generated images mixed from all the T2I models. For each T2I-SID pair in Tab. 2, we calibrate the SIDs (C-Cal. Tab. 2) using the following setting. For successful PolyJuice attacks (n < 1000), use 2n real images, synthetic images generated by the unsteered T2I models, and n PolyJuice-steeeed images.
A.6
Implementation of Spectral Fingerprint Analysis
Given generated image O0, assume that the high-frequent details in σ deterministic component arising from the generative model, i.e. the fingerprint F, and (ii) a random component ( (O, /) [12].
of (i)
For each group images, we first apply a denoising filter and estimate noise residual by computing the difference from the original image. Then we visualize the average residuals in the frequency

---

Algorithm 2 Find the Best Threshold for SIDs (from [47])
Require: Ground truth labels Ymue SID predicted scores Upred
where, Mire
0| |truue
1
1: indices &- ARGSORRTTTTOA (yrue)
2: Vrme: :preed <- Vtrue innicess), Upree [indices]
Sort Vtrue and Upred according to Virue
3: N 4- LL.NN
4: if max({ypre [0 [N I min (Ypead (L. [L:V/2] N)] then
Perfectly separable real and fake
5: return (yrea 0: [N/2]]] + min (Ypred [L [[N/2] NN)
6: end if
7: best acc, Theat f- 0,0
8: for all r E Ypred do
Greedily test each ypred as a threshold
9: temp f- Ypred
10: for = 0 to N do
11: temp(i] 1l
{ [temp[i] 2 r}
12: end for
acc ,/. - [templ
y/moe [i]
14: if acc 2 best_acc then
Thesr TT best_acc +- acc
16:
end if
18:
end for
return Tbest
domain, as shown in Figs. 6 and 20. The algorithm for computing spectral fingerprints is provided in Algorithm
Algorithm 3 Computing Spectral Fingerprint of Images
Require: A set of images X
1: Xdenoised < NL-Mcan
Denoise the images
2: RResidual
X X denoised
Compute the residuals Perform Fast Fourier Transform
3: F < #FT(XResiaaae
4: S ||FT] (FT[SHIFT(F))
Shift the zero-frequency component to the center of the spectrum
5: Stog f- log(1 + 5)
6: return Stog
A.7 Implementation of Projected Steering Directions
Fig. shows the update directions of PolyJuice-steered T2I models projected subspace. For given timestep f, we first solve the eigenvalue problem associated with Eq. (3), to obtain orthogonal projection matrix U, The sayyu-uutteeeinn direction the current timestep, is computed from convex combination over the columns of U, as described by Eq. (4).
Next, for each latent 2,, map the latent to the subspace by computing U.T 2,. We also consider the latent at the next timestep + 1 and similarly project it as U, 21+1. Subsequently, the update vector to the next step can be defined as u, = U (z++1 2z).
For both unsteered and PolyJuice-steene steered T2I latents, plot the unit update direction û, positioned at the corresponding mapped points U,T zz.
A.8 Experiments Compute Resources
For all experimental copps-including dataset generation, direction computation, and attacks-wo used eight NVIDIA RTX A6000 GPUs, each with 48 GB of memory. The primary computational bottleneck arises from the memory requirements of the T2I models during image generation; PolyJuice itself adds negligible overhead. For the highest image resolution considered this paper, image generation consumed approximately 75% of GPU memory, equivalent to 36 GB.
18

---

B Qualitative Analysis of Generated Attacks
We provide some additional qualitative examples of successful attacks from PolyJuice-steered T2I models in Figs. 12 to 17. In general, most of the images look realistic, even though we do not explicitly enforce any realism constraint. However, we notice that there are some characteristies of PolyJuice-generater generated attacks against specific SID models, which we discuss later in B.1.
Figure 12: Attacks generated by PolyJuice-steered FLUX[dev] model that were able to deceive RINE.
Figure 13: Attacks generated by PolyJuice-steered SDv3.5 model that were able deceive UFD.
19

---

Figure 14: Attacks generated by PolyJuice-steered SDv3.5 model that were able deceive RINE,
Figure 15: Attacks generated by Poly steered FLUX model that were able deceive UFD.
20

---

Saur Nass
Figure 16: Attacks generated by PolyJuice-steered FLUX model that were able to deceive RINE.
Figure 17: Attacks generated by PolyJuice-steered FLUX model that were able to deceive UFD.
21

---

B.1
Common Patterns in Successful Attacks
A park bench that has teddy A kite sitting in the middle of hear on it.
Two women in room with one of them holding cake.
A replica of hear and her cub a glass case an exhibit.
large bady of water.
X
Figure 18: (Top) Unsteered T2I- 22I-generated images that RINE correctly detects as fake. (Bottom) PolyJuice-steeree steerrd images that successfully deceive RINE real.
Common Patterns in Successful Attacks on RINE. In successful attacks against RINE, observe that the generated images often exhibit low brightness. In Fig. 18, we show some images generated by unsteered (top) and Poly.Jicee-stennt COCO validation set. Further, the unsteered and steered images share the same random initial latent (i.e. they are all generated with the random seed 0). Although RINE successfully detects the unsteered images as fake, it is deceived by the relatively darker PolyJuice-steeree images. This suggests vulnerability of RINE to synthetic images that appear underexposed or have low brightness levels.
(bottom) T2I models, using captions from the
couple standing sitting middle on branch.
animals standing forest next tiver.
Two men sitting next each other on wooden brach.
large body water.
X
Figure 19: (Top) Unsteered T21- 122l-generated images that UFD correctly detects as fake. (Bottom) PolyJuice-steered images that successfully deceive UFD as real.
Common Patterns in Successful Attacks on UFD. In successful attacks against UFD, we observe that the generated images often exhibit warm colors. Fig. 19 shows some examples unsteered images (top) that UFD detects as fake, and the corresponding PolyJuice-steeed images (bottom) that fools the detector. Even on obviously fake images, such as the third column in Fig. 19 (cartoon of wild animals,
22

---

generated by SDv3.5), PolyJuice produces an image with warmer tone that UFD cannot detect as fake. This suggests vulnerability of UFD synthetic images with warm color temperatures.
C Additional Results
C.1 Spectral Fingerprint Analysis
(a) Real Samples
(b) TP Samples
(c) FN Samples
(d) PolyJuice Attacks
(e) Real Samples
(f) TP Samples
(g) FN Samples
(h) PolyJuice Attacks
Figure 20: More results on Average Frequency Spectra COCO images and generated counterparts
where 1st and 2nd rows correspond to samples from FLUX and FLUX]seh]- respectively.
Fig. 6 in the main paper illustrates the spectral fingerprints of real, unsteered, and PolyJuice-steent samples generated by SDv3.5. Here, we extend the analysis to the other two T2I models, FLUXX and FLUX [sch). The first and second rows of Fig. 20 show the spectral fingerprints of samples generated by [aeg and FLUX|scb). respectively. effectively obscure the characteristic frequency patterns of their underlying T2I models, producing spectra that more closely resemble those of real images compared to unsteered attacks.
observe that PolyJuice-stedd steeeeee attacks
C.2 Realness Shift in T2I Latent Space
(a)
(b)
(c)
(d)
Figure 21: More visualizations of the distribution shift in the T2I model's latent space.
In Fig. the main paper, we showed the realness shift in the latent space T2I model for one timestep. In Fig. 21, we show the shift for four timesteps. We observe that there exists a clear shift between the predicted real and fake samples. However, the degree of linear separability of these distributions is not constant across different timesteps.
23

---

C.3 More Visualizations on the Effect of PolyJuice in Image Generation Process
In Fig. 22, we show additional ipage-space visualizations of the effect of PolyJuice, by estimating clean image at various timesteps. As noted å A.4, we only apply PolyJuice over a continuous interval of inference steps [a, b] S [0, T' can be seen from the figure. In both Figs. 22a and 22b, the original unsteered T2I image generation process (bottom rows) produces an image that is detected by the SID fake. PolyJuice steers the T2I to generate images that deceive the detectors (top rows).
PolyJuice
Unsteered
(a) Intermediate steps for "A replica of a bear and her cub in a glass case in an exhibit."
PolyJuice
Unsteered
X
(b) Intermediate steps for "Too men sitting next to cach other wooden bench."
Figure 22: Visualizing the effect of PolyJuice on the image generation process. (Top) Image that
successfully deceives RINE. (bottom) Image that successfully deceives UFD.
C.4 Validity PolyJuice Across Diverse Prompts
Validity Across Diverse Prompt Categories: To determine whether the effects PolyJuice are applicable across variety of categories of generated content, inspect the COCO validation prompts associated with the attacks generated by PolyJuice and categorize the attacks according to available meta-labels. Tab. 8 shows the fraction of successful attacks per prompt category. We observe that PolyJuice generally improves the success rate across all categories, demonstrating that the discovered direction is universally valid across diverse prompts and image categories.
We also evaluate PolyJuice attacks (directions learned from COCO) on subset of text prompts from the PartiPrompts dataset [66], and present the results Tab. 9. The results demonstrate the generalizability of the PolyJuice attacks to text prompts outside COCO.
Validity Across Prompts Beyond COCO:
24

---

Table 8: Success rate per prompt category in unsteered vs. PolyJuice (on COCO).
UFD
RINE
Unsteered
PolyJuice
Unsteered
PolyJuice
Person
13.7
69.5
16.1
83.5
Animal
12.5
70.8
9.3
90.7
Food
14.4
58.1
11.3
88.8
Vehicle
10.0
73.5
20.1
79.5
Furniture
18.2
66.1
17.3
82.6
Table 9; Attack success rate (%) of PolyJuice on text descriptions from the PartiPrompts dataset.
T21
Detector
Unsteered
PolyJuice (ours)
UFD
13
75 (+62)
SD3.5
RINE
8
100 (+92)
UFD
78
96 (+18)
FLUX(dev]
RINE
42
86 (+44)
UFD
84 (+23)
FLUX
RINE
31
56 (425)
C,5 Additional SID Detectors
We PolyJuice-steered SDv3.5 against four additional SID models: NPR[61], FatFormer(44 [4] J, DRCT [8], and 4)D for further evaluating the effectiveness of PolyJuice. The additional results shown in Tab. 10. These results show the effectiveness of PolyJuice in improving the success rate of the attacks by 56.7% on CoDE, 75% on DRCT, 82% on NPR, and 56% on FatFormer.
Table 10: SR (%) of unsteered SD3.5 samples vs. PolyJuice.
Detector
Unsteered SD3.5
PolyJuice-Steered SD3.5
UFD
12.8
80.6 (+68)
RINE
15.3
99.7 (+84)
CoDE (linear)
43.3
100.0 (+56)
DRCT (UFD)
25.3
100.0 (+74)
NPR FatFormer
6.0
87,6 (+81)
5.5
62.1 (+56)
C.6 Comparison against Transferred Attacks From Diffusion-based White Box Methods
Transferred Attacks are an existing approach attacking black-box models with white-box methods. First, we consider the RINE model as our target black-box model, and train a surrogate detector (a standard ResNet-50 model) to match its responses. We then extend DiffPGD [65] to FL.UX [dev] and SDv3.5, and subsequently perform white-box attacks on the surrogate (the attacks have 100% success rate). The successful attacks are then 'transferred" to the true black-box detector (RINE). From Tab. 11, we observe that PolyJuice has better attack success rate than the transfer methods.
Table 11: SR(%) of PolyJuice vs. regular (x*) and realistic (xf)) attack from transferred DifIPGD
FLUX[dev]
SD3.5
Unsteered DifiPGD (x*)
52.4
15.3
57.2
23.5
DifPPDD (x
70.1
32.1
PolyJuice (ours)
81.2
99.7
25

---

D Potential Defense Mechanisms Against PolyJuice
One possible defense strategy against malicious usage of PolyJuice is collapsing the FN and TP clusters of the data. PolyJuice finds a subspace that is orthogonal to the direction of shift between these two clusters, while preserving the information of the target attribute (i.e., real vs. fake). Approaches from invariant representation learning [53, 64] and fairness [14-16] can be adopted for this purpose. As an example, the notation of U-FaTE [15], the target Y would real vs. fake label, while attribute to be removed, S, would be TP vs. FN.
Additionally, advocate for detector owners proactively PolyJuice generate challenging attack samples and integrate them into their training or fine-tuning data in order to make it robust to malicious usage of PolyJuice.
26

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We support our claims with our extensive experimental results in 4.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations and also future directions for improvement in $ 7.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
27

---

Justification:
No theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide all the implementation details necessary to reproduce the experi- ments our paper, the supplementary. Further, we aim release the code and data upon acceptance of the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
28

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Although the question ambiguous to us, we interpret the question as "whether the paper is submitting all code and data as part of supplemental material, at the time of submission", as opposed to "whether the paper will fully release code and data in the future".
All the text-to-image generative models use in this paper (SDv3.5, FLUX (LUX are publicly available, and so are the text captions we use in image generation (adopted from MS-COCO). We also include pseudo-code and detailed instructions reproduce the main results in the supplementary material.
The data used in this paper involves a large number of generated images, that may inherit the biases associated with generative models. Further, PolyJuice is red-teaming method that may have some potential for misuse (as noted in $ 7. As such, aim to release the full code and dataset at a later time, after an extensive internal audit to address ethical concerns.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: htappss:///ips..//eeee cc/
publ e gu des CodeSubm sss onPol i cy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips... cc/ cttppccpppcccccccccccccc.cccccca pttppp/pliieeeeeeeeeeeeppppppeee
Code PoddSSulie Submi s ooooonioiin for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the necessary details in the experiments section ($ 4), and also provide additional implementation details in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
29

---

Justification: We provide the average and standard deviation of our results, computed over multiple ext-tt generative models (SDv3.5, [dev) (LUX and image resolutions (256 x 256, 512 x 512, 1024 x 1024), in Tab. 1.
However, our results depend on generating high-resolution images from large text-to-image generative models. As the experiments are computationally expensive, it is infeasible for us report bars for every result cell in the table. To report unbiased and reproducible results, we choose the random seed 0 for all generation tasks.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide compute information in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https:///// /tap:p::://uurree n ps. cc/ hac//publitt e/Eth csGut nes?
Answer: [Yes]
Justification: To our knowledge, we abide by all the guidelines presented in the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
30

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss positive societal impacts throughout the paper, and also discuss potential negative societal impacts in the concluding remarks.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeet of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [Yes]
Justification: We do not scrape any data from the internet, but use a well-established and audited dataset (COCO), As noted earlier in the checklist (item 5: open access), all materials related to PolyJuice (model, data, code) will be extensively audited for safety prior to release.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
31

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We attribute original owners and respect their license, wherever applicable.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paper aaaww awi withcode com/ dat aset has curated licenses for some datasets. Their licensing guide help determine the license of a dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is available online, the authors are encouraged to reach out the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
At the moment of submission not releasing any new asset.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
32

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No crowdsourcing and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standarr components.
Please refer to our L.M policy (https //nurruri ps cc/ Conferencess / 2025// LLM) for what should or should not described.
33


Reviews:
[{"review_id": "zqE0Is4jbS", "reviewer": "Reviewer_71g9", "summary": "This paper introduces PolyJuice, a novel and versatile black-box red-teaming framework designed to launch unrestricted adversarial attacks against synthetic image detectors (SIDs). Unlike prior methods that rely on white-box access or image-specific optimization, PolyJuice identifies and exploits systematic weaknesses in SIDs by discovering a universal shift in the T2I latent space. By leveraging this insight, PolyJuice can efficiently steer generated images toward SID failure modes, significantly increasing attack success rates. Moreover, augmenting training datasets with PolyJuice-generated examples leads to notable improvements in SID robustness, with detection accuracy increasing by up to 30%.", "questions": "1. Please refer to Weaknesses\n2. Is the observed latent space distribution shift consistent across different synthetic image detectors? Specifically, does PolyJuice need to re-estimate the steering direction for each SID individually, or can the same direction be transferred between detectors?\n3. How does the latent steering affect the visual quality and semantic consistency of the generated images? Are there metrics or human evaluations to ensure the attack does not simply degrade image realism?", "limitations": "The proposed method relies on identifying a latent space shift that separates correctly and incorrectly classified samples for a specific detector. This shift may not generalize across detectors, potentially limiting the reusability of the learned steering direction.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The manuscript is generally well-written and easy to follow, with clear motivation and logical flow.\n2. The topic of improving the robustness of AIGC image detection addresses an urgent and practical challenge in the era of powerful text-to-image models.\n3. The proposed method is novel in that it introduces a black-box, image-agnostic red-teaming strategy by exploiting latent space distribution shifts — a departure from typical white-box or per-image attack strategies.\n4. The method not only enables effective attacks but also contributes positively by improving detector robustness when used for data augmentation.\n\nWeaknesses：\n1. It is unclear whether the observed latent space distribution shift (between correctly and incorrectly classified images) is consistent across different detectors. If the distribution shift is detector-specific, how transferable or generalizable is the proposed PolyJuice approach?\n2. The method assumes that a linear direction in latent space can universally guide images towards detector failure modes. Is this assumption valid across diverse prompts and image categories? A discussion or ablation study would help clarify.\n3. While the method improves attack success rates, it is important to analyze whether the manipulated/generated images remain visually realistic and prompt-faithful. Otherwise, there’s a risk that PolyJuice may just generate degraded or unrealistic images that trivially evade detection.\n4. The authors evaluate PolyJuice primarily on UFD and RINE as target synthetic image detectors. However, it remains unclear how well the method generalizes to more recent and potentially more robust detectors such as FatFormer (CVPR 2024) and NPR (CVPR 2024).", "was_revised": true, "final_justification": "Thank the author for their response. The author needs to discuss in detail the impact of the method on image quality in the final version.", "rebuttal": "---\n\n---\n\n# Summary of the Review & Response\n\n---\n\n---\n\nWe thank the reviewer for their positive feedback. We are pleased that the reviewer finds the manuscript to be *well-written, clearly motivated,* and *easy to follow* (S1), and we view this as a key strength of our presentation. The reviewer identifies that our paper addresses *an urgent and practical challenge* regarding AI-generated image detection (S2) through a *novel attack strategy* (S3). Further, the reviewer highlights our positive contribution to improving detection methods (S4).\n\n**Primary concerns** and questions are regarding:\n\n**1)** whether the attack directions are reusable between detectors (W1, **L1**) \n\n**2)** whether the method generalizes over diverse prompts (W2) \n\n**3)** prompt faithfulness and quality of the generated images (W3) \n\n**4)** experiments on additional SID models (W4) \n\n**We address:** \n\n**1)** by clarifying that the attack directions are SID-specific, and that transfer attacks are more relevant for white-box approaches\n\n**2)** by highlighting the diversity of the prompts used in PolyJuice evaluation (**Table R1**); we also extend PolyJuice to captions beyond COCO using PartiPrompts (**Table R5**, response to reviewer 9sRv, W1)\n\n**3)** by computing CLIP alignment score (prompt faithfulness) and CLIP-IQA score (quality) in **Table R2**\n\n**4)** by evaluating the effectiveness of PolyJuice on 4 additional SID models (**Table R3**)\n\n---\n\n---\n\n# Response to Weaknesses\n\n---\n\n---\n\n> **W1. It is unclear whether the observed latent space distribution shift is consistent across different detectors. If the distribution shift is detector-specific, how transferable or generalizable is the proposed PolyJuice approach?**\n> \n\nWe first clarify that the distribution shift is detector-specific, as different detectors have distinct vulnerabilities. The shifts are not consistent unless the models share similar weaknesses. As shown in Sec B.1 of the supplementary material, the common patterns differ across models, reflecting variations in training data and techniques. In general, transferability is more relevant in the context of white-box attacks where one must craft an attack on a surrogate detector in order to apply a transfer attack to a target black-box model. In contrast, we focus on black-box attacks where we are able to target each model directly.\n\n---\n\n> **W2. The method assumes that a linear direction in latent space can universally guide images towards detector failure modes. Is this assumption valid across diverse prompts and image categories? A discussion or ablation study would help clarify.**\n> \n\nThank you for your suggestion, we agree that such an ablation can be useful to the paper. We first inspect the successful attacks generated by PolyJuice using COCO validation prompts and categorize the attacks according to available meta-labels. **Table R1** shows the fraction of successful attacks per prompt category. We observe that PolyJuice improves the success rate across all categories, demonstrating that the direction is universally valid across diverse prompts and image categories. We also extend PolyJuice to another diverse dataset, i.e., PartiPrompts (see **Table R5**, in response to reviewer 9sRv-W1).\n\n**Table R1.** Success rate per prompt category in unsteered vs. PolyJuice (on COCO).\n\n|  | **UFD** | **UFD** | **RINE** | **RINE** |\n| --- | --- | --- | --- | --- |\n|  | **Unsteered** | **PolyJuice** | **Unsteered** | **PolyJuice** |\n| **Person** | 13.7 | 69.5 | 16.1 | 83.5 |\n| **Animal** | 12.5 | 70.8 | 9.3 | 90.7 |\n| **Food** | 14.4 | 58.1 | 11.3 | 88.8 |\n| **Vehicle** | 10.0 | 73.5 | 20.1 | 79.5 |\n| **Furniture** | 18.15 | 66.1 | 17.3 | 82.6 |\n\n---\n\n> **W3. While the method improves attack success rates, it is important to analyze whether the manipulated/generated images remain visually realistic and prompt-faithful. Otherwise, there’s a risk that PolyJuice may just generate degraded or unrealistic images that trivially evade detection.**\n> \n\nFirst, we want to clarify our stance on how an SID should process an “unrealistic” image versus a “degraded” one. Since the goal of red teaming is to discover and mitigate the failure modes, finding attacks that are not necessarily realistic (e.g., cartoonish features that fool the SID) is also important to us, since it reveals the failure mode of the target SID. An ideal SID must trivially detect these kinds of unrealistic images as fake. \n\nWe agree with the reviewer’s remark that it is important to generate attacks that are prompt faithful, and of reasonable image quality. To measure prompt faithfulness, we compute the **CLIP alignment score** between the generated attacks and the input prompts; to measure image quality, we use the **CLIP IQA score** (Wang et al. 2023). We present the alignment and image quality score alongside the success rate for FLUX[dev] in **Table R2**. We find that the CLIP scores and CLIP-IQA scores are comparable to those of the unsteered models, suggesting that PolyJuice is not trivially evading detection by degrading the images.  \n\n**Table R2.** Quality & alignment scores vs. SR (%) on UFD and RINE, using FLUX[dev]\n\n|  | **UFD** | **UFD** | **UFD** | **RINE** | **RINE** | **RINE** |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | **SR**  | **CLIP-IQA**  | **CLIP Score**  | **SR** | **CLIP-IQA** | **CLIP Score** |\n| 256-Unsteered | 67.6 | 0.8427 | 30.57 | 52.4 | 0.8427 | 30.57 |\n| **256-PolyJuice (ours)** | **96.3** | **0.8410** | **30.45** | **81.2** | **0.8457** | **30.52** |\n| 512-Unsteered | 84.0 | 0.8535 | 30.92 | 77.2 | 0.8535 | 30.92 |\n| **512-PolyJuice (ours)** | **98.9** | **0.8487** | **30.94** | **96.7** | **0.8526** | **30.84** |\n| 1024-Unsteered | 75.6 | 0.8657 | 30.86 | 82.4 | 0.8657 | 30.86 |\n| **1024-PolyJuice (ours)** | **98.4** | **0.8633** | **30.91** | **94.9** | **0.8503** | **30.69** |\n\n---\n\n> **W4. The authors evaluate PolyJuice primarily on UFD and RINE as target synthetic image detectors. However, it remains unclear how well the method generalizes to more recent and potentially more robust detectors such as FatFormer (CVPR 2024) and NPR (CVPR 2024).**\n> \n\nFirst, we note that RINE (ECCV 2024), which is evaluated in our paper,  is a recent and concurrent work to the detectors noted by the reviewer. We thank the reviewer for suggesting these SID models that can enrich the empirical results of the paper. As such, we adopt additional SID models: NPR and FatFormer (suggested by reviewer 71g9), as well as DRCT and CoDE (suggested by reviewer 2n7t) for further evaluating the effectiveness of PolyJuice.  The additional results are shown in **Table R3**. These results show the effectiveness of PolyJuice in improving the success rate of the attacks by 56.7% on CoDE, 75% on DRCT, 82% on NPR, and 56% on FatFormer. These results will be added to Table 1 of the paper in the final revision.\n\n**Table R3.** SR(%) of unsteered SD3.5 samples vs. PolyJuice.\n\n| **Detector** | **Unsteered SD3.5** | **PolyJuice-Steered SD3.5** |\n| --- | --- | --- |\n| UFD | 12.8 | 80.6 (+68) |\n| RINE | 15.3 | 99.7 (+84) |\n| CoDE (linear) | 43.3 | 100.0 (+56) |\n| DRCT(UFD) | 25.3 | 100.0 (+74) |\n| NPR | 6.0 | 87.6 (+81) |\n| FatFormer | 5.5 | 62.1 (+56) |\n\n---\n\n---\n\n# Response to Questions\n\n---\n\n---\n\n> **Q2. Is the observed latent space distribution shift consistent across different synthetic image detectors? Specifically, does PolyJuice need to re-estimate the steering direction for each SID individually, or can the same direction be transferred between detectors?**\n> \n\nAs mentioned in the response to W1, different models have different vulnerabilities, so the directions are different. As such, the directions need to be estimated for each SID. However, estimating the steering directions requires a negligible computational cost since once we generate the images and latents for the first SID, we need to 1) predict the labels using the new SID, 2) calculate the eigenvalue problem in Eq. 3. \n\n---\n\n> **Q3. How does the latent steering affect the visual quality and semantic consistency of the generated images? Are there metrics or human evaluations to ensure the attack does not simply degrade image realism?**\n> \n\nThe latent steering performed by PolyJuice has a negligible effect on the visual quality and semantic consistency of the generated images, as shown in qualitative analysis in Sec B of the supplementary material. Further, we employ **CLIP IQA** **score** and **CLIP alignment score** to measure image quality and prompt faithfulness of the generated attacks compared to unsteered images, and the results are provided in **Table R2**. The results support our remark that the quality and consistency are practically unchanged by PolyJuice while the attack success rate (SR) is improved significantly. \n\n---\n\n---\n\n# Response to Limitations\n\n---\n\n---\n\n> **L.  The proposed method relies on identifying a latent space shift that separates correctly and incorrectly classified samples for a specific detector. This shift may not generalize across detectors, potentially limiting the reusability of the learned steering direction.**\n> \n\nWe mentioned that the directions are not necessarily transferable across SID due to the difference in their vulnerabilities. However, if two SIDs’ predictions have a significant overlap with each other, then the directions from one SID can potentially be useful in attacking the other SID. \n\n---\n\n---\n\n**Please consider updating your scores if we have addressed your concerns. If you have more questions, we would be happy to answer them.**\n\n---\n\n---\n\n# References\n\nWang et al. \"Exploring clip for assessing the look and feel of images.\" AAAI (2023)."}, {"review_id": "o1COkIYaAO", "reviewer": "Reviewer_2n7t", "summary": "The paper introduces **PolyJuice**, a universal red-teaming method that steers text-to-image generators toward regions that fool synthetic-image detectors (SIDs) while requiring only black-box label access. \nIt computes a single \\\"realness shift\\\" direction per diffusion timestep via supervised PCA on latent vectors of true-positive vs. false-negative samples, then applies this vector during sampling. \nThe pre-computed directions transfer across prompts and resolutions, enabling query-efficient, image-agnostic attacks. \nExperiments with three generators (SD v3.5, FLUX-dev, FLUX-sch) and two SIDs (UFD, RINE) show up to 84 p.p. increase in false-negative rate and demonstrate that augmenting training data with PolyJuice images can cut detector errors by up to 30 p.p.", "questions": "See mainly weaknesses for longer discussions but would appreicate crisp answes to the following;\n\n- What is the minimum number of SID queries needed to approximate the steering direction within 5 pp of reported success rates?\n- How does PolyJuice compare to a transfer-based black-box attack (e.g., DiffPGD trained on a surrogate SID)?\n- Have you evaluated on non-COCO domains like, face-focused, medical or multi-modal detectors? \n- How sensitive are results to the dimensionality *d* of the SPCA subspace and to the choice of λₜ?\n- Could non-linear kernels in SPCA further improve attack success? Any preliminary experiments?", "limitations": "- No discussion of query-based cost constraints or rate-limiting defenses at commercial endpoints.(as the claim in fooling even commercial APIs)\n- Highly limited literature survey neglecting latest SOTA models on SID and black-box attacks using Diffusion Models\n- Does not address potential legal/ethical implications of releasing universal attack directions.\n- Assumes access to clean TP/FN split; impact of noisy SID labels is not explored.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n- **Novelty**: Black-box, *universal* unrestricted attack on SIDs leveraging a latent-space distribution shift.\n- **Efficiency**: One-off offline direction discovery and low→high-resolution transfer avoid costly per-image optimization.\n- **Effectiveness**: Large gains in SID false-negative rates across multiple generators and resolutions.\n- **Practical utility**: PolyJuice images improve detector robustness when added to training data.\n- **Clarity**: Motivation, algorithm and experiments are explained with helpful figure. \n \n# Weaknesses\n- **Baseline Coverage**: No comparison to prior black-box or transfer-based UA attacks; only \\\"no-steering\\\" baseline. A quick search revealed Kotyan et al 2024, proposing a black-box attack using evolutionary algorithm. I think the authors should conduct a more thorough literature survey and conduct a better quantitative study to show their attack's effectiveness over other black-box attacks. Infact, I would even urge to compare against White-Box Attacks to quantify the gap between White-Box and Black-Box Attacks, Good black-box attacks should minimize the performance gap. Simply showing effectiveness of their own approach by being ignorant and neglecting existing works do not give a good baseline comparision. \n- **Limited Detectors**: Evaluation covers just two academic SIDs that are not even state-of-the art SID. Once again a shallow literature survey from this perspective is revealed, a quick search clearly indicates UFD is not state-of-the-art, Baraldi et al. 2024 (DRCT) and Chen et al. (CoDE) are much general SID models and would test the capabilities of PolyJuice better. A bad detector will be easy to fool, but a strong general detector will be harder. It is imperative to test and validate the attack against SOTA SID models.\n- **Image Quality Metrics**: While the authors conduct evaluation using performance, lack of reporting of image quality of adversarial samples and comparision to non-adversarial samples cast a doubt over claims indicated in the article. Zhong et al 2023 demonstrate that performance of SID models decrease when image quality is affected, therefore it is impertative to check and validate that image quality of adversarial samples are not degraded, if image quality is degraded, then the claims of the article is wrong as PolyJuice is greedily fooling SID by decreasing the image quality rather than directing image-generation models towards the blind-spot. Slighty degraded images compared to non-degraded non-adversarial images also point to distribution shift affecting the SID models not explictly trained for this distrubution shift. \n- **Potential Gaps in Theroetical Claims and Assumptions**: Latent Size of Modern diffusion models are far bigger than the sample size so in this “large-p, small-n” regime, the sample covariance is necessarily rank-deficient, so the eigenvectors are extremely sensitive to noise. No formal discussion of query noise or finite-sample error; suggest adding concentration inequality or error bound.As a result, the steering direction may capture sampling artefacts (like poor image quality) rather than a genuine class-separating shift. Further for steering direction, no proof is supplied that this convex combination maximises HSIC in the directional (rank-1) setting; the classical SPCA solution would instead select the single leading eigenvector. A short lemma is needed to justify why the proposed weighting improves—or even preserves—the HSIC objective.\n- **Reverse Process Validity**: Adding $\\lambda_t \\delta_t$ at each step perturbs the reverse posterior. The proof lacks a guarantee that the perturbed trajectory remains within the manifold on which the score network was trained, raising the possibility of off-manifold artefacts that are trivially hard for the detector. Samuel et. al 2023 showed that if latent vector is manipulated much such that their norm changes than the image quality decreases reiterating that images generated might be degraded compared to original samples leading to misclassification by SID.\n- **Ablations for Hyperparameter Sensitivity**: Any point of hyperparameter is relegated to appendix; however no ablations are conducted to validate the selection of the hyper-parameters. An ablation study can better validate these choices. \n- **Overarching Red-Teaming Claims**: The authors suggest that the attack proposed by them can be used as a red-teaming tool. The main purpose of a red-teaming tool is to identify an actionable vulnerability that can be used to improve robustness. Current article, only proposes an adversarial attack, it hypothesizes on the vulnerability and make no contributions in improving the defence of the SIDs or show evidence tfor the same. Therefore, I think the claim is overarching and should be avoided. The contribution is simply a black-box adversarial attack on SID models. \n- **Untested Commercial Claims** - Authors highllight that the method can be used on commerical SID models but lacks tests to validate it. To test true \"black-box\" claim, it is essential to apply the attack to commercial SID models like Sightengine (https://dashboard.sightengine.com/ai-image-detection) and AI or Not: (https://www.aiornot.com/dashboard/home) and demonstrate the capability, otherwise I urge the authors strongly to not overexaggerate their claims and refrain from making unwarranted claims.\n\nIn general, I find two major limitations in terms of literature survey and overexaggerated claims;\n- Literature survey of the article highly unsatisafactory and outdated in 2025, both from the point of black-box attack using Diffusion Models and choosing Synthetic Image Detectors for the experiments, failing to account many studies that came in 2024. \n- Claims made in the article are not accounted and rather boldly presented as contributions. \n\n# Comment on Scope (Won't term it weakness but definitely very narrow scope impacting significance)\nThe proposed does not use domain information specific to SID yet it's application was limited to SID. I would appreciate if authors can comment upon why narrow domain was targetted, when the proposed approach can be used for any image classfiication tasks? Object Classification, NSFW Classifcation? If the authors can demonstrate this, it would raise the significance and applicability of this research to a broader community. Further even in SID, specicialised domain like Facial Images are not evaluated. Recently, Guo et al 2025 proposed an interpretable SID using LLM showing a shift of SID task from Binary Clasification to more complex task. It would be nice to see if the proposed attack can also fool such latest models diverging SID from simple binary classification. \n\n## References\n- Samuel, D., Ben-Ari, R., Darshan, N., Maron, H., & Chechik, G. (2023). Norm-guided latent space exploration for text-to-image generation. Advances in Neural Information Processing Systems, 36, 57863-57875.\n- Zhong, N., Xu, Y., Li, S., Qian, Z., & Zhang, X. (2023). Patchcraft: Exploring texture patch for efficient ai-generated image detection. arXiv preprint arXiv:2311.12397. https://fdmas.github.io/AIGCDetect/data/\n- Baraldi, L., Cocchi, F., Cornia, M., Baraldi, L., Nicolosi, A., & Cucchiara, R. (2024, September). Contrasting deepfakes diffusion via contrastive learning and global-local similarities. In European Conference on Computer Vision (pp. 199-216). Cham: Springer Nature Switzerland.\n- Chen, B., Zeng, J., Yang, J., & Yang, R. (2024, July). Drct: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In Forty-first International Conference on Machine Learning.\n- Kotyan, S., Mao, P. Y., Chen, P. Y., & Vargas, D. V. (2024). Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models!. arXiv preprint arXiv:2402.04699.\n- Guo, X., Song, X., Zhang, Y., Liu, X., & Liu, X. (2025). Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 105-116).", "was_revised": true, "final_justification": "Authors have addressed my concerns in the rebuttal and I hope with the additional discussion that followed, the article's quality will be improved. I highly encourage the authors to add the results they have posted in the rebuttal in their final version.", "rebuttal": "---\n---\n\n# Summary of Reviews & Response\n\n---\n---\n\nWe are encouraged that the reviewer highlights the novelty (S1), efficiency (S2), effectiveness (S3), and practical utility (S4) of our red-teaming approach as well as the clarity of our paper (S5) and its broader applicability beyond SID (in “comment on scope”).\n\n**Reviewer’s primary concerns** are:\n\n**1)** literature review (W1 & 2, Q2, L2)\n\n**2)** paper’s claims (W4, 5, 7 & 8) \n\n**3)** image quality evaluation (W3)\n\n**We address:** \n\n**1)** by demonstrating PolyJuice’s effectiveness on four additional SID models in **Table R3**, and by comparing it to a transfer-based black-box attack (Q1) in **Table R4**. We point out that the paper already evaluates against RINE (ECCV 2024), a recent SID that is as recent as or newer than the reviewer’s suggestions (W2) and has a lower FNR (**Table R3**).\n\n**2)** by clarifying misunderstandings on latent space size, data requirements, and SPCA components. We refer to Table 3, where we empirically show that PolyJuice reduces SID error (and also in **Table R7 reviewer 9sRv-W4**), directly supporting our red-teaming claims. Although we discuss PolyJuice’s applicability to commercial SIDs, we make no performance claims, as access to those models was not granted.\n\n**3)** by providing image quality score (CLIP IQA) and prompt alignment score in **Table R2.**\n\n---\n---\n\n# Response to Weaknesses\n\n---\n---\n\n> **W1a. Comparison to Kotyan et al. (EvoSeed)**\n\nEvoSeed is **not scalable** to modern T2I models. It needs up to $\\tau \\times (4 + 3 \\times \\log(n))$ = **16,500** imgs / 1 atk for SD. To match our experiments (SR(%) on 1000 atks), it needs up to **16.5M** images, which is **practically infeasible**. \n\n---\n\n> **W1b.** **Comparison to transfer-based attack**\n\nWe perform a transfer attack on the RINE detector with DiffPGD and a surrogate (Q2)*.* **Table R4** shows that PolyJuice achieves a better SR(%) than the transfer attacks.\n\n**Table R4.** SR(%) of PolyJuice vs. regular ($x^n$) & realistic ($x^n_0$) attack from transferred DiffPGD$^t$.\n\n|  | FLUX[dev] | SD3.5 |\n| --- | --- | --- |\n| Unsteered | 52.4 | 15.3 |\n| DiffPGD$^t (x^n)$  | 57.2 | 23.5 |\n| DiffPGD$^t(x^n_0)$  | 70.1 | 32.1 |\n| **PolyJuice (ours)** | **81.2** | **99.7** |\n\n---\n\n> **W1c. Comparison to white-box attack**\n\nWe believe such a comparison is not informative. A black-box attack with 40% SR appears weak vs. a white-box attack with 90%, but if the theoretical max. for black-box attacks is 41%, then it's very strong. So, comparing against white-box is not enough to judge a black-box attack.\n\n---\n\n> **W2. Not SOTA / Limited SID**\n\nWe believe there is a misunderstanding. While the reviewer questions UFD’s relevance, they overlook our inclusion of RINE (ECCV 2024), a recent and competitive SID published alongside CoDE and after DRCT, which qualifies it as a SOTA academic SID. UFD remains a critical baseline, widely used or built upon by DRCT, CoDE, NPR, FatFormer, and RINE. \n\nTo address the reviewers’ suggestions (2n7t & 719g), we evaluate PolyJuice against four new SIDs and report strong improvements in **Table R3**.\n\n**Table R3.** SR(%) of unsteered SD3.5 samples vs. PolyJuice.\n\n| SID | Unsteered | PolyJuice |\n| --- | --- | --- |\n| UFD | 12.8 | 80.6  **(+68)** |\n| RINE | 15.3 | 99.7  **(+84)** |\n| CoDE (linear) | 43.3 | 100.0 **(+56)** |\n| DRCT(UFD) | 25.3 | 100.0 **(+74)** |\n| NPR | 6.0 | 87.6  **(+81)** |\n| FatFormer | 5.5 | 62.1  **(+56)** |\n\n---\n\n> **W3a. Image Quality Metrics**\n\n**Table R2** reports (i) CLIP-IQA scores for image quality and (ii) CLIP alignment scores for prompt consistency. The results show that PolyJuice significantly boosts attack success while preserving both quality and alignment, confirming it targets SID blind spots rather than exploiting degradation.\n\n**Table R2.** Quality & alignment scores vs. SR (%) on UFD & RINE, using FLUX[dev].\n\n|  | UFD | UFD | UFD | RINE | RINE | RINE |\n| --- | --- | --- | --- | --- | --- | --- |\n|  | SR  | CLIP-IQA  | CLIP Score  | SR | CLIP-IQA | CLIP Score |\n| 256-Unsteered | 67.6 | 0.8427 | 30.57 | 52.4 | 0.8427 | 30.57 |\n| **256-PolyJuice (ours)** | **96.3** | **0.8410** | **30.45** | **81.2** | **0.8457** | **30.52** |\n| 512-Unsteered | 84.0 | 0.8535 | 30.92 | 77.2 | 0.8535 | 30.92 |\n| **512-PolyJuice (ours)** | **98.9** | **0.8487** | **30.94** | **96.7** | **0.8526** | **30.84** |\n| 1024-Unsteered | 75.6 | 0.8657 | 30.86 | 82.4 | 0.8657 | 30.86 |\n| **1024-PolyJuice (ours)** | **98.4** | **0.8633** | **30.91** | **94.9** | **0.8503** | **30.69** |\n\n---\n\n> **W3b. Regarding Zhong et al.**\n\nZhong et al. show that **UFD has up to 8% drop** when applied to degraded images, compared to clean images. In stark contrast, PolyJuice improves the attack success rate against UFD by up to **67%**. This substantial gap shows that the improved attack success is not caused by degradation (supported by **Table R2**). \n\n---\n\n> **W3c.** **Stance on slight degradation**\n\nAlthough **Table R2** shows that PolyJuice preserves image quality, it’s worth noting that many recent SIDs are explicitly trained to be robust against degradations like blurring, pixelization, etc. (e.g. CoDE, Sec. 3-2, pg. 5). Therefore, if a slightly degraded image still bypasses such defenses, it exposes a critical failure mode that needs to be addressed.\n\n---\n\n> **W4a. “large-p, small-n” assumption**\n\nThe reviewer’s assumption is incorrect. For 256x256 images, the latent size in the FLUX/SD3.5 is **p=16,384** (i.e. 16x32x32), while we use **n=40,000** data points. Moreover, due to the resolution transferability of PolyJuice (Sec 4.4), we do not need to estimate this direction again for higher resolutions. **Table R6 (reviewer 9sRv-W2)**  also shows that the eigenvector is not sensitive to the number of data points.\n\n---\n\n> **W4b. Selection of SPCA eigenvectors**\n\nFor $Y$ with $m$ classes, subspace’s dimensionality (num. of non-zero $\\sigma_i$) is $d=m-1$. In our binary case, $d=1$ and Eq. (4) becomes $\\delta=\\sigma_0U_0$ as is expected by the reviewer. We will clarify this in the revised paper.\n\n---\n\n> **W5. Samuel et al. & Reverse Process Validity**\n\nSamuel et al. is only concerned with the **initial Gaussian noise prior**, $x_T \\sim \\mathcal{N}(0, \\mathbf{I})$, but **not the trajectory**. They show that if $x_T$ is perturbed such the norm deviates from $\\sqrt{d}$, the image quality significantly drops. There is no contradiction with our findings, as we do not apply any perturbation on the initial noise; we only apply the direction starting from step 1 (L134 & Eq 6).\n\nPrevious papers (Universal Guidance) justify empirically choosing steering strength. **Table R2** shows that PolyJuice’s success is not due to image degradation. \n\n---\n\n> **W6. Ablation on Hyperparameters**\n\nWe ablate w.r.t the number of samples by using n = {16K, 40K, 52K}. The results, shown in **Table R6 (reviewer 9sRv, W2)**, indicate that PolyJuice directions are not sensitive to n. We also visualize the correlation between successful attacks and (i) steering strength $\\lambda$ and (ii) steering timesteps in **Fig. 9** and **Fig. 10**, respectively.\n\n---\n\n> **W7. Overarching Red-Teaming Claims**\n\nWe show in Sec. 4.3 that using PolyJuice attacks to calibrate the detectors improves their detection rate by up to 30%. In fact, this is acknowledged as a strength by the reviewer **(S4. Practical utility),** and also by reviewer 71g9 (**S4**). **Table R7 (rev. 9sRv, W4)** also supports this.\n\n---\n\n> **W8. Untested Commercial Claims**\n\nWe clarify that PolyJuice does not claim to demonstrate empirical performance on commercial models but rather enables the possibility of red teaming them due to its black-box design.\n\nDuring the rebuttal period, we were unable to access commercial SIDs due to either their ToS or the companies’ disagreement. \n\n---\n\n> **Comment on Scope**\n\nWe appreciate the reviewer’s recognition of the method’s broad applicability. Our work focuses on red teaming SIDs due to the urgent need to address risks posed by AI-generated content (AIGC), making it a timely and critical problem. While this paper targets SIDs, we hope future works explore its applications to other domains.\n\n---\n---\n\n# Response to Questions\n\n---\n---\n\n> **Q1. …approximate the steering direction within 5 pp …?**\n\nAs shown in **Table R6 (reviewer 9sRv-W2)**, approximating the steering directions with 16K points yields the same result as the 40K we use in the paper (SR of  96.3%). Using more data points, 52K, leads to a very small increase (96.5%), which alludes to the low sensitivity of directions to n.\n\n---\n\n> **Q2. …DiffPGD trained on a surrogate SID?**\n\nAddressed in W1 & **Table R4**; we apply DiffPGD (with 2 T2Is) on surrogate SID (ResNet50) and transfer to black-box RINE. We find that PolyJuice has better attack performance.\n\n---\n\n> **Q3.  Non-COCO domains**\n\nPolyJuice's primary focus is on red teaming general AI-gen. image detectors, rather than domain-specific ones. As a dataset beyond COCO, we also evaluate on PartiPrompts (**Table R5, reviewer 9sRv-W1**).\n\n---\n\n> **Q4. Sensitivity to $d$ and $\\lambda$**\n\nThe dimensionality of the subspace $d$ is determined by the number of classes of $Y$ and in the binary case is equal to one; therefore there is no variability here wrt $d$.  For $\\lambda$ sensitivity, please refer to Sec. A.3 supplementary material.\n\n---\n\n> **Q5. Possible usage of non-linear SPCA**\n\nThank you for the thoughtful question. As noted in L320, while a non-linear kernel may offer a boost in success rate by improving separability, it adds significant complexity due to the need for solving a pre-image problem.\n\n---\n---\n\n# Response to Limitations\n\n---\n---\n\nWe address **(L1)** in response to W5 of the reviewer 9sRv, **(L2)** in W1, W2, (**L3)** in Sec 6 & W4 5 of the **reviewer 9sRv.** We would appreciate clarification on the noisy TP+FN scenario described in **(L4)**.\n\n---\n---\n\n**Please consider raising your score if we addressed your concerns. We are happy to answer more questions.**\n\n---\n---"}, {"review_id": "e5FNBxoVQf", "reviewer": "Reviewer_qKp6", "summary": "This paper introduces **PolyJuice**, a novel black-box and image-agnostic red-teaming method designed to improve the effectiveness of Synthetic Image Detectors (SIDs). Recognizing the limitations of existing red-teaming solutions—which often require white-box access to proprietary SIDs and rely on computationally expensive, image-specific optimizations—PolyJuice offers a universal approach. The core insight behind PolyJuice is the observation of a discernible distribution shift in the latent space of Text-to-Image (T2I) models between images correctly classified by an SID and those misclassified (i.e., synthetic images that the SID erroneously identifies as \"real\").", "questions": "See weakness please.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n**1. Strong Empirical Results:** I trust this paper provides compelling quantitative results, showing PolyJuice's effectiveness in boosting attack success rates (up to 84%) and, conversely, its utility in improving SID robustness (up to 30% reduction in FNR) when used for data augmentation.\n\n**2. Addresses a Critical Gap in Real-world:** PolyJuice directly tackles the pressing issue of the \"arms race\" between rapidly advancing T2I models and the SIDs designed to counter them. Its ability to generate challenging, misclassified synthetic images is vital for evolving SID capabilities. The black-box nature of PolyJuice is highly significant for real-world applications, as many advanced SIDs are proprietary and only offer API access, making traditional white-box attacks infeasible.\n\n**3. Practical Threat Model:** This paper employs a practical **black-box** threat model, which is more practical than the previous settings. I think this is important for discuss it in real-world applications.\n\n**Weaknesses**\n\n\n**1. Strong Empirical Results:** The writing and the general structure of this paper needs to be improved, including how the latent space shifting helps to evade the detection. I saw too many unnecessary bolded text in the paper, which may reduce the readability of the paper.\n\n**2. Threat Model is Missing:** I didn't find the threat model for the proposed attacks in this paper, including the description to the attacker's capability, goal and attack scenarios. \n\n**3. Confused about the perturbation optimization:** If I didn't get it wrong, PolyJuice first get the hard predictions from the querying to black-box APIs, and it formulate a distribution with latent space in t2i models. I am confusing about the definition of of latent space here. Is the latent space of vae or just the embedding space in CLIP? And how PolyJuice find the optimization direction of the images?", "was_revised": true, "final_justification": "I think the auhors need to improve the overall presentation of the whole paper.", "rebuttal": "---\n\n---\n\n# Summary of the Review & Response\n\n---\n\n---\n\nWe thank the reviewer for their thoughtful comments. We are encouraged that the reviewer finds our empirical results strong (S1), our problem domain to be of critical importance (S2), and our black-box threat model practical in the context of real-world settings (S3).\n\n**Concerns** by the reviewer include: \n\n**1)** certain stylistic choices made in the paper (W1)\n\n**2)** lack of a formal definition of the threat model (W2)\n\n**3)** Potential confusion about the latent space and the computation of the steering directions (W1 & W3)\n\n**We address**\n\n**1)** by explaining our rationale behind our stylistic choices\n\n**2)** by formally defining our threat model\n\n**3)** by providing clarifying details\n\n---\n\n---\n\n# Response to Weaknesses\n\n---\n\n---\n\n> **W1a. The writing and the general structure of this paper needs to be improved, including how the latent space shifting helps to evade the detection.**\n> \n\nWe explain in Section 3.1 how PolyJuice identifies a direction in the T2I latent space that statistically correlates with the SID’s prediction of “realness”. This is based on a distribution shift between the latents of samples predicted as real versus fake (see Fig. 1b). By universally steering the latent in this direction during the image generation process (Fig. 2), we create samples that are more likely to land in SID failure regions (as further visualized in Fig. 5 and Fig. 7).\n\n---\n\n> **W1b. I saw too many unnecessary bolded text in the paper, which may reduce the readability of the paper.**\n> \n\nWe put these short, bold sentences at the beginning of the paragraphs so that a first-time reader can quickly skim the whole paper and get a general idea of the paper. We would appreciate it if the reviewer could point to the exact places where the bold text is reducing the readability of the paper, so we can address them in the revised version. \n\nWe hope these clarifications make the core mechanism of PolyJuice more accessible and improve the overall readability of the paper.\n\n---\n\n> **W2. Threat Model is Missing: I didn't find the threat model for the proposed attacks in this paper, including the description to the attacker's capability, goal and attack scenarios.**\n> \n\nWe are encouraged that in item 3 of the strength section, the reviewer finds our threat model practical and applicable to real-world scenarios.  We adopt the suggestion of the reviewer and formally define the threat model for our method in the following.\n\n*We extend the notation defined in Sec. 2 of the paper to our threat model.*\n\n**Threat Model:**\n\n- **Attacker’s Goal:**\nGiven a text prompt $c$ and a latent $z$, the attacker aims to generate synthetic images using a text-to-image (T2I) generative model $G:\\mathcal{Z}\\times\\mathcal{C}\\rightarrow\\mathcal{X}$ that deceives a target synthetic image detector (SID) $f:\\mathcal{X}\\rightarrow[0,1]$ into misclassifying them as real (class $0$).\n- **Attacker’s Capability:**\n    - **Black-box access to the SID:** The attacker can only query the SID and observe hard labels $Y$ (real/fake), without access to model weights or gradients.\n    - **Full access to the T2I generative model:** The attacker can manipulate the latent space $Z$ and control the generation process of a text-to-image (T2I) model.\n    - **Sufficient number of queries:** The attacker can **generate a dataset** of image-latent-label triplets to analyze the SID's behavior in response to various inputs.\n- **Attack Scenario:**\n    - **Step 1:** The attacker queries the black-box SID with fake images and obtains hard labels, constructing a dataset of TP and FN samples.\n    - **Step 2:** The attacker pre-computes steering directions in the T2I latent space that correlate with increased probability of being misclassified as real by the SID (Eq. 3).\n    - **Step 3:** At test time, the attacker applies this universal direction to arbitrary prompts, producing images that evade detection by the SID (Eq. 6).\n\nWe will add this formal threat model to the revised version of the paper.\n\n---\n\n> **W3a. …Is the latent space of vae or just the embedding space in CLIP?**\n> \n\nThe latent space $Z$ mentioned in the paper is the latent space of the VAE used by a latent diffusion/flow-based T2I model, and not the embedding space of a CLIP model. \n\n---\n\n> **W3b. …how PolyJuice find the optimization direction of the images?**\n> \n\nWe first form a set of true positive (TP) and false negative (FN) images using hard labels $Y$ obtained from the black box SID. Our goal is to find the direction of the distribution shift from TP to FN (see Fig. 1b) in each image generation step. These directions are estimated by maximizing the statistical dependence (measured through HSIC) between the latents $Z$ and the labels $Y$, i.e., calculating SPCA. This optimization has a closed-form solution (L121 - L123).  At inference time, these directions are used to steer the trajectory of the T2I model (Eq. 6) such that the final generated image is guided towards high error rate regions (see Fig. 2).\n\n---\n\n---\n\n**Please consider raising your score if we have addressed your concerns. If you have additional questions, we would be happy to answer them.**\n\n---\n\n---"}, {"review_id": "2b845nUePM", "reviewer": "Reviewer_9sRv", "summary": "This papers proposes a novel method attacking synthetic image detectors, named PolyJuice. By constructing a dataset of T2I embedded images labelled as true positive (correctly predicted as fake) and false negative (falsely predicted as real), the method constructs a direction vectors of maximal change between TP and FN samples. During the attack, these direction vectors are applied to the T2I latent space, resulting in SID models false predicting synthetic images as real. \n\nBy conducting various experiments, the authors demonstrate the ability of PolyJuice to enhance the ability of T2I models in generated images that prevent detection from SID methods.", "questions": "The work is novel and of a high-quality. Please see my weaknesses above for my questions.", "limitations": "The authors touch upon the limitation of their method, and suggest a direction of future research to address it.\n\nThe authors mention that the motivation of this work is for responsible red-teaming, however more discussion should be had on how to mitigate the risk of such adversarial images. For instance, the authors visualise the CLIP embeddings of both PolyJuice and non-PolyJuice aided synthetic images, which show two clear clusters of images, could this be used to develop some mitigation strategy? \n\nLastly, a discussion of how the result of this work can influence policy should be had, even if only brief. I.e. The authors mention that images generated by their method seem to consist of warmer colours? Could such aspects of images help identify synthetic images which could be written into a policy?", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper proposed a novel approach at attacking SID models by only using T2I embeddings of TP and FP images.\n- The paper is well written, clearly highlighting the importance of this research direction and the limitations it addresses.\n- The proposed method is able to achieve high success rates in attacking SID models. \n- The authors make use of multiple SID and T2I method within their evaluation, as-well as investigations in the impact of PolyJuice on T2I models.\n\nWeaknesses:\n - The authors only make use of the COCO datasets. Despite them providing reasons for only using this dataset, it would still be useful to make use of other datasets, or even using text prompt outside the COCO dataset textual descriptions for comparison.\n - The performance of the method is highly dependent on the ability of SPCA (which the authors highlight when transferring direction from low-res to high-res images, i.e. curve of dimensionality causes performance issues), another variable that may cause issues with the final produced direction vector is the side of the dataset. The authors do not conduct any ablation study on the impact of the number of TP & FN has on the performance of PolyJuice. \n- Further to the above point, could the authors give more detail of how they constructed the 20k TP and 20k FN images? The writing suggests they are generated by the targeted SID model? However more information of how they generated those images would benefit this paper.\n- The authors defence mechanism involves the use of PolyJuice images to calibrate SID models, could the authors also train a SID classifier using PolyJuice images within the dataset and evaluate its robustness.\n - The paper lacks more discussion on the defence methods against this attack, which raises ethical questions.", "was_revised": true, "final_justification": "The authors have well addressed my concerns and questions which has increased my confidence in this paper. However, I do not believe this work and the authors response necessitated a higher rating (from accept to strong accept).", "rebuttal": "---\n\n---\n\n# Summary of the Review & Response\n\n---\n\n---\n\nWe greatly appreciate the reviewer's positive assessment of our work. The reviewer finds our approach novel (S1), our paper well written, well motivated (S2), and our work to be novel and of high quality in general (Q).  We are glad that the reviewer identifies our usage of multiple SID and T2I methods in evaluation (S4), and the overall high success rate of our black-box method (S3) as strengths. \n\n**Primary concerns** by the reviewer include: \n\n**1)** extension to other text-description datasets (W1)\n\n**2)** ablation on the number of samples needed to compute PolyJuice directions (W2 & W3)\n\n**3)** potential defense mechanisms (W4 & W5 and Lm1 &Lm2)\n\n**We address:** \n\n**1)** by extending the evaluation to a new text description dataset (PartiPrompts) in **Table R5**\n\n**2)** by providing ablation results on the effect of the number of samples on the success rate and clarifying the pipeline for selecting these samples (**Table R6**)\n\n**3)** by discussing the potential defense mechanism that will be added to the final version of the manuscript (**Table R7**)\n\nIn the following, we address each of the concerns raised by the reviewer in detail.\n\n---\n\n---\n\n# Response to Weaknesses\n\n---\n\n---\n\n> **W1.** **The authors only make use of the COCO datasets. Despite them providing reasons for only using this dataset, it would still be useful to make use of other datasets, or even using text prompt outside the COCO dataset textual descriptions for comparison.**\n> \n\nWe thank the reviewer for their suggestion, which improves the breadth of the paper. As noted in Sec. 4, we use COCO as it contains diverse text prompts that cover different domains, such as humans, animals, natural scenery, and common objects. This ensures the training data is not domain-specific. Per the reviewer’s suggestion, we evaluate PolyJuice attacks (directions learned from COCO) on a subset of text prompts from the PartiPrompts dataset (Yu et al, 2022) and present the results in **Table R5**. The results demonstrate the generalizability of the PolyJuice attacks to text prompts outside COCO.\n\n**Table R5.**  Attack success rate (%) of PolyJuice on text descriptions from the PartiPrompts dataset.\n\n| **T2I** | **Detector** | **Unsteered** | **PolyJuice (ours)** |\n| --- | --- | --- | --- |\n| SD3.5 | UFD | 13 | 75   **(+62)** |\n| SD3.5 | RINE | 8 | 100 **(+92)** |\n| Flux-Dev | UFD | 78 | 96   **(+18)** |\n| Flux-Dev | RINE | 42 | 86   **(+44)** |\n| Flux-Schnell | UFD | 61 | 84   **(+23)** |\n| Flux-Schnell | RINE | 31 | 56   **(+25)** |\n\n---\n\n> **W2. …The authors do not conduct any ablation study on the impact of the number of TP & FN has on the performance of PolyJuice.**\n> \n\nTo address this concern, we estimate the shift directions using 16K, 40K, and 52K data samples from FLUX[dev] and use them to perform attacks against UFD. The results presented in **Table R6** demonstrate that approximating the steering directions with 16K data points yields the same result as the 40K we use in the paper (SR of  96.3%). Using more data points (52K) leads to a very small increase in success rate (SR of 96.5%), which implies the low sensitivity of PolyJuice SR to the number of training samples. This discussion and results will be added to the revised paper.   \n\n**Table R6.** Effect of the number of samples on PolyJuice's success.\n\n| **Training Samples** | **Success Rate** |\n| --- | --- |\n| 16K | 96.3 |\n| 40K | 96.3 |\n| 52K | 96.5 |\n\n---\n\n> **W3 Further to the above point, could the authors give more detail of how they constructed the 20k TP and 20k FN images?  …**\n> \n\nFirst, using a T2I model (e.g. FLUX), we generate a set of fake images using the text prompts available in the COCO training set. These fake images are then labeled by a given black-box SID as fake (TP) or real (FN). To have a balanced dataset, we sample an equal number (up to 20K) of FN and TPs from the labeled data. We will further clarify these details in the experiment settings subsection of Sec. 4 in the revised paper. \n\n---\n\n> **W4. The authors defence mechanism involves the use of PolyJuice images to calibrate SID models, could the authors also train a SID classifier using PolyJuice images within the dataset and evaluate its robustness.**\n> \n\nWe would like to note that the calibration strategy used in the paper is based on EER (Equal Error Rate), which is a widely accepted method in the literature. Per the reviewer’s suggestion, we take the RINE detector and fine-tune it (by training an MLP head) using the attacks generated by PolyJuice. We then compare the detection rate of the original RINE, PolyJuice-calibrated RINE (Sec 4.3, Table 3), and the PolyJuice-trained RINE. **Table R7** shows the results of the comparison, which demonstrate the contribution of PolyJuice in improving the defense of the SID (lowering FNR). From the result, we also find that the calibration of SIDs leads to a better FNR than fine-tuning. \n\n**Table R7.** Comparison of the FNR of the original RINE, PolyJuice-calibrated RINE, and PolyJuice-trained RINE.\n\n|  | **RINE** | **PolyJuice-Cal. RINE** | **PolyJuice-trained RINE** |\n| --- | --- | --- | --- |\n| **SD3.5** | 15.1 | 3.8 | 6.9 |\n| **FLUX[dev]** | 52.0 | 21.8 | 30.3 |\n| **FLUX[sch]** | 39.6 | 18.4 | 22.6 |\n\n---\n\n> **W5. The paper lacks more discussion on the defence methods against this attack, which raises ethical questions.**\n> \n\nWe thank the reviewer for identifying this concern. First, PolyJuice requires a certain number of queries to the SID in order to construct the dataset and compute the steering direction. Therefore, commonly adopted security barriers, such as a rate limits, can raise the difficulty of applying PolyJuice maliciously. Apart from security barriers, a model can become robust to PolyJuice by learning a representation space that collapses the TP and FN distributions. For more details, please refer to the response to L1.\n\nIn summary, while we acknowledge the potential for misuse, we believe that raising awareness of this vulnerability ultimately benefits the community. By highlighting the issue, our work can inspire further research into developing effective mitigation strategies.\n\n---\n\n---\n\n# Response to Limitations\n\n---\n\n---\n\n> **L1. …For instance, the authors visualise the CLIP embeddings of both PolyJuice and non-PolyJuice aided synthetic images, which show two clear clusters of images, could this be used to develop some mitigation strategy?**\n> \n\nOne possible mitigation strategy can be training a head on top of the CLIP feature space that collapses the FN and TP clusters. We want to find a subspace that is orthogonal to the direction of shift between these two clusters, while preserving the information of the target attribute (i.e., real vs. fake). Ideas from invariant representation learning, such as AIFL (Xie et al. 2017) or FairerCLIP (Dehdashtian et al. 2024), can be adopted for this purpose. Specifically, in the notation of FairerCLIP, the target $Y$ would be real vs. fake label, while the attribute to be removed, $S$, would be TP vs. FN. This discussion can enrich the paper, so it will be added to the revised version of the paper. \n\n---\n\n> **L2. Lastly, a discussion of how the result of this work can influence policy should be had, even if only brief. I.e. The authors mention that images generated by their method seem to consist of warmer colours? Could such aspects of images help identify synthetic images which could be written into a policy?**\n> \n\nWe would like to clarify that the common pattern analyzed in Section B.1 of the supplementary material is SID-specific and is not generalizable to other SIDs, since the vulnerability of each model is different. As a result, PolyJuice needs to be applied on each SID to find the vulnerability of the model, and this is not necessarily the same for other detectors. However, a per-SID policy can be written based on the common patterns.\n\n---\n\n---\n\n**Please consider raising the scores if we address your concerns. If you have more questions, we would be happy to answer them.**\n\n---\n\n---\n\n# **Reference**\n\nYu et al. \"Scaling autoregressive models for content-rich text-to-image generation.\" TMLR (2022).\n\nXie, Qizhe, et al. \"Controllable invariance through adversarial feature learning.\" NeurIPS (2017).\n\nDehdashtian et al. \"Fairerclip: Debiasing clip's zero-shot predictions using functions in rkhss.\" ICLR (2024)."}]

Author rebuttal:

Meta-review: ### Paper summary
This paper introduces a black-box red-teaming method designed to deceive synthetic image detectors. The core idea is to identify a discriminative direction within the latent space of a generative model, a vector separating real from synthetic characteristics, and then steer the image generation process along this vector. This steering creates outputs that are more effective at fooling the detectors. The authors demonstrate through experiments that their method is not only robust but also transferable across different detectors, making it a valuable tool for auditing and understanding the vulnerabilities of such systems.

### Meta-Review and Recommendation
The review process for this paper was highly constructive. The reviewers initially raised valid points regarding the need for additional experiments, clarity, and the scope of the claims. The authors' detailed rebuttal successfully addressed most of these concerns, leading to consensus for acceptance (rating it borderline accept or higher). One of the remaining issues is the measurement of the image quality; the authors have committed to including metrics like FID in the final version. 
Given the paper's contributions and the effective resolution of all major concerns during the discussion period, the AC recommends an  Accept for the paper. The work represents a solid and timely contribution, and the authors have shown they are committed to delivering a polished final manuscript.
Decision: Accept (poster)