Title: PolyJuice Makes It Real: Black-Box, Universal Red Teaming for Synthetic Image Detectors
Venue: NeurIPS 2025
Accepted: yes
Keywords: Red Teaming

Diffusion Models

Generative Models

Synthetic Image Detection

Text-to-Image Models

Concept Steering
Abstract: Synthetic image detectors (SIDs) are a key defense against the risks posed by the growing realism of images from text-to-image (T2I) models. Red teaming improves SID’s effectiveness by identifying and exploiting their failure modes via misclassified synthetic images. However, existing red-teaming solutions (i) require white-box access to SIDs, which is infeasible for proprietary state-of-the-art detectors, and (ii) generate image-specific attacks through expensive online optimization. To address these limitations, we propose PolyJuice, the first black-box, image-agnostic red-teaming method for SIDs, based on an observed distribution shift in the T2I latent space between samples correctly and incorrectly classified by the SID. PolyJuice generates attacks by (i) identifying the direction of this shift through a lightweight offline process that only requires black-box access to the SID, and (ii) exploiting this direction by universally steering all generated images towards the SID’s failure modes. PolyJuice-steered T2I models are significantly more effective at deceiving SIDs (up to 84%) compared to their unsteered counterparts. We also show that the steering directions can be estimated efficiently at lower resolutions and transferred to higher resolutions using simple interpolation, reducing computational overhead. Finally, tuning SID models on PolyJuice-augmented datasets notably enhances the performance of the detectors (up to 30%).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis papers proposes a novel method attacking synthetic image detectors, named PolyJuice. By constructing a dataset of T2I embedded images labelled as true positive (correctly predicted as fake) and false negative (falsely predicted as real), the method constructs a direction vectors of maximal change between TP and FN samples. During the attack, these direction vectors are applied to the T2I latent space, resulting in SID models false predicting synthetic images as real. \n\nBy conducting various experiments, the authors demonstrate the ability of PolyJuice to enhance the ability of T2I models in generated images that prevent detection from SID methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper proposed a novel approach at attacking SID models by only using T2I embeddings of TP and FP images.\n- The paper is well written, clearly highlighting the importance of this research direction and the limitations it addresses.\n- The proposed method is able to achieve high success rates in attacking SID models. \n- The authors make use of multiple SID and T2I method within their evaluation, as-well as investigations in the impact of PolyJuice on T2I models.\n\nWeaknesses:\n - The authors only make use of the COCO datasets. Despite them providing reasons for only using this dataset, it would still be useful to make use of other datasets, or even using text prompt outside the COCO dataset textual descriptions for comparison.\n - The performance of the method is highly dependent on the ability of SPCA (which the authors highlight when transferring direction from low-res to high-res images, i.e. curve of dimensionality causes performance issues), another variable that may cause issues with the final produced direction vector is the side of the dataset. The authors do not conduct any ablation study on the impact of the number of TP & FN has on the performance of PolyJuice. \n- Further to the above point, could the authors give more detail of how they constructed the 20k TP and 20k FN images? The writing suggests they are generated by the targeted SID model? However more information of how they generated those images would benefit this paper.\n- The authors defence mechanism involves the use of PolyJuice images to calibrate SID models, could the authors also train a SID classifier using PolyJuice images within the dataset and evaluate its robustness.\n - The paper lacks more discussion on the defence methods against this attack, which raises ethical questions.\n\nQUESTIONS:\nThe work is novel and of a high-quality. Please see my weaknesses above for my questions.\n\nLIMITATIONS:\nThe authors touch upon the limitation of their method, and suggest a direction of future research to address it.\n\nThe authors mention that the motivation of this work is for responsible red-teaming, however more discussion should be had on how to mitigate the risk of such adversarial images. For instance, the authors visualise the CLIP embeddings of both PolyJuice and non-PolyJuice aided synthetic images, which show two clear clusters of images, could this be used to develop some mitigation strategy? \n\nLastly, a discussion of how the result of this work can influence policy should be had, even if only brief. I.e. The authors mention that images generated by their method seem to consist of warmer colours? Could such aspects of images help identify synthetic images which could be written into a policy?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces **PolyJuice**, a novel black-box and image-agnostic red-teaming method designed to improve the effectiveness of Synthetic Image Detectors (SIDs). Recognizing the limitations of existing red-teaming solutions—which often require white-box access to proprietary SIDs and rely on computationally expensive, image-specific optimizations—PolyJuice offers a universal approach. The core insight behind PolyJuice is the observation of a discernible distribution shift in the latent space of Text-to-Image (T2I) models between images correctly classified by an SID and those misclassified (i.e., synthetic images that the SID erroneously identifies as \"real\").\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n**1. Strong Empirical Results:** I trust this paper provides compelling quantitative results, showing PolyJuice's effectiveness in boosting attack success rates (up to 84%) and, conversely, its utility in improving SID robustness (up to 30% reduction in FNR) when used for data augmentation.\n\n**2. Addresses a Critical Gap in Real-world:** PolyJuice directly tackles the pressing issue of the \"arms race\" between rapidly advancing T2I models and the SIDs designed to counter them. Its ability to generate challenging, misclassified synthetic images is vital for evolving SID capabilities. The black-box nature of PolyJuice is highly significant for real-world applications, as many advanced SIDs are proprietary and only offer API access, making traditional white-box attacks infeasible.\n\n**3. Practical Threat Model:** This paper employs a practical **black-box** threat model, which is more practical than the previous settings. I think this is important for discuss it in real-world applications.\n\n**Weaknesses**\n\n\n**1. Strong Empirical Results:** The writing and the general structure of this paper needs to be improved, including how the latent space shifting helps to evade the detection. I saw too many unnecessary bolded text in the paper, which may reduce the readability of the paper.\n\n**2. Threat Model is Missing:** I didn't find the threat model for the proposed attacks in this paper, including the description to the attacker's capability, goal and attack scenarios. \n\n**3. Confused about the perturbation optimization:** If I didn't get it wrong, PolyJuice first get the hard predictions from the querying to black-box APIs, and it formulate a distribution with latent space in t2i models. I am confusing about the definition of of latent space here. Is the latent space of vae or just the embedding space in CLIP? And how PolyJuice find the optimization direction of the images?\n\nQUESTIONS:\nSee weakness please.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces **PolyJuice**, a universal red-teaming method that steers text-to-image generators toward regions that fool synthetic-image detectors (SIDs) while requiring only black-box label access. \nIt computes a single \\\"realness shift\\\" direction per diffusion timestep via supervised PCA on latent vectors of true-positive vs. false-negative samples, then applies this vector during sampling. \nThe pre-computed directions transfer across prompts and resolutions, enabling query-efficient, image-agnostic attacks. \nExperiments with three generators (SD v3.5, FLUX-dev, FLUX-sch) and two SIDs (UFD, RINE) show up to 84 p.p. increase in false-negative rate and demonstrate that augmenting training data with PolyJuice images can cut detector errors by up to 30 p.p.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n- **Novelty**: Black-box, *universal* unrestricted attack on SIDs leveraging a latent-space distribution shift.\n- **Efficiency**: One-off offline direction discovery and low→high-resolution transfer avoid costly per-image optimization.\n- **Effectiveness**: Large gains in SID false-negative rates across multiple generators and resolutions.\n- **Practical utility**: PolyJuice images improve detector robustness when added to training data.\n- **Clarity**: Motivation, algorithm and experiments are explained with helpful figure. \n \n# Weaknesses\n- **Baseline Coverage**: No comparison to prior black-box or transfer-based UA attacks; only \\\"no-steering\\\" baseline. A quick search revealed Kotyan et al 2024, proposing a black-box attack using evolutionary algorithm. I think the authors should conduct a more thorough literature survey and conduct a better quantitative study to show their attack's effectiveness over other black-box attacks. Infact, I would even urge to compare against White-Box Attacks to quantify the gap between White-Box and Black-Box Attacks, Good black-box attacks should minimize the performance gap. Simply showing effectiveness of their own approach by being ignorant and neglecting existing works do not give a good baseline comparision. \n- **Limited Detectors**: Evaluation covers just two academic SIDs that are not even state-of-the art SID. Once again a shallow literature survey from this perspective is revealed, a quick search clearly indicates UFD is not state-of-the-art, Baraldi et al. 2024 (DRCT) and Chen et al. (CoDE) are much general SID models and would test the capabilities of PolyJuice better. A bad detector will be easy to fool, but a strong general detector will be harder. It is imperative to test and validate the attack against SOTA SID models.\n- **Image Quality Metrics**: While the authors conduct evaluation using performance, lack of reporting of image quality of adversarial samples and comparision to non-adversarial samples cast a doubt over claims indicated in the article. Zhong et al 2023 demonstrate that performance of SID models decrease when image quality is affected, therefore it is impertative to check and validate that image quality of adversarial samples are not degraded, if image quality is degraded, then the claims of the article is wrong as PolyJuice is greedily fooling SID by decreasing the image quality rather than directing image-generation models towards the blind-spot. Slighty degraded images compared to non-degraded non-adversarial images also point to distribution shift affecting the SID models not explictly trained for this distrubution shift. \n- **Potential Gaps in Theroetical Claims and Assumptions**: Latent Size of Modern diffusion models are far bigger than the sample size so in this “large-p, small-n” regime, the sample covariance is necessarily rank-deficient, so the eigenvectors are extremely sensitive to noise. No formal discussion of query noise or finite-sample error; suggest adding concentration inequality or error bound.As a result, the steering direction may capture sampling artefacts (like poor image quality) rather than a genuine class-separating shift. Further for steering direction, no proof is supplied that this convex combination maximises HSIC in the directional (rank-1) setting; the classical SPCA solution would instead select the single leading eigenvector. A short lemma is needed to justify why the proposed weighting improves—or even preserves—the HSIC objective.\n- **Reverse Process Validity**: Adding $\\lambda_t \\delta_t$ at each step perturbs the reverse posterior. The proof lacks a guarantee that the perturbed trajectory remains within the manifold on which the score network was trained, raising the possibility of off-manifold artefacts that are trivially hard for the detector. Samuel et. al 2023 showed that if latent vector is manipulated much such that their norm changes than the image quality decreases reiterating that images generated might be degraded compared to original samples leading to misclassification by SID.\n- **Ablations for Hyperparameter Sensitivity**: Any point of hyperparameter is relegated to appendix; however no ablations are conducted to validate the selection of the hyper-parameters. An ablation study can better validate these choices. \n- **Overarching Red-Teaming Claims**: The authors suggest that the attack proposed by them can be used as a red-teaming tool. The main purpose of a red-teaming tool is to identify an actionable vulnerability that can be used to improve robustness. Current article, only proposes an adversarial attack, it hypothesizes on the vulnerability and make no contributions in improving the defence of the SIDs or show evidence tfor the same. Therefore, I think the claim is overarching and should be avoided. The contribution is simply a black-box adversarial attack on SID models. \n- **Untested Commercial Claims** - Authors highllight that the method can be used on commerical SID models but lacks tests to validate it. To test true \"black-box\" claim, it is essential to apply the attack to commercial SID models like Sightengine (https://dashboard.sightengine.com/ai-image-detection) and AI or Not: (https://www.aiornot.com/dashboard/home) and demonstrate the capability, otherwise I urge the authors strongly to not overexaggerate their claims and refrain from making unwarranted claims.\n\nIn general, I find two major limitations in terms of literature survey and overexaggerated claims;\n- Literature survey of the article highly unsatisafactory and outdated in 2025, both from the point of black-box attack using Diffusion Models and choosing Synthetic Image Detectors for the experiments, failing to account many studies that came in 2024. \n- Claims made in the article are not accounted and rather boldly presented as contributions. \n\n# Comment on Scope (Won't term it weakness but definitely very narrow scope impacting significance)\nThe proposed does not use domain information specific to SID yet it's application was limited to SID. I would appreciate if authors can comment upon why narrow domain was targetted, when the proposed approach can be used for any image classfiication tasks? Object Classification, NSFW Classifcation? If the authors can demonstrate this, it would raise the significance and applicability of this research to a broader community. Further even in SID, specicialised domain like Facial Images are not evaluated. Recently, Guo et al 2025 proposed an interpretable SID using LLM showing a shift of SID task from Binary Clasification to more complex task. It would be nice to see if the proposed attack can also fool such latest models diverging SID from simple binary classification. \n\n## References\n- Samuel, D., Ben-Ari, R., Darshan, N., Maron, H., & Chechik, G. (2023). Norm-guided latent space exploration for text-to-image generation. Advances in Neural Information Processing Systems, 36, 57863-57875.\n- Zhong, N., Xu, Y., Li, S., Qian, Z., & Zhang, X. (2023). Patchcraft: Exploring texture patch for efficient ai-generated image detection. arXiv preprint arXiv:2311.12397. https://fdmas.github.io/AIGCDetect/data/\n- Baraldi, L., Cocchi, F., Cornia, M., Baraldi, L., Nicolosi, A., & Cucchiara, R. (2024, September). Contrasting deepfakes diffusion via contrastive learning and global-local similarities. In European Conference on Computer Vision (pp. 199-216). Cham: Springer Nature Switzerland.\n- Chen, B., Zeng, J., Yang, J., & Yang, R. (2024, July). Drct: Diffusion reconstruction contrastive training towards universal detection of diffusion generated images. In Forty-first International Conference on Machine Learning.\n- Kotyan, S., Mao, P. Y., Chen, P. Y., & Vargas, D. V. (2024). Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models!. arXiv preprint arXiv:2402.04699.\n- Guo, X., Song, X., Zhang, Y., Liu, X., & Liu, X. (2025). Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 105-116).\n\nQUESTIONS:\nSee mainly weaknesses for longer discussions but would appreicate crisp answes to the following;\n\n- What is the minimum number of SID queries needed to approximate the steering direction within 5 pp of reported success rates?\n- How does PolyJuice compare to a transfer-based black-box attack (e.g., DiffPGD trained on a surrogate SID)?\n- Have you evaluated on non-COCO domains like, face-focused, medical or multi-modal detectors? \n- How sensitive are results to the dimensionality *d* of the SPCA subspace and to the choice of λₜ?\n- Could non-linear kernels in SPCA further improve attack success? Any preliminary experiments?\n\nLIMITATIONS:\n- No discussion of query-based cost constraints or rate-limiting defenses at commercial endpoints.(as the claim in fooling even commercial APIs)\n- Highly limited literature survey neglecting latest SOTA models on SID and black-box attacks using Diffusion Models\n- Does not address potential legal/ethical implications of releasing universal attack directions.\n- Assumes access to clean TP/FN split; impact of noisy SID labels is not explored.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces PolyJuice, a novel and versatile black-box red-teaming framework designed to launch unrestricted adversarial attacks against synthetic image detectors (SIDs). Unlike prior methods that rely on white-box access or image-specific optimization, PolyJuice identifies and exploits systematic weaknesses in SIDs by discovering a universal shift in the T2I latent space. By leveraging this insight, PolyJuice can efficiently steer generated images toward SID failure modes, significantly increasing attack success rates. Moreover, augmenting training datasets with PolyJuice-generated examples leads to notable improvements in SID robustness, with detection accuracy increasing by up to 30%.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The manuscript is generally well-written and easy to follow, with clear motivation and logical flow.\n2. The topic of improving the robustness of AIGC image detection addresses an urgent and practical challenge in the era of powerful text-to-image models.\n3. The proposed method is novel in that it introduces a black-box, image-agnostic red-teaming strategy by exploiting latent space distribution shifts — a departure from typical white-box or per-image attack strategies.\n4. The method not only enables effective attacks but also contributes positively by improving detector robustness when used for data augmentation.\n\nWeaknesses：\n1. It is unclear whether the observed latent space distribution shift (between correctly and incorrectly classified images) is consistent across different detectors. If the distribution shift is detector-specific, how transferable or generalizable is the proposed PolyJuice approach?\n2. The method assumes that a linear direction in latent space can universally guide images towards detector failure modes. Is this assumption valid across diverse prompts and image categories? A discussion or ablation study would help clarify.\n3. While the method improves attack success rates, it is important to analyze whether the manipulated/generated images remain visually realistic and prompt-faithful. Otherwise, there’s a risk that PolyJuice may just generate degraded or unrealistic images that trivially evade detection.\n4. The authors evaluate PolyJuice primarily on UFD and RINE as target synthetic image detectors. However, it remains unclear how well the method generalizes to more recent and potentially more robust detectors such as FatFormer (CVPR 2024) and NPR (CVPR 2024).\n\nQUESTIONS:\n1. Please refer to Weaknesses\n2. Is the observed latent space distribution shift consistent across different synthetic image detectors? Specifically, does PolyJuice need to re-estimate the steering direction for each SID individually, or can the same direction be transferred between detectors?\n3. How does the latent steering affect the visual quality and semantic consistency of the generated images? Are there metrics or human evaluations to ensure the attack does not simply degrade image realism?\n\nLIMITATIONS:\nThe proposed method relies on identifying a latent space shift that separates correctly and incorrectly classified samples for a specific detector. This shift may not generalize across detectors, potentially limiting the reusability of the learned steering direction.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-05"}