Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approach (2025)

Abstract: Prompt trading has emerged as a significant intellectual property concern in recent years, where vendors entice users by showcasing sample images before selling prompt templates that can generate similar images. This work investigates a critical security vulnerability: attackers can steal prompt templates using only a limited number of sample images. To investigate this threat, we introduce \textbf{Prism} , a prompt-stealing benchmark consisting of 50 templates and 450 images, organized into Easy and Hard difficulty levels. To identify the vulnerabity of VLMs to prompt stealing, we propose \textbf{EvoStealer} , a novel template stealing method that operates without model fine-tuning by leveraging differential evolution algorithms. The system first initializes population sets using multimodal large language models (MLLMs) based on predefined patterns, then iteratively generates enhanced offspring through MLLMs. During evolution, EvoStealer identifies common features across offspring to derive generalized templates. Our comprehensive evaluation conducted across open-source (InternVL2-26B) and closed-source models (GPT-4o and GPT-4o-mini) demonstrates that EvoStealer’s stolen templates can reproduce images highly similar to originals and effectively generalize to other subjects, significantly outperforming baseline methods with an average improvement of over 10%. Moreover, our cost analysis reveals that EvoStealer achieves template stealing with negligible computational expenses. Our code and dataset are available at https://whitepagewu.github.io/evostealer-site.

## introduction recent advancements in text-to-image generation @xcite , particularly in multimodal large language models * equal contribution. ## related work we discuss two lines of related work: the text-toimage prompt stealing attacks and the evolutionary algorithms in llms. ## data consturction in this section, we introduce the threat model of prompt template stealing, providing a detailed description of the attacker's existing conditions, constraints, and objectives. we then detail our methodology for developing prism, a comprehensive benchmark designed to realistically simulate this attack scenario. the specifics are presented below. ## evostealer in this section, we introduce the three main steps of evostealer: image element extraction, differential evolution, and fitness function. the details are presented below. ## experiments we employ prism to evaluate the vulnerability of image generation models to prompt template stealing. following recent works @xcite @xcite , we employ subject similarity, style similarity, and semantic similarity metrics to evaluate the performance of image generation models against prompt template stealing (section 5.3). these metrics demonstrate higher agreement with human annotations than previous approaches. additionally, we conduct human evaluation to measure the quality of prompt stealing. ## analysis in this section, we analyze the effects of evostealer's components, the iteration number, and the experimental costs. ## case study to clearly demonstrate evostealer's advantages over baseline methods, we select an easy and a hard example for case study, with the results shown in figure 4 . the results show that, on in-domain data, evostealer generates images that closely match the style of the original images, with all four synthesized images maintaining stylistic consistency. in contrast, the four images generated by the other baseline methods exhibit significant style variation. on out-of-domain data, evostealer maintains the same style as in-domain images, successfully achieving subject generalization. in contrast, the other baseline methods fail to generalize. additionally, we analyze three distinct failure cases (see appendix g for details). ## conclusion this paper investigates prompt template stealing-whether attackers can extract generalizable templates that maintain stylistic consistency using minimal sample images. to explore this scenario, we provide prism, a two-tier benchmark consisting of 50 templates and 450 images, organized into easy and hard difficulty levels. we also intro-duce evostealer, a template stealing method that combines differential evolution algorithms with mllms, enabling template stealing without the need for fine-tuning. extensive experiments and analysis validate its effectiveness and practicality.