FC -Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowcharts (2025)

Abstract: Multimodal Large Language Models (MLLMs) have become powerful and widely adopted in some practical applications.However, recent research has revealed their vulnerability to multimodal jailbreak attacks, whereby the model can be induced to generate harmful content, leading to safety risks. Although most MLLMs have undergone safety alignment, recent research shows that the visual modality is still vulnerable to jailbreak attacks.In our work, we discover that by using flowcharts with partially harmful information, MLLMs can be induced to provide additional harmful details. Based on this, we propose a jailbreak attack method based on auto-generated flowcharts, FC-Attack.Specifically, FC-Attack first fine-tunes a pre-trained LLM to create a step-description generator based on benign datasets.The generator is then used to produce step descriptions corresponding to a harmful query, which are transformed into flowcharts in 3 different shapes (vertical, horizontal, and S-shaped) as visual prompts.These flowcharts are then combined with a benign textual prompt to execute the jailbreak attack on MLLMs.Our evaluations on Advbench show that FC-Attack attains an attack success rate of up to 96% via images and up to 78% via videos across multiple MLLMs.Additionally, we investigate factors affecting the attack performance, including the number of steps and the font styles in the flowcharts. We also find that FC-Attack can improve the jailbreak performance from 4% to 28% in Claude-3.5 by changing the font style.To mitigate the attack, we explore several defenses and find that AdaShield can largely reduce the jailbreak performance but with the cost of utility drop.

## introduction with the advancement of large language models (llms), multimodal large language models (mllms) that integrate vision (images and videos) and text, such as gpt-4o @xcite and qwen2.5-vl @xcite , have demonstrated emergent abilities and achieved impressive performance on downstream tasks related to visual understanding @xcite . despite being powerful, recent studies @xcite have revealed that mllms are vulnerable to jailbreak attacks whereby the adversary uses malicious methods to bypass safeguards and gain harmful knowledge. such vulnerabilities pose remarkable safety risks to the internet and the physical world. for instance, in january 2025, the world witnessed the first case where chatgpt was used to conduct an explosion (the times, 2025). to better safeguard mllms and proactively address their vulnerabilities, model researchers make many efforts in this regard, such as @xcite providing a quantitative understanding regarding the adversarial vulnerability of mllms. previous studies often create adversarial datasets tailored to specific models, which tend to perform poorly on other models. currently, jailbreak attacks against mllms can be broadly categorized into two main types: optimization-based attacks @xcite and prompt-based attacks @xcite . optimization-based attacks use white-box gradient methods to craft adversarial perturbations on visual prompt aligned with harmful text. they are effective but slow and have limited transferability in black-box scenarios. in contrast, prompt-based jailbreaks require only black-box access and work by injecting malicious visual cues into benign prompts to exploit mllms' text-focused safety alignment. to better improve the attack transferability and its effectiveness, we propose a novel promptbased jailbreak attack, namely fc-attack. concretely, fc-attack converts harmful queries into harmful flowcharts (images and videos) as visual prompts, allowing users to input benign textual prompts to bypass the model's safeguards. specifically, fc-attack consists of two stages: (1) step-description generator building: in this stage, the step description dataset is synthesized using gpt-4o, and a pre-trained llm is fine-tuned to obtain a step-description generator. (2) jailbreak deployment: this stage uses the generator to produce steps corresponding to the harmful query and generates three types of harmful flowcharts (vertical, horizontal, and s-shaped) as visual prompts. together with the benign textual prompt, the visual prompt is fed into mllms to achieve the jailbreak. note that the harmful flowcharts are generated automatically without hand-crafted effort. our evaluation on the advbench dataset @xcite shows that fc-attack outperforms previous attacks and achieves an attack success rate (asr) of over 90% on multiple open-source models, including llava-next, qwen2-vl, and internvl-2.5, and reaches 94% on the production model gemini-1.5. although the asr is lower on gpt-4o mini, gpt-4o, and claude-3.5, we how later that it can be improved in certain ways. to further investigate the impact of different elements in flowcharts on the jailbreak effectiveness of mllms, we conduct several ablation experiments, including different types of user queries (as shown in figure 1 ), numbers of descriptions, and font styles in flowcharts. these experiments show that mllms exhibit higher safety in the text modality but weaker in the visual modality. moreover, we find that even flowcharts with a onestep harmful description can achieve high asr, as evidenced by the gemini-1.5 model, where the asr reaches 86%. furthermore, font styles in flowcharts also contribute to the asr increase. for instance, when the font style is changed from "times new roman" to "pacifico", the asr increases from 4% to 28% on the model with the lowest asr (claude-3.5) under the original style. to mitigate the attack, we consider several popular defense approaches, including llama-guard-3-11b-vision (meta llama, 2025), jailguard @xcite , adashield-s @xcite , and adashield-a @xcite . among them, adashield-a demonstrates the best defense performance by reducing the average asr from 58.6% to 1.7%. however, it also reduces mllm's utility on benign datasets, which calls for more effective defenses. overall, our contributions are as follows: ## threat model adversary's goal. the adversary's goal is to exploit attacks to bypass the protective mechanisms of mllms and access content prohibited by safety policies, e.g., openai's usage policy @xcite . this goal takes real-world scenarios into account, where adversaries manipulate the capabilities of mllms to easily acquire harmful knowledge and thereby commit criminal acts with minimal learning effort. these objectives pose severe societal impacts and risks to the model providers. adversary's capabilities. in this paper, we consider a black-box scenario where the adversary cannot directly access the model's structure, parameters, or output logits, but can only obtain the model's final output (texts). in this scenario, adversaries interact with the model through an api provided by the model owner. moreover, the interaction is limited to a single-turn conversation, ## our method in this section, we introduce the framework of fc-attack (as shown in figure 2 ), which consists of two stages: step-description generator building and jailbreak deployment. ## conclusion in this paper, we propose fc-attack, which leverages auto-generated flowcharts to jailbreak mllms. experimental results demonstrate that fc-attack achieves higher asr in both opensource and production mllms compared to other jailbreak attacks. additionally, we investigate the factors influencing fc-attack, including different types of user queries, the number of steps in flowcharts, and font styles in flowcharts, gaining insights into the aspects that affect asr. finally, we explore several defense strategies and demonstrate that the adashield-a method can effectively mitigate fc-attack, but with the cost of utility drop.