QG uard:Question-based Zero-shot Guard for Multi-modal LLM Safety (2025)

Abstract: The recent advancements in Large Language Models(LLMs) have had a significant impact on a wide range of fields, from general domains to specialized areas. However, these advancements have also significantly increased the potential for malicious users to exploit harmful and jailbreak prompts for malicious attacks. Although there have been many efforts to prevent harmful prompts and jailbreak prompts, protecting LLMs from such malicious attacks remains an important and challenging task. In this paper, we propose QGuard, a simple yet effective safety guard method, that utilizes question prompting to block harmful prompts in a zero-shot manner. Our method can defend LLMs not only from text-based harmful prompts but also from multi-modal harmful prompt attacks. Moreover, by diversifying and modifying guard questions, our approach remains robust against the latest harmful prompts without fine-tuning. Experimental results show that our model performs competitively on both text-only and multi-modal harmful datasets. Additionally, by providing an analysis of question prompting, we enable a white-box analysis of user inputs. We believe our method provides valuable insights for real-world LLM services in mitigating security risks associated with harmful prompts.

## introduction the recent advancements in llms have had a significant impact across various fields, from general domains to those requiring specialized knowledge @xcite . especially, multi-modal large language models (mllms) are capable of answering both specific and general user queries based on detailed reasoning and understanding of visual inputs @xcite @xcite . ## method we propose qguard, a simple yet effective safety guard model based on question prompting. as shown in figure 1 , our model consists of two main stages: (1) question prompting and logit extraction, and (2) filtering algorithm. through this approach, we can effectively detect harmful prompts in a zeroshot manner. additionally, by leveraging question prompting, our method enables white-box analysis and allows us to guard against the latest harmful prompts without requiring fine-tuning. ## experiments to evaluate the performance of our model, we conduct experiments on two tasks: the first is harmful prompt detection using text only, and the second is multi-modal harmful prompt detection involving both images and text. ## conclusion we propose a simple yet effective method using question prompting for detecting harmful prompts in a zero-shot manner. our approach leverages pretrained mllm without fine-tuning and classifies harmful prompts through guard questions, question prompting, and a filtering algorithm. experimental results show that our model outperforms fine-tuned baselines. the method also enables white-box analysis, providing transparency in classification. by refining guard questions, our approach can flexibly adapt to new harmful prompts with minimal computational overhead, making it a practical solution for real-world llm safety applications. we believe that our approach presents a practical and effective solution for real-world llm safety applications.