Customize Multi-modal RAI Guardrails with Precedent-based predictions
Authors: Cheng-Fu Yang
Thanh Tran
Christos Christodoulopoulos
Weitong Ruan
Rahul Gupta
Kai-Wei Chang
Venue: COLM 2025 (2025)
Keywords: Customizable Guardrail
TL;DR: We propose a precedent-based approach to enhance the adaptability and interpretability of multimodal guardrails, enabling customizable policy enforcement without extensive retraining.

Abstract: A multi-modal guardrail must effectively filter image content based on user-defined policies, identifying material that may be hateful, reinforce harmful stereotypes, contain explicit material, or spread misinformation. Deploying such guardrails in real-world applications, however, poses significant challenges. Users often require varied and highly customizable policies and typically cannot provide abundant examples for each custom policy. Consequently, an ideal guardrail should be scalable to the multiple policies and adaptable to evolving user standards with minimal retraining. Existing fine-tuning methods typically condition predictions on pre-defined policies, restricting their generalizability to new policies or necessitating extensive retraining to adapt. Conversely, training-free methods struggle with limited context lengths, making it difficult to incorporate all the policies comprehensively. To overcome these limitations, we propose to condition model's judgment on "precedents", which are the reasoning processes of prior data points similar to the given input. By leveraging precedents instead of fixed policies, our approach greatly enhances the flexibility and adaptability of the guardrail. In this paper, we introduce a critique-revise mechanism for collecting high-quality precedents and two strategies that utilize precedents for robust prediction. Experimental results demonstrate that our approach outperforms previous methods across both few-shot and full-dataset scenarios and exhibits superior generalization to novel policies.

Published as a conference paper at COLM 2025
Customize Multi-modal RAI Guardrails with Precedent-based predictions
Cheng-Fu Yang", Thanh Tran?, Christos Christodoulopoulos Weitong Ruan", Rahul Gupta², Kai-Wei Chang' Universiit of California, Los Angeles Ammazon AGl
IInformation's Commissioner's Office cfyangecs, ucla.edu
Abstract
A multi-modal guardrail must effectively filter image content based on user-defined policies, identifying material that may be hateful, reinforce harmful stereotypes, contain explicit material, or spread misinformation. Deploying such guardrails in real-world applications, however, poses sig- nificant challenges. Users often require varied and highly customizable policies and typically cannot provide abundant examples for each custom policy. Consequently, an ideal guardrail should scalable to the multiple policies and adaptable to evolving user standards with minimal retraining. Existing fine-tuning methods typically condition predictions on pre-defined policies, restricting their generalizability to new policies or necessitating ex- tensive retraining to adapt. Conversely, training-free methods struggle with limited context lengths, making it difficult to incorporate all the policies comprehensively. To overcome these limitations, we propose to condition model's judgment on "precedents" which are the reasoning processes of prior data points similar to the given input. By leveraging precedents in- stead of fixed policies, our approach greatly enhances the flexibility and adaptability of the guardrail. In this paper, we introduce critique-revise mechanism for collecting high-quality precedents and two strategies that utilize precedents for robust prediction. Experimental results demonstrate that our approach outperforms previous methods across both few-shot and full-dataset scenarios and exhibits superior generalization to novel policies.
Introduction
Responsible AI (RAI) (Amazon, 2024) aims to develop AI systems that are safe, fair, helpful, and interpretable. Guardrails (Gehman et al., 2020; Welbl et al., 2021) are approaches designed to realize this goal, it can help ensure that Al behavior aligns with societal values, ethics, and the specific needs of diverse users. As Al systems increasingly interact with diverse types of media, the need for multi-modal guardrails becomes crucial. This system is required to filter out image contents that may promote hate and violence, reinforce harmful stereotype, contain sexual material, or propagate misleading or fraudulent information (Qu et al., 2024; Crone et al., 2018; Liu et al., 2023; Wu et al., 2020; Helff et al., 2024).
Different AI service/mode providers often require different policies to regulate their AI systems (Dong et al., 2024b): one may prioritize filtering out erotic content, while another may be more concerned with violent material. Furthermore, even within the same cate- gory, definitions of what content should be blocked can vary significantly due to cultural iifeerences-certaii gestures or expressions might be considered offensive in one culture but neutral in another. An effective RAI guardrail system should, therefore, be both cus- tomizable and transparent. Not only should it accommodate user-defined policies, but
Work was done at Amazon. Thee code is available at:https:/ http:///hnbbbmo/eeyyeyy/////////

---

Published as a conference paper at COLM 2025
'Refer the reasoning process of penviar image and dhicdd iiput
The rillowiing contents are insaie Secuall Nustity Made
VLM Guardrail
VLM Guardrail
Unssfe
Traiiing
Pre-dittined Palley
Datatane
Retrieved Precedent
Polay: Public and Personal reelh
?
Vilaiing Polcc?? Yes.
VLM Guardrail
Pattorale:
fellwiing contento are LeTaAd
Retrieval
Smoking i couse various isues,
including respiitury problems, heort diseres, and concer
Novil Podke
Irrelvient contost
(b)
Figure 1: (a) Models fine-tuned on pre-defined policies struggle with novel policies and require extensive retraining adapt. As the number of policy scales, during inference it becomes infeasible to include all the polices and may include irrelevant ones. (b) Our proposed method addresses these limitations by conditioning on structured precedents, which explicitly capture the model's reasoning processes from past decisions. This enables the guardrail to flexibly adapt and accurately handle dynamically evolving and novel policies without extensive retraining.
should also provide clear explanations for each filtering decision, enhancing user trust and accountability.
However, a particular challenge arises from the need for flexibility in incorporating new rules, which often come in the form of few-shot learning tasks (Dong et al., 2024a). When users introduce new, specific filtering requirements, there is typically little data available to define these unique policies. As shown in Fig. 1 (a), prior fine-tuning methods methood (Schramowski et al., 2022; Qu et al., 2023; Helff et al., 2024; Schuhmann, 2023; Praneeth, 2019), become less feasible as they are pre-trained on pre-defined policies and involve substantial com- putational overhead for retraining with each new policy. While in-context learning meth- ods (Dwivedi et al., 2023; Oba al., 2023) offers an alternative, becomes less viable the number of policies grows. Limited context length constrains the ability to include all policies in the prompt, and including examples from irrelevant policies may degrade the guardrail's effectiveness. Consequently, there is need for methods that support scalable, flexible, and efficient customization of RAI guardrails without excessive retraining or labeling demands.
To address the above challenges, we propose to condition model's prediction on precedent database, as shown Fig. 1 (b). precedent is a structured example of model's reasoning process, capturing how it previously predicts whether an image is policy-violating (PV) according to a specified policy and the rationale behind its decision. As model learns to reason with precedents, which capture contextually relevant nuances that static policy definitions may fail to encapsulate, it can naturally evolve and expand to accommodate new rules without extensive retraining.
In this paper, we first introduce a critique-revise mechanism to collect high-quality prece- dents, where the model reviews its own analyses, critiques incorrect predictions, and revises them accordingly. Importantly, this entire process operates without human in the loop. To utilize the precedent database effectively, we introduce two complementary methods that can be applied at training and inference time, respectively. First, for users with access to local model like LLaVA (Liu et al., 2024), we demonstrate that fine-tuning with precedents collected via the critique-revise mechanism yields significant performance gains. Second, at inference time, the database can be leveraged through retrieval-augmented generation (RAG) to retrieve the most relevant precedent. This allows our method to scale efficiently even with large number of diverse policies, and applicable for proprietary models like GPT-4V (Achiam et al., 2023) and Claude (Anthropic, 2023).
We evaluate our method on UnsafeBench (Qu et al., 2024), which encompasses 11 distinct RAI policies. Compared to prior baselines, our approach achieves substantial improve- ments of 6.6% and 6.8% in Fl scores across fhw-shot and full-dataset settings, respectively. Moreover, when isolate each policy from the training set and treat it as novel cate- gory- simu-aiin the model's generalization capability to unseen policies-oee -our method outperforms existing approaches by 16.7% in terms of F1. These results underscore our
-our
2

---

Published as a conference paper at COLM 2025
Training
Precedent
@upervised
VLM Guardrail
Criique
Labal
Policy: Deggll Actialy
Revise
VLM
Image
Guardrail
moaring tussie..
YBs
Label Nos (PV)
Guardrail
Inference
Captlom: persor weading
plowes..
Bartoon
Procedent
shows person exongging into
Examine
VLM Guardrail
Prediction:
Image
YesMo
Constructed Precedent
(a) Precedent Construction
(b) Precedent Utilization
Figure 2: An overview of our framework leveraging Visual-Language Model (VLM) with precedents. (a) During precedent construction, an instance is created through a critique- revise cycle where the VLM generates captions and rationales for images, receives critique, and revises its outputs. This enables our method to collect samples that were initially misclassified. (b) During training, the VLM fine-tuned with the constructed precedents through supervised learning to enhance performance. At inference, our method retrieves the most relevant precedents for each input image, aiding in accurate policy-violation prediction with contextual support.
method's effectiveness in efficiently scaling customizable RAI guardrails across diverse and dynamically evolving policy landscapes.
2 Related Work
Datasets for uate the effectiveness of current models in detecting policy-violating (PV) visual content. For instance, Wu al. (2020) introduced video dataset specifically for violence detection, while Lee et al. (2024) benchmarked text-to-image (T2I) models to assess issues related to gen- der, faimess,, and toxicity. Jha et al. (2024) proposed methods evaluate nationality-basee stereotypes in 12I models, highlighting biases that may arise acToss cultural representations. In the realm of multi-modal large language models (MLLM), Liu et al. (2023) examined the resilience of these models against malicious attacks. Meanwhile, Open (OpenAT, content no longer publicly available) established 11 categories for categorizing unsafe images, providing a foundational taxonomy for PV content detection. Building on this taxonomy, UnsafeBench (Qu et al., 2024) gathered an extensive dataset, which includes both real and synthetic images, thereby offering comprehensive resource for evaluating model performance across diverse RAI policies.
Visual Content. Several efforts have been made to eval-
Current Multi-modal RAI Guardrails. To filter out content for given RAI policy, the most common approach is to fine-tune a classifier on the collected PV data. For example, NSFW Detector (Schuhmann, 2023) is trained et 2021) to detect explicit content, while NudeNet (Praneeth, 2019) leverages the YOLO backbone (Redmon, 2016) to build a nudity detection model. Q16 (Schramowski et al., 2022) is a binary CLIP (Radford et al., 2021) classifier that predicts the morality of the given image. Similarly, MultiHeaded (Qu al., 2023) applies linear probing to fine-tune separate classification heads for five different RAI policies. While these methods demonstrate effectiveness in detecting specific types of PV content, they are typically limited to fixed policies and require extensive labeled data to build. While in-context learning has not been extensively explored for multi-modal guardrails, it has proven effective in reducing biases in language model outputs (Dwivedi et al., 2023; Oba et al., 2023). However, as the number of RAI policies increases, studies (Shi et al., 2023) have found that including irrelevant context in prompts can significantly degrade model performance.
subset of LAION-400M (Schuhmann
3 Method
For the sake of clarity, we first define the task's 's setting and notations. Specifically, the users can customize their RAI guardrail with P different policies, with each policy p contains N-shot data points. Each data point includes an image x and a binary label 1, indicating whether the image is policy-violatng (PV) not. In Sec. 3.1, first introduce the concept
3

---

Published as a conference paper at COLM 2025
User:
You were tasked to analyze the image based its content. However, you failed to produce the correct analysis. This was your previous caption: [Caption]]
Policy: illggs Activty
Definition: Drug
and other illggl activiiee
User: Critique the provided caption. Specifically,
Viclating the Pollcy? Yos. [
Caption: y
Did the cuption overlook any significant elenents or objerts in the iraae that might be relevant [pal
The image features person's hand holding a pill bottle, with the word "Drugs" written on it.
Image I
The gils scattered around the bottle, emphasizing theme drugs.
Did the caption misinterpret any sonething
Rationale: 7
objects, mistaking else?
The imgg contoins pil bottle with the word 'Orugs' wniten on and the persan's hand pointing down towards the pils. This sugpests connection
which illegol octivity
Assistant:
[Critique Response]
Precedent pd (.
Based the critique, revise caption to better reflect the content of the inage.
Assistant:
[Revised Caption]
(a)
(b)
Figure 3: (a) An illustration of a precedent. A precedent composed of a policy and the corresponding definition, an evaluation of whether the given image violates the policy, caption of the image, and a rationale explaining the decision. (b) Prompt template used to guide the model in analyzing, critiquing, and revising image captions based on the given policies.
of precedent and how to utilize the limited user inputs to collect high-quality precedents. Then, in Sec. 3.2, we demonstrate how the constructed precedent database can enhance model's performance. During training, fine-tuning with the collected precedents enhances the model's 's robustness against novel instances, while at inference time, retrieval-augmenntr generation (RAG) allows the model to reference the most relevant precedents for its decision- making. We illustrate our framework in Fig. 2.
3.1 Constructing the Precedent Database
Definition of Precedents. We show an example of precedent in Fig. 3a. Given an image and the associated policy p, we construct the precedent by first generating the caption of the image y. Based on the image and caption, we ask the model to predict the PV label I and rationale r behind the decision. If the predicted label is consistent with the actual label I, the resulting precedent is added our database as tuple pd (x, (x,y, L,,, p). Empirically, we find that having the model generate rationale based on its own predicted label, rather than the given ground-truth, significantly reduces rationale hallucination, as the rationale naturally aligns with the model's 's internal reasoning rather than being artificially justified after the fact.
Improving precedents with the critique-revise mechanism. Inconsistencies still occur between predicted and ground-truth labels, given that our model not extensively fine- tuned with the specific input images and policies. Additionally, the few-shot nature of the task makes extensive fine-tuning infeasible. In our preliminary analysis we discovered that the inconsistent cases often stem from captions that are missing important details in the image. To address this issue, we employ a critique-revise mechanism where the model reviews its own generated captions, checking whether missed any important nuances or misinterpret any objects. Then, based on the critique, produce revised caption Vrev, as shown in Fig. 3b. Áfter obtaining the revised caption, we cast the model to make predictions and rationales again based on the improved caption. If the new prediction is correct, we add this refined precedent pd = (x, yrevr Trevy p) to the database, enhancing the model's contextual understanding for similar testing cases. Notably, this process operates without additional human annotations, making an efficient and scalable method to generate useful precedents for challenging instances that model fail to classify initially.
4

---

Published as a conference paper at COLM 2025
3.2 Utilizing the Precedent Database
After constructing the precedent database, we propose two strategies to utilize it. First, can be employed as an additional resource during training, where fine-tuning with the collected precedents enhances the model's robustness against challenging cases. Second, at inference time, the database can be leveraged through retrieval-augmented generation (RAG), enabling the model to reference the most relevant precedents when making decisions.
Reflective Fine-tuning. Prior works have demonstrated that training with samples that gen- erated through reflective prompting could substantially improve model's robustness (Dou et al., 2024; Bai et al., 2022), and is particularly effective when these samples originate from the same model used for training. Similarly, the revised caption generated by our critique-revise mechanism can be viewed as the outcome of model's ss reflection. Leveraging this, we fine-tune our model with a supervised objective: given an image x, the model is tasked to predict the corresponding caption Vrev and its PV label I, as shown in Fig. 2 (b). Our experiments indicate that supervision from the revised caption yrev is crucial, as provides the model with richer contextual information, helping it capture subtle details that improve its accuracy on the novel cases.
Retrieving the most relevant precedent. Due to the limited context length of language models, it is impractical to include all precedents as in-context examples in the prompt. In addition, studies have shown that including irrelevant examples can degrade model performance (Shi et al., 2023). In response, we could employ retrieval model to select the most relevant precedent from the database. Given a test image, the retrieval process can be performed based on the similarity between either the image embeddings or the textual embeddings of the captions. For image retrieval, we use the visual encoder from CLIP with 11T for text retrieval, we use the Contriever (Izacard et al., 2021) model, extracting sentence embeddings via mean pooling. Empirically, our results indicate that using image similarity as the retrieval metric yields the best overall performance.
Inference. Given a test image x, we first generate its caption, 9. We then incorporate the rationale and policy of the retrieved precedent into the prompt, asking the model to determine whether the image is policy-violating (PV) or not. The detailed prompt template is provided in Appendix A.
Experiments
4.1 Dataset and Evaluation Protocols
Datasets. To demonstrate the flexibility of our method, we evaluate on UnsafeBench (Qu et al., 2024), which contains both real and synthetic images across 11 policies: Hate, Ha- rasmmet,, Violence, Self-Harm, Sexual, Shocking, Illegal Activity, Deception, Political, Public and Personal Health, and Spam. We provide the detailed definition of each policy in Appendix A.3. The dataset contains 8100/2000 images for the training/ validation split, respectively. Each category contains at least 640 training images, with each image annotated with binary label indicating whether it is unsafe or not.
Evaluation Protocols and Metrics. To evaluate model's adaptability under the low-resource setting, our main experiment was conducted using only 2.5% of the available training data, equivalent to 16 images per policy. Notably, to reflect more realistic scenario, we adjust the evaluation protocol in the original UnsafeBench paper. We do not provide the model with the associated policy for each image. Instead, model would need to find out which policy the image might violate. For metrics, we adopt the standard classification metrics: precision, recall, Fl score, and accuracy. In our task, we define policy-violating (PV) image as positive, with non-violating images as negative.
4.2 Baseline Methods
For baseline methods, we consider three different type of models based on their access to additional training resource and fine-tuning availability.
Pre-trained Classifiers. The pre-trained classifiers used in our experiments are trained on ad- ditional datasets specifically curated for certain categories. In particular, we consider NSFW
5

---

Published as a conference paper at COLM 2025
Method
Hate
Haras- Viole-
Self-
Sexual Shock- Illegal Decep-Polit Polit- Health Spam
Overall
sment nce
Harm
ing
tion
ical
Pre-trained Classifiers'
SD Filter
0.785
NSFW
NSte_Detector Detector
0.738
NudeNet
0.624
fine-tune access
GPT-4o
w/ ICL
0.475
0.667
0.576
0.465
0.787
0.606
0.459
0.526
0.523
0.273
0.290
0.584
w/ Precedent (RAG)
0,510
0.686
0.671
0.490
0.768
0.778
0.832
0.778
0.775
0.743
0.659
0.726
Claude-3 Sonnet
w/ ICL
0.261
0.427
0.613
0.492
0.733
0.671
0.515
0.295
0.296
0.406
0.303
0.569
w/ Precedent (RAG)
0.386
0.406
0.697
0.576
0.861
0.713
0.795
0.651
0.683
0.736
0.546
0.691
LLaVA
w/ ICI.
0.383
0.125
0.604
0.491
0.775
0.697
0.354
0.241
0.182
0.406
0.246
0.552
w/ Precedent (RAG)
0.468
0.255
0.737
0.500
0.845
0.733
0.648
0.717
0.351
0.487
0.447
0.613
Fine-tuned Methods
MultiHended w/ SFT
0.059
0.496
0.545
0.269
0.759
0.570
0.279
0.407
0.610
0.443
0.543
0.507
Q16 SFF SFT
0.568
0.450
0.639
0.435
0.332
0.768
0.575
0.661
0.438
0.439
0.229
0,533
LLaVA
LlavaGuard (w/ SFT)
0.289
0.379
0.844
0.330
0.590
0.667
0.400
0.622
w/ Precedent (Re-FT)
0.604
0.500
0.642
0.521
0.835
0.723
0.652
0.638
0.599
0.667
0.427
0.653
w/ Precedent (Re-FT RAG)
0.610
0.500
0.667
0.526
0.873
0.735
0.690
0.720
0.579
0.733
0.427
0.688
Table 1: Fl scores of various classifiers across different RAI policies. All models, except the pre-trained classifiers, are trained/ /proep/dd with only 16 images per policy. Pre-trained classifiers specifically designed for detecting sexual content and are not applicable to other categories.
Detector (Schuhmann, 2023), pre-trained the subset of LAION-400M. NudeNet (Praneeth, 2019) is trained on internet images and SD Filter (Rando et al., 2022) is pre-trained CLIP.
Proprietary Models. To demonstrate that our method can work with models without fine-tuning access, we select proprietary models such as GPT-4o (Achiam et al., 2023) and Claude-3 Sonnet (Anthropic, 2023). We also report results on LLa (v1.5) (Liu al., 2024) without fine-tune the model for comparison. For these models, we construct two variants for comparison. The first variant uses in-context learning (ICL), where all policies and their definitions are included in the prompt. The second variant, Precedent (RAG), retrieves the most relevant precedents and incorporates them into the prompt to guide the model's response. Since different prompt templates result can yield varying results across models, we report the best performance achieved for each model using different templates. Note that Chain-of-thor of-thougttt (CoT) reasoning has been applied to all methods. The detailed prompt templates can be found in Appendix A.
Local Models. We compare approach against two main categories of baselines: CLIP- based methods and LLaVA- methods. All baseline methods are fine-tuned using the same data for fair comparison. For CLiP-based baselines, we select MultiHeaded (Qu et al., 2023) and Q16 (Schramowski et al., 2022), which employ linear probing and prompt tuning, respectively, on top of pre-trained CLIP visual encoder. For VL-bseed baselines, we focus examining the difference between conditioning on fixed policy and conditioning on precedents. Specifically, we consider LlavaGuard (Helff al., 2024), which performs supervised LoRA fine-tuning (Hu et al., 2021) (w/ SFT) conditioned on fixed policies. To maintain fairness, we fine-tune using the same model size (LLaVA- v1.5). Additionally, we introduce two variants conditioned on precedents to demonstrate the effectiveness of our method: the first variant involves fine-tuning with precedents generated through our critique-revise mechanism described in Sec.3.2, denoted as w/ Precedent (Re-FT); the second variant further enhances inference using aetrieeal-augmented generation (RAG) to dynamically select the most relevant precedents, denoted as w/ Precedent (Re-FT + RAG).
4.3 Quantitative Analysis
We present the performance of different baseline models and our methods in Table 1. For reference, we list the performance of pre-trained classifiers which require additional training

---

Published as a conference paper at COLM 2025
Method
Acc
Precision
Recall
F1
MultiHeaded
0.550
0.602
0.435
0.505
Q16
0.635
0.608
0.429
0.503
LlavaGuard
0.718
0.784
0.359
0.492
Ours
0.745
0.672
0.647
0.659
Table 2: Performance comparison under the leave-one-out (LOO) setting. In this experiment, models are trained on 10 categories and then adapted to a novel 11th category using few- shot data, simulating real-world scenarios where models must quickly adapt to new or evolving policies. The results represent the average performance, as each category is treated as the novel category once, and the process is repeated 11 times. Note that we use LLaVA- Precedent (Re-FT+RAG) as our method.
data. However, they are specialized for specific tasks and do not generalize across different policies.
Fine-tuning free methods. For models that are not available to be fine-tuned, find that adding precedents via RAG consistently boosts performance across categories, particularly for those that do not fit the conventional definition of "unsafe" content. For instance, LLaVA improves by 8% on the Public and Personal Health category, and 20% on Spam. Similarly, GPT- improves by 47% and 37% on these two categories, respectively. Overall, the inclusion of precedent, compared to models using ICL, results in improvements of 6%, 14% and 12% for LLaVA, GPT-4, and Claude, respectively, using only 16 images per policy, and, without additional fine-tuning, This suggests that, compared to conventional ICL methods, leveraging precedents with RAG enables models to generalize more effectively across diverse RAI policies.
Fine-tuned Methods. For local models that are able to be fine-tuned, we find that vision and language model (VLM) like LLaVA is more advantageous than the CLiP-based model, with standard fine-tuning (LlavaGuard (Helff et al., 2024)) surpass MultiHeaded (Qu et al., 2023) (CLIP with linear probing) and Q16 (Schramowski et al., 2022) (CLIP with prompt tuning) by around 10%. One likely reason for VLM's superior performance is its ability to incorporate the descriptions of each RAT category, allowing it to leverage specific policy information to make more accurate predictions.
Within the LLa -assed variants, our proposed method (Re-FT)-which conditions predic- clear advantages over LlavaGuard-whiee is fine-tuned on predefined policies. Specifically, precedents substantially enhance the model's generaliza- tion capability in challenging scenarios, as indicated by notable improvements in categories that previously exhibited low performance, such as Hate (from 0.289 to 0.604) and Deception (from 0.330 to 0.658). These findings underscore that conditioning predictions on precedents significantly boosts the model's robustness across diverse and complex RAT categories.
tions on precedents-ddmmonstre
Lastly, we find that the best performance is achieved when Re-FT is combined with Retrieval- Augmented Generation (RAG) during inference, resulting in an overall Fl score 0.688. The combination of these two methods demonstrates the complementary strengths of each approach: Re-FT enhances the model's ability to generalize by focusing on difficult cases during training, while RAG improves inference by retrieving contextually relevant examples to aid decision allows LL to achieve robust performance across diverse RAI categories, even with limited data (only 16 images per policy), highlighting the effectiveness of our proposed methods.
This synergy between training and inference strategies
4.4 Adaptability and Robustness Analysis
Adaptability to Novel Policies. Table 2 presents the performance comparison of different methods under the leave-one-out (LOO) setting, we use the same 11 categories as the main experiment. In this setting, model is trained on 10 categories with abundant data (160 images per policy) and then adapted to a novel 11th category using few-shot data (16 images). We repeat this process 11 times and report the average performance the novel category. This
7

---

Published as a conference paper at COLM 2025
# of Policies
LLaVA- ICL
LLaVA- Pre.
GPT4- ICL
GPT4-
Multi- Headed
Q16
Llava- Guard
LLaVA-
Pre.
Pre.
(RAG)
(RAG)
(Re-FT
RAAG
1
0.639
0.654
0.712
0.771
0.585
0.634
0.708
0.733
11
0.552
0.613
0.584
0.726
0,485
0.533
0.622
0.688
A(L)
-0.087
-0.041
-0.128
-0.045
-0.100
-0.101
-0.086
-0.045
Table 3: F1 scores when models are evaluated with single policy vs. multiple policies (1 vs. 11). Lower discrepancy (A) values indicate higher robustness against increasing number of policies. Note that "Pre." " indicates "Precedent".
Fieett Model Peffrmance Sciling whth
sueed Performane Scaling with Training
300
60
2.0
G7T4 Pr. FAG
number Labeled
oara Poins
Performance scaling of fine-tuning- models (left) and fine-tuned models (right) as the number of labeled data points increases. We test models with different scales: 16, 160, and 640 images per policy. Note that "Pre." indicates "Precedent".
Figure 4:
scenario simulates real-world conditions where models need to adapt quickly to new or evolving policies.
From the table, we observe that prior fine-tuning methods, such as LlavaGuard, struggle to adapt effectively to novel categories in the LOO setting, yielding an overall F1 score of 0.492. Because these models rely on pre-defined policies the training data, they may fail to recognize images from new policies as policy-violating (PV), leading to high precision but low recall. In contrast, our method leverages retrieved precedents that incorporate contextual information such as relevant policies and rationales from similar cases to make more informed decisions, resulting in a 16.7% improvement in F1. This adaptability is crucial in real-world scenarios, where policies frequently evolve and models must quickly and reliably adjust to new or modified PV conditions.
Scaling the Number of RAI Policies. Here, analyze how the model's performance varies as we introduce more RAI policies, testing the robustness of our method in comparison with prior SFT and ICI models. Specifically, we first evaluate the model with single policy at a time, where the model only needs to determine an image violates that specific policy. In the second setting, which is our default, the model evaluated with all 11 RAI policies simultaneously. Here, the model must first identify the relevant policy associated with the image, and then assess whether the image violates that policy. From the results in Table 3, we observe that the number of policies increases, all models experience decline in performance. However, prior ICL and SFT approaches, such as LLaVA-ICL and GPT4-ICI, show larger discrepancies (A -0.087 and -0.128, respectively). In contrast, our methods consistently exhibit low discrepancy values (A -0.041, -0.045, -0.045), suggesting that the inclusion of precedents and retrieval mechanisms helps maintain performance when handling multiple policies. The findings validate that our proposed method provides robust and scalable solution for handling diverse RAI policies.
Scaling Up Training Data. Finally, we evaluate the impact of scaling up the training data model performance, demonstrating the scalability and robustness of our method the number of available labeled data increases. We conduct experiments with different data scales: 16, 160, and 640 labeled images per policy, corresponding to 2.5%, 25%, and 100% of
8

---

Published as a conference paper at COLM 2025
Config ID
Subject
Th
Policy
Rationale
F1
(A)
1
0.552
2
text
closest
+0.112
3
text
0.7
x
+0.136
4
text
0.8
X
V
+0.150
5
text
0.8
+0.170
text
0.8
X
+0.154
7
img
0.7
+0.190
8
img
0.8
+0.214
Table 4: Ablation analysis of how to utilize precedent with appropriate RAG configurations. The table explores variations in retrieval subject (text vs. image), thresholding (whether minimum similarity threshold applied or the most similar precedent is retrieved), and the inclusion of policy definition and rationale of the precedent in the model.
the training dataset, respectively. We present the results in Fig. 4. The plot on the left shows the performance of fine-tuning-free that the increased data allows us to build more comprehensive precedent database, which enhances the model's ability to retrieve relevant information and consistently boosts the performance even without fine-tuning, In the right plot, we observe the results for fine-tuned methods. Here, our proposed method (LLa VA w/ Precedent (Re-FT +RAG)) consistently outperforms other models data scales up. The combined strengths of reflective fine- tuning and retrieval-augmenter generation enable model to leverage the additional data effectively. Notably, as shown on the right hand side of Fig. with 640 data points per policy, our method achieves results that surpass those of large proprietary models like GPT-4 and Claude-3. This suggests that our method can achieve competitive or superior results without relying on proprietary resources, making it a scalable and efficient for real-world applications with diverse policies.
methods. As the amount of data grows, we observe
Ablation Analysis. From Table 4 we observe that even the least effective RAG configuration (model ID 2) improves the baseline model by 11%. We also find that setting a higher simi- larity threshold helps the model include only relevant precedents, as demonstrated by the performance improvement when comparing model ID and 4. Our empirical results demon- strate that using images the retrieval subject (e.g., comparing ID 5 and 8) consistently yields better results. This improvement can be attributed to images capturing more detailed information than captions. Next study how each elements in the constructed precedents contribute to the overall performance. From Table we observe that removing either the policy definition (model İD 4) the rationale for the precedents (model ID 6) results in 2% decrease in overall performance. This suggests that both components play a significant role in enhancing the model's capabilities by providing essential contextual information. Due to page limit, ablation analysis of other elements can be found in Appendix B.
4.5 Qualitative Analysis
The examples in Fig. 5 (a) illustrate how the introduction of precedents enhance model predictions by providing contextually relevant rationale and related RAI policies. Take the image from the third row as example, the image is initially classified "Safe". However, when the retrieved precedent indicates that such actions could be perceived as disruptive, unwanted, or indicative of harassment, the model revises its prediction to "Unsafe". Simi- larly, the rationale emphasizing the presence of marijuana imagery or actions that could be perceived as an illegal activity demonstrates how contextual reasoning can improve model's capability. Figure 5 (b) further shows how the critique-revise mechanism refines the model's outputs by addressing potential failure cases. For instance, by identifying overlooked elements, such as the prohibitory sign in the image in the second row, the model revises the caption to incorporate the symbol's 's implications, suggesting potential discrimination against religious beliefs. Likewise, after undergoing the self-critique process, the model recognizes the man's hand on the woman's legs, and consequently updates the caption to include this element to reflect the proactive content in the image. The above examples

---

Published as a conference paper at COLM 2025
Image
Retrieved Precedent
Rationale Of Precedent
Prediction (w/o w/ precedent.)
Image
Original Caption
Caption Critique
Revised Caption
FHIMERT EAIAEE
The imae ortare poalars cou be rslead policcal may depir mikoy lik which somo conters, Theerios the may uniar contant widked lhk.
penun in
fuet nut the inage postert which
tmager collage ol yofiid posanss eair ding in tiee dila
Safe- Unsafo
buildng
cheact or amert, triiing the laing.....
relover tos political
alouic rivee
image fertames dislect one rep sereenning cooss and represer sie or Elaved criesor the emage aht thu sitlk Sotn rembod ane the cnes son puingga andine ot (oved oppeir to
The image realingg heo
conle and itte bowis
The inage contann par rinng at purtannnplnnt bwilecggoent wiich lleggt sctivty mory pluces.
Safe- 1 Unssfe
The caplicy one roodke
right saver thwiinags
ziso coesane "ho
to these syedooa,, thene
a "no sulghent lombel
in deddnnicaa dennert
cantar imsipe..
imagee
rano This prosance
serrau eageaale
rymtos wigght bupgard
rblruttin
The image showe sumnn ancohenga soir bi dtraphenn or anwadded herssament
Safe- I Unsofe
The image
promoing nntoleranss
or diicrn mateer apaimal
repreeraater miliiiuus
aga not goout
image bu
indizziing a companten
posailly
tade or dicciiitaate
The cusiken olerner
dpeets Smoking krown to ca shir potkine, hourt obeanes, The mngg snounggun yricara wichh can regalwey inpactpabilc (a)
Sale-
of Aiann and Aoran
sttng ut lacil
undtaa yoman
and vomun othee They am both
sooma lig.
Unsafe
aeauaiity
contert wikked to
(b)
Figure 5: Qualitative examples of (a) how precedents improves model predictions by providing contextual rationale (marked in italic) and relevant policy (marked in bold), and (b) how the critique-revise mechanism helps to overcome the failure cases that stemming from the incorrect descriptions.
showcase the value of leveraging precedents and reflective critique for more accurate and context-aware content moderation. More qualitative examples can be found in Appendix C,
5 Conclusion
In this paper, we introduce the concept of precedent to address the challenges when we deploy a multi-modal Responsible AI (RAI) guardrail in real-world applications, where the model needs to deal with evolving policies and the scarcity of labeled data. At the core of our approach is the precedent database, enhanced by critique-revise mechanism. It enables models to adapt and refine their predictions effectively while building a comprehensive set of precedents. To further leverage the precedent database, we introduced two complemen- tary methods: reflective fine-tuning with precedents to bolster model robustness against novel instances and retrieval-augmeente generation during inference to enable reasoning with the most relevant precedents. Together, these methods offer a scalable, flexible solution for customizing RAI guardrails without the need for extensive retraining large labeled datasets. Our results demonstrate that this approach not only generalizes well with lim- ited labeled cata, but also provides flexibility to accommodate user-specific and culturally nuanced policies. However, we acknowledge that the performance of our framework is bounded by the capabilities of current vision-language models (VLMs). If the model fails to associate presented objects with relevant RAI concepts, particularly for novel or abstract cases, it may not reliably identify unsafe content. Future work may involve exploring more efficient retrieval strategies, expanding the precedent database, and enhancing integration with real-time policy to further refine the system. Additionally, combining our precedent- based framework with symbolic logic or rule-based engines represents a promising avenue. For example, precedents might be encoded as grounded programs or logical formulas and then fused with probabilistic programming or program synthesis techniques. Finally, extending this methodology to other modalities such as video or audio-offer another compelling direction for exploration.
10

---

Published as a conference paper at COLM 2025
Ethics Statement
We acknowledge the ethical considerations inherent in developing multimodal guardrails aimed at filtering image content based on customizable user-defined policies. First, the definition of harmful or inappropriate content can vary widely across cultural and individ- ual contexts. By leveraging user-provided precedents, our approach aims respect and accommodate these diverse perspectives rather than imposing universal standard. Second, our experiment use publicly available datasets UnsafeBench licensed for academic research only, which contains disturbing and unsafe images. The dataset has undergone ethical review by an ethical review board (ERB) from the original paper. The ERB has approved this study, confirming there are ethical concerns provided annotators are not exposed to illegal content such as child sexual abuse materials, which are explicitly absent from the dataset. To minimize potential harm from exposure to harmful content, annotations were conducted exclusively by the authors themselves. To account for both ethical concerns and reproducibility, the dataset only available upon request and for research purposes.
We acknowledge potential biases inherent in our datasets and models. However, significant advantage of our proposed model is transparency in its reasoning process. The model explicitly indicates which policy a given input may violate and provides detailed reasoning behind this assessment. Such transparency essential for enabling responsible use and facilitating auditing and oversight. Finally, we intend our model to be deployed responsibly in content moderation applications, specifically tailored align with users' customizable safety policies. We strongly recommend deploying our method with clear usage guidelines, continuous oversight, and ethical auditing to maximize its positive impact and mitigate potential risks.
Acknowledgements
We thank the anonymous reviewers and members of the UCLA-NLP+ group for their valuable feedback. The views and conclusions expressed in this work are those of the authors and do not necessarily reflect the official policies or positions of Amazon.
11

---

Published as a conference paper at COLM 2025
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. ar preprint ar arXiv:2303.08774, 2023.
Amazon. Amazon Nova and our commitment to responsible responibllto AI, 2024. URL https://www. mrazonnscience//ii
Anthropic. The Claude Model Family: Opus, Sonnet, Haiku, 2023.
URL
https: / /wwww//wwn/nnn//// wwt.haapc.hooccoo////oo http://cww..cceeeoooooaa//aaaaaa Model Card Claude pdd..
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitu- tional ai: Harmlessness from ai feedback. arXiv preprint ar arXit:2212.08073, 2022.
Damien L Crone, Stefan Bode, Carsten Murawski, and Simon M Laham. The socio-moral image database (smid): A novel stimulus set for the study of social, moral and affective processes. PloS one, 13(1):e0190954, 2018.
Yi Dong, Ronghui Mu, Gaojie Jin, Yi Qi, Jinwei Hu, Xingyu Zhao, Jie Meng, Wenjie Ruan, and Xiaowei Huang. Building guardrails for large language models. arXiv preprint ar arXi::242200182, 2024a.
Yi Dong, Ronghui Mu, Yanghao Zhang, Siqi Sun, Tianle Zhang, Changshun Wu, Gaojie Jin, Yi Qi, Jinwei Hu, lie Meng, et al. Safeguarding large language models: A survey, arxiv preprint ar axXio:240.00262,, 2024b.
Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu, Kai-Wei Chang, and Nanyun Peng. Reflection- reinforced self-training for language agents. arviv preprint ar Xix::2066.01499 2024.
Satyam Dwivedi, Sanjukta Ghosh, and Shivam Dwivedi. Breaking the bias: Gender fair- ness in llms using prompt engineering and in-context learning. Rupkatha Journal on Interdisciplinary Studies in Humanities, 15(4), 2023.
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Realtox- icityprompts: Evaluating neural toxic degeneration in language models. arviv preprint arXie:2009.1 11462, 2020.
Lukas Helff, Felix Friedrich, Manuel Brack, Patrick Schramowski, and Kristian Kersting, Llavaguard: Vlm-based safeguard for vision dataset curation and safety assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
8322-8322 8326, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arxiv preprint ar Wiw::2106.099990 09685, 2021.
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Ar- mand Joulin, and Edouard Grave. Unsupervised dense information retrieval with con- trastive learning. ar Xiu preprint ar xxXi:22112001118, 2021.
Akshita Jha, Vinodkumar Prabhakaran, Remi Denton, Sarah Laszlo, Shachi Dave, Rida Qadri, Chandan K Reddy, and Sunipa Dev. Beyond the surface: global-scale analysis of visual stereotypes in text-to-image generation. arXiv preprint ar Ktt::4401003111 2024.
Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, et al. Holistic evaluation of text-to-image models. Advances in Neural Information Processing Systems, 36, 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
12

---

Published as a conference paper at COLM 2025
X Liu, Y Zhu, J Gu, Y Lan, C Yang, and Y Qiao. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models. ar ar preprint arXiv:2311.17600, 2023.
Daisuke Oba, Masahiro Kaneko, and Danushka Bollegala. In-contextual bias suppression for large language models. arviv preprint arXtv=2309.07251, 2023.
OpenAI. Openai content policy.
http://www..rreeeeeeeee///////// //labs. openai cwm/ http:pccesccooteettolllgg
Bedapudi Praneeth. Nudenet. httos://pppi.orr//mggjeet///at/ 2019.
Yiting Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafe diffusion: On the generation of unsafe images and hateful memes from text- to-image models. In Proceedings of the 2023 ACM SIGSAC Conference Computer and Commurications Security, pp. 3403-3417, 2023.
Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, and Yang Zhang. Unsafebench: Benchmarking image safety classifiers on real-world and ai-generated images. ar Xiv preprint ar arXi::200.00440 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 1748-8763. PMLR, 2021.
Javier Rando, Daniel Paleka, David Lindner, Lennart Heim, and Florian Tramèr. Red- teaming the stable diffusion safety filter. ar preprint ar XrXiv:2210.04610, 2022.
J Redmon. You only look once:
Unified, real-time object detection. Proceedings of the IEEE conference computer vision and pattern recognition, 2016.
Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and turn reflecting on inappropriate content? In Proceedings of the 2022 ACM Conference on Fairness, Accountabillity and Transparency, Pp.
1350-1361, 2022.
Christoph Schuhmann. LLLPbased-NSF--Deeeeteee 2023.
Nsfw
detector.
Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arviv preprint arXiv:2111.02114, 2021.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In International Conference Machine Learning, pp. 31210-31227. PMLR, 2023.
Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxifying language models. arviv preprint Xkio:2109.074455 2021.
Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Computer Vision-FCCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXX 16, pp. 322-339. Springer, 2020.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs. arXiv preprint arXin:2312.07104 2023.
13


Reviews:
[{"review_id": "qdw0xi0Agt", "reviewer": "Reviewer_oTxU", "summary": "This paper considers the guardrail in the context of visual language models, where the goal is to filter out image content based on user-defined policies. This helps avoid hateful, harmful, and incorrect information. To this end, this work focuses on the scalability of multi-modal guardrail, aiming to extend to novel user standards with minimal training. This work proposes to condition the model’s judgment on “precedents”, which are the reasoning processes of prior data points similar to the given input. Such a manner is shown to be effective in difference scenarios.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The problem of the scalability of multi-modal guardrails is reasonable and useful. This work discusses the limitations of existing fine-tuning methods and training-free methods. The former is hard to generalize to novel standards, while the latter struggles with limited context lengths. To address both issues, this work proposes to retrieve similar data points, which is sound to me.\n- The framework is clear and easy to follow. The precedent construction and utilization are clearly illustrated. Both steps are reasonable. \n- The experimental results show the proposed method can bring improvements over several baselines, such as LLaVA. Table 2 also shows the adaptability of the proposed method to novel standards.", "reasons_to_reject": "This submission is overall good. However, it would be better to include a paragraph discussing potential limitations and future directions.", "questions_to_authors": "One important question: Will the Precedent Database be made publicly available?", "extra_scores": {}}, {"review_id": "hrPQpSbR5f", "reviewer": "Reviewer_Hnir", "summary": "This paper proposes to customize a guardrail system by providing \"precedents\", or few-shot examples of a particular type of outputs to be filtered. The paper is well organized and clearly written. The problem is somewhat motivated, and the experimental results are encouraging.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. As the content created by a genAI system becomes more and more complex and diverse, we might need such a few-shot example based guardrail system.\n2. The method is reasonably and carefully designed. \n3. The experiments are comprehensive and the empirical results are encouraging.", "reasons_to_reject": "1. At the current stage, a keyword based filtering customization system seems sufficient. However, I cannot rule out the usefulness of such a method in the future. Therefore, I describe it as \"somewhat motivated\".\n2. The word \"user\" is a bit confusing. It seems to refer to the service provider that outsources the filter system, instead of end users.", "questions_to_authors": "N/A", "extra_scores": {}}, {"review_id": "1UhJtyYzBh", "reviewer": "Reviewer_zikp", "summary": "This paper tackles the issue of customizing user-defined RAI (Responsible AI) policies by introducing a method, which involves storing preceding examples of each independent policy and then retrieving them during inference . This method provides models with task-related examples to improve performance. And the extensive experiments have proved its effectiveness of identifying policy-violating examples in various situations.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Evaluation shows the effectiveness of the proposed method for improving the performance of identifying policy-violating examples.\n- This method can be integrated with different models in various situations.\n- it reduces the affection of noise from input distribution.", "reasons_to_reject": "- Retrieving relevant precedents is the key part while doing inference, but in this paper it does not provide more information about it.\n- Retrieval methods might affect the result of the model performance, so it is recommended that some more experiments should be conducted to prove if the retrieval methods will affect the result.\n- It was to improve the flexibility of customizing user-defined RAI policies, but it has only conducted experiments on two scenarios of one-policy and eleven-policies cases. That might weaken the confidence level of the flexibility of the method.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LC0xk3oSzt", "reviewer": "Reviewer_3C7s", "summary": "This paper proposes a new framework for customizing multi-modal Responsible AI (RAI) guardrails via precedent-based reasoning. Instead of relying on static policy definitions or extensive fine-tuning, the authors introduce a precedent database—a repository of reasoning examples that the model can condition on. These precedents are generated via a critique-revise mechanism, wherein the model self-analyzes and revises its predictions without human intervention. Experiments on the UnsafeBench dataset demonstrate superior performance in both few-shot and full-dataset scenarios, with significant improvements over baselines, especially in the generalization to novel policies.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The use of precedents to drive policy-violation detection is novel and inspired, borrowing legal metaphors to increase interpretability and flexibility.\n\n2. This framework avoids the need for exhaustive re-training and supports adaptation to user-specific and evolving RAI policies, a highly practical and underexplored problem.\n\n3. Self-supervised precedent construction via critique-revise is effective and human-free, which makes the approach cost-efficient and easy to scale.\n\n4. The authors explore robustness across various configurations, including number of policies, scale of data, and retrieval strategies (Table 4), providing a holistic view of system behavior.", "reasons_to_reject": "1. While the critique-revise loop is effective, comparisons to human-annotated precedents or other automated methods for precedent construction would be valuable.\n\n2. Though the authors mention potential extensions to video/audio, all experiments focus solely on images. A concrete demonstration on at least a second modality would strengthen the generality claim.\n\n3. The authors mention multiple prompt templates were tried but only “best” ones reported. Some transparency around variance across prompts would help understand reliability.\n\n4. The pipeline is relatively complex (critique, revise, retrieve, fine-tune) and while effective, it would be useful to include computational overhead analysis or training cost comparison.", "questions_to_authors": "1. Could the critique-revise mechanism be extended to text-only RAI tasks (e.g., hate speech detection)? Would it work without images?\n\n2. Have you evaluated how often the retrieved precedent actually comes from the same policy as the test image? Are there errors due to cross-policy retrieval?\n\n3. How sensitive is the system to errors in the precedent rationale? Could the model be misled by faulty past reasoning?\n\n4. Could this precedent approach be combined with existing symbolic logic or rule-based systems for hybrid moderation?\n\n5. What are the limits of this method in open-world settings where even precedents may fail to capture unseen or abstract concepts?", "extra_scores": {}}]

Author rebuttal:

Meta-review: Applying guardrails for content moderation in real world deployments of generative AI often requires adapting guardrails to changing policies that define acceptable vs unacceptable content. The authors, recognizing this challenge, develop an approach to condition the judgements of guardrail models on precedents generated from training data of the guardrail model, augmented with a rationale that explains the judgement. By doing so, the authors are able to ground the judgements better in rationales, thereby allowing adaptation to changing moderation policies. The authors call attention to an important problem and provide an interesting and practical approach to solve it. While the reviewers bring up valid concerns on the comprehensiveness of evaluation, I believe the COLM audience would benefit from the presentation of this paper, that brings attention to an important and understudied problem, while also proposing a promising solution that can spark further work.

[Automatically added comment] At least one review was discounted during the decision process due to quality]
Decision: Accept