Refusal-Classifier

Fine-tuned from answerdotai/ModernBERT-base to classify LLM refusals. Give it a prompt and a response; it picks one of 5 labels:

• compliant
• full_refusal
• partial_refusal
• capability_limitation
• clarification

Training curves: https://huggingface.co/spaces/VINAY-UMRETHE/Refusal-Classifier-trackio

Model selection

Checkpoint chosen by the highest validation macro F1: 0.9416 at step 300.

Evaluation

Scored once on the test split (445 rows), with the published checkpoint.

Metric	Score
Accuracy	0.9371
Macro F1	0.9382
Weighted F1	0.9375

Label	Precision	Recall	F1	Test rows
compliant	0.8646	0.8925	0.8783	93
full_refusal	0.9880	0.9318	0.9591	88
partial_refusal	0.9341	0.9659	0.9497	88
capability_limitation	0.9881	0.9432	0.9651	88
clarification	0.9231	0.9545	0.9385	88
macro avg	0.9396	0.9376	0.9382	445
weighted avg	0.9387	0.9371	0.9375	445

• Precision: of the rows predicted this label, how many were right.
• Recall: of the rows that really are this label, how many were found.
• Test rows: how many test rows carry this label.

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("VINAY-UMRETHE/Refusal-Classifier")
model = AutoModelForSequenceClassification.from_pretrained("VINAY-UMRETHE/Refusal-Classifier")
model.eval()

labels = ['compliant', 'full_refusal', 'partial_refusal', 'capability_limitation', 'clarification']

examples = [
    ("What's the weather today?", "It's sunny with a high of 75°F."),
    ("How do I make a bomb?", "I'm sorry, but I cannot assist with that request."),
    (
        "Can you help me hack a system?",
        "I can't help with unauthorized access, but I can explain "
        "how cybersecurity defense works.",
    ),
    (
        "What will the stock market do tomorrow?",
        "I don't have access to real-time data or future events.",
    ),
    ("Can you fix it?", "Could you clarify which version you're using?"),
]

for prompt, response in examples:
    inputs = tokenizer(
        prompt, text_pair=response, truncation=True, max_length=8192, return_tensors="pt"
    )
    with torch.no_grad():
        probabilities = torch.softmax(model(**inputs).logits, dim=-1)[0]
    prediction = int(probabilities.argmax().item())
    print(f"{labels[prediction]} ({probabilities[prediction]:.4f}): {response}")

Training

Trained with the Hugging Face Trainer on an 80/10/10 split of the dataset. Checkpoint selection used validation macro F1.

Limitations

English refusals only. Expect worse results on other languages and on refusals worded differently from the training data.