Ali-Bhai/deberta-tool-hallucination-detector

DeBERTa Tool Hallucination Detector

This model is a 4-way classifier for hallucination detection in tool-calling answers.

Labels

[
  "clean",
  "tool_output_conflict",
  "overgeneration",
  "missing_tool_action_recommendation"
]

Input format

The model uses a paired input:

• Segment A: Answer:\n{output}
• Segment B: Question:\n{query}\n\nTool responses:\n{context}\n\nAvailable tool names:\n{available_tool_names}

Calibration

The final prediction applies a validation-only conflict gate:

{
  "mode": "conflict_gate",
  "name": "conflict_gate__minp_0.00__margin_0.20__fallback_best_non_conflict",
  "min_conflict_prob": 0.0,
  "min_conflict_margin": 0.2,
  "fallback_strategy": "best_non_conflict"
}

Final test metrics

{
  "accuracy": 0.927536231884058,
  "macro_f1": 0.9249477834493081,
  "weighted_f1": 0.9287091179440344,
  "binary_macro_f1": 0.9417994376757264,
  "clean_f1": 0.9381443298969072,
  "clean_recall": 0.9479166666666666,
  "conflict_f1": 0.8461538461538461,
  "conflict_precision": 0.8048780487804879,
  "conflict_recall": 0.8918918918918919,
  "overgeneration_f1": 0.9577464788732394,
  "missing_tool_f1": 0.9577464788732394
}

Loading

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("Ali-Bhai/deberta-tool-hallucination-detector", use_fast=True)
model = AutoModelForSequenceClassification.from_pretrained("Ali-Bhai/deberta-tool-hallucination-detector")

For calibrated predictions, use inference.py in this repository.