Debarun12/cybersec-hallucination-guard

model_card_content = """--- license: apache-2.0 base_model: Qwen/Qwen2.5-1.5B-Instruct library_name: peft tags: - cybersecurity - hallucination-detection - rag - lora - question-answering - groundedness language: - en pipeline_tag: text-generation

Cybersec Hallucination Guard

Cybersec Hallucination Guard is a LoRA fine-tuned adapter for Qwen2.5-1.5B-Instruct, built to detect when a retrieved context does not contain enough information to answer a given question — reducing hallucinated responses in Retrieval-Augmented Generation (RAG) pipelines within the cybersecurity domain.

Instead of generating a confident but fabricated answer when the source material doesn't support it, the model is trained to explicitly refuse and respond with "I couldn't find this information."

Model Description

• Base model: Qwen2.5-1.5B-Instruct
• Fine-tuning method: LoRA (Low-Rank Adaptation)
• Domain: Cybersecurity
• Task: Groundedness / hallucination detection for RAG systems
• Input: A question and a retrieved context
• Output: A structured decision (Grounded: yes/no) followed by either a grounded answer or an explicit refusal

Intended Use

This adapter is designed to sit between a retrieval step and an answer-generation step in a RAG pipeline. Given a question and the context returned by a retriever, it:

1. Evaluates whether the context actually contains the answer
2. If yes → generates a concise, grounded answer with a supporting source quote
3. If no → refuses honestly instead of guessing

Intended users: Developers building RAG systems, researchers exploring hallucination mitigation, and anyone experimenting with groundedness-aware QA in a security context.

Not intended for: General-purpose chat, open-domain QA outside cybersecurity content, or use as a standalone knowledge source (it does not answer questions from its own parametric knowledge — it strictly evaluates and answers from the provided context).

Input / Output Format

Input prompt format:

Question: <question>
Context: <retrieved context>

Output format (grounded case):

Grounded: yes
Answer: <answer derived from context>
Source: "<supporting quote from context>"

Output format (ungrounded case):

Grounded: no
Answer: I couldn't find this information.

How to Use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-1.5B-Instruct"
ADAPTER = "Debarun12/cybersec-hallucination-guard"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, ADAPTER)
model.eval()

prompt = \"\"\"Question: What is MAC address filtering?
Context: A firewall is a network security device that monitors incoming and outgoing traffic based on predefined security rules.
\"\"\"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=80, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True)[len(prompt):])

Training Data

The model was fine-tuned on a custom dataset built from cybersecurity source documents (PDFs and Word files covering security policies, frameworks, attack types, and best practices), processed as follows:

1. Source documents were chunked into context-sized passages
2. A local LLM generated question-answer pairs grounded in each chunk (positive examples)
3. Hard negative examples were constructed by pairing questions with topically similar but non-answering contexts, using embedding similarity search followed by LLM verification
4. Easy negative examples (missing/unrelated context) were added for baseline refusal behavior
5. All (question, context) pairs were checked for label contamination to ensure no pair appeared as both grounded and ungrounded

Dataset size: ~2,400 training examples (train/val/test split)

Evaluation

Evaluated on a held-out test set of 299 examples:

Category	Accuracy
Overall	91.0%
No context provided	100%
Positive (answerable)	95.5%
Random / unrelated context	88.2%
Hard negative (topically related, non-answering)	55.2%

Known limitation: The model performs well on clear-cut cases but is more likely to hallucinate a grounded answer when the retrieved context is topically related to the question but doesn't contain the specific answer. This reflects the general difficulty of fine-grained groundedness detection and is an active area for improvement (e.g., via more extensive hard-negative training data).

Limitations

• Trained on English-language cybersecurity content; performance on other domains or languages is untested
• Hard-negative detection accuracy is a known weak point (see evaluation above)
• Occasionally continues generating beyond the intended output format; downstream use should apply output parsing/truncation
• Not evaluated for adversarial or intentionally misleading contexts

License

This adapter is released under the Apache 2.0 license, consistent with the base model's license. See Qwen2.5-1.5B-Instruct for base model licensing details.

Citation

If you use this model, please reference:

@misc{cybersec-hallucination-guard,
  author = {Debarun},
  title = {Cybersec Hallucination Guard: A LoRA-tuned Groundedness Model for Cybersecurity RAG},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Debarun12/cybersec-hallucination-guard}
}

"""

if name == "main": with open("README.md", "w", encoding="utf-8") as f: f.write(model_card_content) print("README.md generated successfully!") print("Upload this file to the root of your Hugging Face model repo:") print("https://huggingface.co/Debarun12/cybersec-hallucination-guard")