jayesh20/qlora-cyber-security-classifier

Model Information

QLoRA Cyber Security Classifier is a LoRA fine-tuned adapter on top of Qwen2.5-7B-Instruct, trained to detect SQL injection attempts and phishing URLs and explain the reasoning behind each classification. It was trained as an instruction-tuned security triage assistant: given a SQL query or a URL, it returns a Classification: label plus a short Reason: for that call.

Model developer: jayesh20

Model Architecture: Qwen2.5-7B-Instruct (decoder-only transformer) with LoRA adapters injected into attention and MLP projection layers, fine-tuned under 4-bit NF4 quantization (QLoRA).

	Training Data	Params (base)	LoRA rank / alpha	Context length	Token count	Base model release
QLoRA Cyber Security Classifier	SQL injection (Kaggle) + Phishing URLs (HF)	7B	16 / 32	256	~3K training examples (subset)	Qwen2.5, Sep 2024

Supported tasks: binary security classification with explanation, for two domains:

• SQL query → SQL Injection / Benign
• URL → Phishing / Legitimate

Model Release Date: July 2026

Status: This is a research/prototype model trained on a limited subset of data under a tight compute budget (single T4 GPU). See Limitations below.

License: Apache 2.0 for the adapter weights. The base model (Qwen2.5-7B-Instruct) carries its own license — check Qwen's license terms before redistribution or commercial use.

Intended Use

Intended use cases: Assistive triage in a security pipeline — flagging suspicious SQL queries or URLs for human review, or as one signal among several in an automated detection tool. Useful for research and prototyping LLM-based security classifiers.

Out of scope:

• Not a standalone production security gate. This does not replace parameterized queries / prepared statements (the actual defense against SQL injection), a WAF, or established phishing-detection services.
• Not evaluated against adversarial/obfuscated inputs (encoded payloads, homoglyph domains, case-mixing evasion).
• Not intended for classification tasks outside SQL queries and URLs.

How to use

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE_MODEL = "Qwen/Qwen2.5-7B-Instruct"
ADAPTER_REPO = "jayesh20/qlora-cyber-security-classifier"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_REPO)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, device_map="auto"
)
model = PeftModel.from_pretrained(base_model, ADAPTER_REPO)
model.eval()

PROMPT = """### Instruction:
{instruction}

### Input:
{input}

### Response:
"""

def predict(text, task="sql"):
    instruction = (
        "Analyze the following input and determine if it is a SQL injection attempt."
        if task == "sql" else
        "Analyze this URL and classify whether it is phishing or legitimate."
    )
    prompt = PROMPT.format(instruction=instruction, input=text)
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(**inputs, max_new_tokens=100, do_sample=False,
                              pad_token_id=tokenizer.eos_token_id)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()

print(predict("SELECT * FROM users WHERE id = 1 OR 1=1 --", task="sql"))
print(predict("http://paypa1-secure-login.com/verify", task="phishing"))

Training Data

Dataset	Source	Role
SQL Injection Dataset	syedsaqlainhussain/sql-injection-dataset (Kaggle)	Labeled SQL queries (benign / injection)
Phishing URL Dataset	pirocheto/phishing-url (HuggingFace)	Labeled URLs (phishing / legitimate)

Both sources were cleaned (leaked header rows and non-numeric label values removed, deduplicated), converted to instruction / input / output format, class-balanced to a max 3:1 ratio, and split 85/10/5 into train/val/test. Training used a 3,000-example subset of the train split (and 300 of val) to fit a constrained compute budget — see Limitations.

Training Procedure

Method: QLoRA — base model loaded in 4-bit NF4, LoRA adapters trained on top via plain transformers.Trainer (no trl dependency).

LoRA target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Hyperparameter	Value
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.05
Max sequence length	256
Per-device batch size	8
Gradient accumulation	2
Effective batch size	16
Learning rate	2e-4 (cosine schedule)
Max steps	300
Precision	bf16 compute, 4-bit NF4 base weights
Hardware	1x Kaggle Tesla T4

Training Loss

Step	Training Loss	Validation Loss
100	0.1922	0.2044
200	0.1901	0.1922
300	0.1493	0.1896

Final training run summary:

Metric	Value
Global steps	300
Epochs completed	~1.6
Average training loss	0.2993
Training runtime	33,671s (~9.35 hours)
Samples/sec	0.143
Steps/sec	0.009

Both training and validation loss decreased steadily with no signs of divergence, but note that loss going down does not by itself confirm classification accuracy — see Evaluation below.

Evaluation

Evaluated on the held-out test split (1,532 examples) using exact-match comparison between the model's generated Classification: label and ground truth.

Class	Precision	Recall	F1-score	Support
benign	1.00	1.00	1.00	585
legitimate	0.99	0.96	0.97	203
phishing	0.96	0.99	0.97	182
sql injection	1.00	1.00	1.00	562
accuracy			0.99	1532
macro avg	0.99	0.99	0.99	1532
weighted avg	0.99	0.99	0.99	1532

Overall test accuracy: 99%. The SQL injection task (benign / sql injection) is essentially perfect on this test split. The phishing task (legitimate / phishing) is slightly softer, with legitimate URLs occasionally misclassified as phishing (96% recall) and phishing URLs very reliably caught (99% recall) — i.e., the model is a little more likely to over-flag a legitimate URL than to miss an actual phishing one.

Note this reflects performance on a held-out split of the same cleaned dataset used for training — it does not measure generalization to attack patterns or URL structures outside that distribution (see Limitations).

Limitations

• Small training subset: trained on 3,000 of the available examples (not the full cleaned dataset), and for only ~1.6 epochs, in order to fit a ~1-hour-scale compute budget on a single T4. This trades off ceiling accuracy for turnaround time — expect headroom for improvement with more data/epochs.
• Templated explanations: the Reason: text is class-templated rather than generated per-example, so explanations are somewhat generic rather than deeply input-specific.
• No adversarial evaluation: the 99% accuracy above is on a clean held-out split from the same source datasets. Obfuscated SQL payloads (encoding, comment tricks, case-mixing) and homoglyph/lookalike phishing domains were not specifically tested, and performance on those is unknown.
• Long training time relative to budget: the run took ~9.35 hours rather than the intended ~1 hour, most likely due to 7B-parameter 4-bit inference overhead plus gradient checkpointing on a single T4 — worth profiling further if iterating on this model.

Citation

@misc{qwen2.5,
  title={Qwen2.5 Technical Report},
  author={Qwen Team},
  year={2024}
}

Model Card Contact

jayesh20