cve-severity-net

Fine-tuned google/flan-t5-base (250M parameters) on 200k NVD CVE description → CVSS v3.1 base vector pairs. Predicts all 8 base metrics from free-text vulnerability descriptions.

Trained on CVEs from 2016–2023. Tested on a held-out 2025 split (temporal, never seen during training).

Evaluation (2025 test set, n=40,179)

Metric	Score
Exact vector match	44.9%
Severity band accuracy	67.4%
Score MAE	0.91

Per-component accuracy

AV	AC	PR	UI	S	C	I	A
93.9%	91.6%	75.8%	92.5%	91.6%	81.3%	81.8%	80.7%

PR (Privileges Required) and C/I/A (Impact) are the hardest components — advisory text rarely encodes privilege context or impact scope explicitly.

Baseline comparison (1000-CVE subsample, seed=42)

Model	Exact Match	Band Accuracy	MAE	Cost/1k CVEs
Majority class (always "High")	0.0%	~45%	~1.8	$0
FLAN-T5-base fine-tuned (this model)	45.9%	66.9%	0.93	$0
Claude Opus 4.8 (zero-shot)	34.5%	62.6%	0.97	$2.07
Claude Opus 4.8 (few-shot)	33.0%	60.0%	1.08	$4.29
GPT-5.4 (zero-shot)	26.5%	57.8%	1.23	$0.68
GPT-5.4 (few-shot)	30.5%	55.9%	1.25	$1.30
Gemma4 E4B local (zero-shot)	12.9%	38.0%	1.93	$0
Gemma4 E4B local (few-shot)	14.3%	37.7%	1.89	$0

Limitations

• English descriptions only — non-English CVEs are excluded during training and prediction
• CVSS v3.1 base vector only — no temporal, environmental, or threat metrics
• 44.9% exact match means ~55% of predictions have at least one wrong component — do not use as a sole authoritative source for security scoring decisions
• Novel attack patterns not represented in 2016–2023 training data may degrade accuracy
• Few-shot prompting of frontier LLMs did not improve their performance on this task

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("harinpurumandla/cve-severity-net")
model = AutoModelForSeq2SeqLM.from_pretrained("harinpurumandla/cve-severity-net")

description = "A heap-based buffer overflow allows unauthenticated remote code execution."
inputs = tokenizer(
    "predict cvss vector: " + description,
    return_tensors="pt",
    truncation=True,
    max_length=512,
)
outputs = model.generate(**inputs, num_beams=4, max_new_tokens=48)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H

Training details

• Base model: google/flan-t5-base
• Dataset: 200k NVD CVE description → CVSS v3.1 vector pairs (2016–2023)
• Epochs: 5, best checkpoint at epoch 4 (val exact vector match: 49.2%)
• Hardware: single NVIDIA RTX 3090, fp16
• Input format: "predict cvss vector: {description}"