DarrenSuw/logic-node-sec-8b-lora

Logic Node: MITRE ATT&CK Extraction LoRA

This is a QLoRA fine-tuned adapter for fdtn-ai/Foundation-Sec-8B, designed to extract structured MITRE ATT&CK techniques from unstructured cybersecurity threat intelligence.

Model Details

• Architecture: LLaMA-3.1-8B base + QLoRA ($r=16$, $\alpha=32$)
• Quantization: NF4 (4-bit) with bfloat16 compute
• Hardware: Trained locally on an NVIDIA RTX 5070 Laptop GPU (8GB VRAM)
• Dataset: 19,578 examples from the TRAM dataset

Evaluation & Metrics

The model was evaluated against a held-out ATT&CK v15.1 corpus:

• Hallucination Rate: 0.0%
• Macro Tactic F1: 0.9082
• Optimal Checkpoint: Step 2,000 (eval_loss: 0.2428)

Full training dynamics and empirical validation are available in the Weights & Biases Report on GitHub.

Usage

Because this is a PEFT adapter, it must be loaded alongside the base model.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "fdtn-ai/Foundation-Sec-8B"
adapter_id = "DarrenSuw/logic-node-sec-8b-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id, load_in_4bit=True)
model = PeftModel.from_pretrained(base_model, adapter_id)