Brusnicki/SAVANT-anomaly-classifier-lora

SAVANT Direct Anomaly Classifier (LoRA Adapter)

This repository contains the LoRA adapter for the direct anomaly classification model described in the SAVANT paper, currently under peer review.

This repository is provided for peer-review purposes only. After the review process, the model will be made publicly available through the authors' main account.

Model Description

LoRA adapter for Qwen/Qwen2.5-VL-7B-Instruct, fine-tuned for binary anomaly detection in autonomous driving scenes. Given a front-camera image, the model classifies the scene as anomaly or normal.

This is the direct classification model of the SAVANT framework — a single-shot approach that achieves the best accuracy among the evaluated configurations.

Performance

Evaluated on a balanced test set of 1,020 driving scene images:

Metric	Value
Accuracy	93.8%
Precision	96.7%
Recall	90.8%
F1-Score	93.6%

Training Details

• Base model: Qwen/Qwen2.5-VL-7B-Instruct
• Method: LoRA (Low-Rank Adaptation)
• Dataset: 4,055 balanced samples (anomaly/normal driving scenes)
• Epochs: 3
• Learning rate: 1e-4 (cosine schedule)
• Batch size: 8 per device
• Hardware: 4x NVIDIA RTX 4090
• Precision: bfloat16 with Flash Attention 2

LoRA Configuration

Parameter	Value
Rank (r)	16
Alpha	32
Dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, fc1, fc2, qkv, mlp.0, mlp.2

Usage

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
import torch

# Load base model
base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "u94fmn391j/SAVANT-anomaly-classifier-lora")

# Load processor
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct")

Limitations

• Trained on the CODA dataset; generalization to other driving domains not evaluated
• Single-frame analysis only (no temporal context)