Soloadio/qwen2vl-7b-fall-lora

Qwen2-VL-7B fall detection LoRA

LoRA adapter for Qwen/Qwen2-VL-7B-Instruct that scores 32-frame camera clips as fall / no fall. Made for the CSC490 fall-detection capstone. Code: research/qwen/.

Research prototype: not tested on continuous video and not a safety device.

Load

import torch
from peft import PeftModel
from transformers import AutoProcessor, Qwen2VLForConditionalGeneration

base_id = "Qwen/Qwen2-VL-7B-Instruct"
base = Qwen2VLForConditionalGeneration.from_pretrained(base_id, dtype=torch.bfloat16, device_map="cuda")
model = PeftModel.from_pretrained(base, "Soloadio/qwen2vl-7b-fall-lora").merge_and_unload().eval()

processor = AutoProcessor.from_pretrained(base_id)
processor.image_processor.size["longest_edge"] = 78000   # pixels per frame; must match training

Tested with transformers 5.18 and peft 0.19.1. The adapter expects the prompt from research/gemma/gemma_binary_scoring.py and the forced answer prefix {"verdict": ". P(fall) comes from the log-probabilities of the two label continuations: research/qwen/run_probs_qwen.py --adapter Soloadio/qwen2vl-7b-fall-lora does this end to end.

Training

• Data: edf, mcfd and up_fall train splits from OmniFall, 2967 camera clips, 32 frames each.
• LoRA on the language model only (vision tower frozen): rank 16, alpha 32, dropout 0.05, q/k/v/o/gate/up/down projections.
• AdamW, lr 1e-4 with cosine decay, 8 clips per step, 1 epoch (371 steps), about 55 min on one H100.
• Loss: cross-entropy on the label tokens after the forced prefix, 0.21 -> 0.09.

Results

Test set, one decision per event (P(fall) averaged over the event's cameras), threshold 0.5.

dataset	balanced acc.	fall recall	precision	AUROC	base model, no LoRA (balanced acc.)
edf	96.7%	100%	81.8%	0.993	78.3%
mcfd	93.8%	100%	81.8%	1.000	80.2%
up_fall	95.2%	100%	87.5%	0.990	95.8%

• The edf and mcfd test sets are small (39 and 25 events), so one event moves a number by 3-4 points.
• mcfd has one actor in one room, so its splits share the same person.
• Clips are cut at labelled event boundaries.