HappyGook/videomae-violence-detector

Model Card for Model ID

This model is a fine-tuned version of MCG-NJU/videomae-base for binary violence classification (violent / non-violent).

It builds on Nikeytas/videomae-crime-detector-production-v1, which was itself fine-tuned from videomae-base on a subset of UCF Crime. Starting from that checkpoint, this model was further fine-tuned on the Bus Violence Dataset to close the domain gap to public-transport surveillance footage.

Dataset & Training

Fine-tuned on:

• UCF Crime (jinmang2/ucf_crime) — inherited from the base checkpoint
• Bus Violence Dataset (Zenodo) — real moving-bus footage, binary violent / non-violent labels, used for domain-specific fine-tuning

Performance

Evaluated on a held-out Bus Violence Dataset test split (n = 280).

Model	n	Accuracy	Error	FPR	FNR	Precision	Recall	TP	TN	FP	FN
Nikeytas/videomae-crime-detector-production-v1 (zero-shot, no bus-violence fine-tuning)	280	0.48	0.52	0.64	0.40	0.48	0.60	84	50	90	56
This model (fine-tuned on Bus Violence Dataset)	280	0.87	0.13	0.11	0.14	0.88	0.86	120	124	16	20

The base UCF-Crime checkpoint transfers poorly to on-bus footage out of the box (near chance-level accuracy, high false-positive rate), which is consistent with the domain-shift findings reported for transport-surveillance data (illumination changes, vehicle/passenger motion, scrolling backgrounds). Fine-tuning on in-domain bus violence footage substantially improves all metrics.

Quick Start

pip install transformers torch torchvision opencv-python pillow

import torch
from transformers import AutoModelForVideoClassification, AutoProcessor
import cv2
import numpy as np

# Load model and processor
model = AutoModelForVideoClassification.from_pretrained("<your-repo-id>")
processor = AutoProcessor.from_pretrained("<your-repo-id>")

def classify_video(video_path, num_frames=16):
    cap = cv2.VideoCapture(video_path)
    frames = []

    total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
    indices = np.linspace(0, total_frames - 1, num_frames, dtype=int)

    for idx in indices:
        cap.set(cv2.CAP_PROP_POS_FRAMES, idx)
        ret, frame = cap.read()
        if ret:
            frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            frames.append(frame_rgb)
    cap.release()

    inputs = processor(frames, return_tensors="pt")
    with torch.no_grad():
        outputs = model(**inputs)
        predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
        predicted_class = torch.argmax(predictions, dim=-1).item()
        confidence = predictions[0][predicted_class].item()

    label = "Violent" if predicted_class == 1 else "Non-Violent"
    return label, confidence

video_path = "path/to/your/video.mp4"
prediction, confidence = classify_video(video_path)
print(f"Prediction: {prediction} (Confidence: {confidence:.3f})")

Technical Specifications

• Base Model: MCG-NJU/videomae-base (via Nikeytas/videomae-crime-detector-production-v1)
• Architecture: Vision Transformer (ViT) adapted for video
• Input Resolution: 224x224 pixels per frame
• Temporal Resolution: 16 frames per video clip
• Output Classes: 2 (binary: violent / non-violent)
• Training Framework: HuggingFace Transformers