CaDet: AI-generated video detector (VAE Rhythm)

A 4x causal VAE decodes frames in groups, and the phase scan reads the resulting periodic trace

CaDet is the detector of the paper VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection (paper, code, project page).

It attaches two parameter-efficient paths to one frozen DINOv3 ViT-L/16 and reads 8 frames per clip:

  • Artifact path β€” LoRA (rank 16) on the attention qkv projections, trained on AIGVDBench plus VAE pairs (real clips and copies that differ only by one video-VAE round trip). One forward pass feeds a generic head (both pair members counted real) and an artifact head on blocks 8 / 12 / 16 (VAE copies counted generated).
  • Semantic path β€” Effort-style tuning of the smallest singular component of each of the 96 attention projections plus a 2-way head, trained on AIGVDBench only.

The video score is the sum of both paths after standardising each on AIGVDBench validation real videos. It is a z-score sum (β‰ˆ 0 for typical real videos, larger = more likely generated), not a probability.

Files

seed0/ seed1/ seed2/          three training seeds (the paper reports their mean)
  artifact_path.safetensors   LoRA adapters + both heads (β‰ˆ 16 MB)
  semantic_path.safetensors   rank-one SVD residuals + 2-way head (β‰ˆ 0.8 MB)
  calibration.json            head and fusion statistics from AIGVDBench validation reals

The DINOv3 backbone weights are not included; timm downloads them (vit_large_patch16_dinov3.lvd1689m) after you accept the DINOv3 license on Hugging Face.

Usage

pip install "vae-rhythm[cadet] @ git+https://github.com/showlab/vae-rhythm"
hf download Jack97/CaDet --local-dir checkpoints/cadet
vae-rhythm predict my_video.mp4 --weights checkpoints/cadet/seed0
from vae_rhythm.cadet.detector import CaDet
det = CaDet("checkpoints/cadet/seed0")          # needs a CUDA GPU (about 3 GB)
print(det.score_video("my_video.mp4"))           # {'score': ..., 'z_semantic': ..., 'z_artifact': ...}

Frames 0, 4, …, 28 of the clip are centre-cropped to a square and resized to 256 Γ— 256 (JPEG q95), as in training. Clips shorter than 29 frames are padded by repeating frames uniformly.

Results (AUROC %, mean of 3 seeds)

Trained only on AIGVDBench's Open-Sora and real clips (plus VAE pairs for the artifact path); all other generators are unseen.

AIGVDBench 20 open 19 unseen open 11 closed 2026 closed (MiniMax H3, Seedance 2.0 / 2.5) DF26 all DF26 open DF26 closed DF26 paired 2AFC
93.9 93.6 95.8 88.3 78.5 78.2 79.0 87.8

Adding the training-free VAE Rhythm plug-in (phase scan, see the code repository) raises DF26 further.

Intended use and limitations

Research on AI-generated video detection and as one piece of evidence in media verification. Do not use the score alone to make consequential decisions about a person or a video.

  • Operating points must be chosen on real videos from the deployment domain; the score distribution of real videos shifts across sources (in the paper, swapping the real source lowers AUROC for every trained detector).
  • Heavily re-encoded, very short, very low-resolution or edited videos are harder; performance on generators released after the training data is not guaranteed.
  • Trained on AIGVDBench (real clips from OpenVid) β€” content outside that distribution (e.g. talking-head livestreams, screen recordings) has not been characterised.

Citation

@article{sun2026vaerhythm,
  title   = {VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection},
  author  = {Sun, Ke and Li, Shuwei and Mei, Haiyang and Shou, Mike Zheng},
  journal = {arXiv preprint arXiv:2611.21114},
  year    = {2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Jack97/CaDet

Paper for Jack97/CaDet