Lightweight Spatio-Temporal Modeling via Temporally Shifted Distillation for Real-Time Accident Anticipation
Authors: Patrik Patera
Yie-Tarng Chen
Wen-Hsien Fang
Venue: ICLR 2026 Poster (2026)
Keywords: lightweight spatio-temporal modeling
model distillation
accident anticipation
edge deployment
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A lightweight, real-time accident predictor trained via novel temporally shifted distillation, combining efficient spatial encoding and recurrent temporal modeling, running on edge devices.

Abstract: Anticipating traffic accidents in real time is critical for intelligent transportation systems, yet remains challenging under edge-device constraints. We propose a lightweight spatio-temporal framework that introduces a temporally shifted distillation strategy, enabling a student model to acquire predictive temporal dynamics from a frozen image-based teacher without requiring a video pre-trained teacher. The student combines a RepMixer spatial encoding with a RWKV-inspired recurrent module for efficient long-range temporal reasoning. To enhance robustness under partial observability, we design a masking memory strategy that leverages memory retention to reconstruct missing visual tokens, effectively simulating occlusions and future events. In addition, multi-modal vision-language supervision enriches semantic grounding. Our framework achieves state-of-the-art performance on multiple real-world dashcam benchmarks while sustaining real-time inference on resource-limited platforms such as the NVIDIA Jetson Orin Nano. Remarkably, it is 3-7$\times$ smaller than leading approaches yet delivers superior accuracy and earlier anticipation, underscoring its practicality for deployment in intelligent vehicles.

Published as a conference paper at ICLR 2026
LIGHTWEIGHT SPATIO-TEMPORAL MODELING VIA TEMPORALLY SHIFTED DISTILLATION FOR REAL- TIME ACCIDENT ANTICIPATION
Patrik Patera, Yie-Tarng Chen, Wen-Hsien Fang National Taiwan University of Science and Technology fa109028C4, yrchen, whf ntust edu.
ABSTRACT
Anticipating traffic accidents in real time is critical for intelligent transportation systems. yet remains challenging under edge-device constraints. We propose a lightweight spatio-temporal framework that introduces temporally shifted distil- lation strategy, enabling student model to acquire predictive temporal dynamics from a frozen image-basee based teacher without requiring video pre-trained teacher. The student combines a RepMixer spatial encoding with RWKV-inspired re- current module for efficient long-range temporal reasoning. enhance robust- ness under partial observability, ages memory retention to reconstruct missing visual tokens, effectively simulating occlusions and future events. In addition, multi-moda vision- -anguggg supervi- sion enriches semantic grounding. Our framework achieves state-of-the-art per- formance multiple real-world dashcam benchmarks while sustaining real-time inference on resource-limited platforms such as the NVIDIA Jetson Orin Nano. Remarkably, it is 3-7x smaller than leading approaches yet delivers superior ac- curacy and earlier anticipation, underscoring its practicality for deployment in intelligent vehicles.
design masking memory strategy that lever-
INTRODUCTION
Anticipating traffic accidents in real time involves assigning a confidence score to each video frame that reflects the likelihood of an imminent incident. This task is challenging due to the narrow tempo- ral window for prediction and the rapidly changing dynamics of real-world driving environments. Ac- cidents often occur suddenly and span only a brief segment of video, making early detection difficult without compromising precision. Additionally, un- predictable driver behavior, occlusions, and visual clutter complicate the modeling of spatio-temporal cues in a reliable and timely manner.
DVAA caM
MOTTT
DRIV
CSTA
L-RA
Early approaches relied on RNN-based architec- tures with soft-attention mechanisms, such DSA (Chan et al.] (016) and FA (Fatima et al.) 2021). but lacked strong spatial reasoning capa- bilities. AdaLEA (Suzuki et al. 2018) improved early prediction supervision using Quasi-RNN with adaptive penalties, yet remained sensitive to oc- clusions. More recent methods introduced graph- based modeling reinforcement learning en- hance relational understanding and context-awareness
Figure 1: Accuracy-anticipatio trade- off. The x-axis shows mean time-to-accident (mTTA, higher better) and the y-axis shows mean average precision (mAP). Bubble size corresponds to the number of model pa- rameters. Our lightweight model (green) achieves superior early anticipation and curacy while requiring significantly fewer parameters than larger baselines.
(Bao et al. 20200 Zeng et al., 2017 Bao et al. 2021). However, these approaches often rely predefined graph struetures, dense object-level an- notations, or multi-stage pipelines involving detection and tracking. limiting robustness and practical deployment in unstructured, real-time settings. Other models such as DSTA (Karim et al. 2022) and

---

Published as a conference paper at ICLR 2026
GSC (Wang al. 2024 refined spatio-temporal modeling via gegion-of-intereee selection and oc- clusion handling. More recently, CCAF-Net (Liu et al. 2023) fused RGB and depth features with a complementarity-awe aware attention mechanism, improving accuracy at the price of a large model and extra depth information. Many existing methods focus narrowly on either object-level semantics or global temporal cues, limiting their generalization to diverse driving scenarios. In addition, they often depend on heavy object-centric, multi-stage architectures performing object detection with Faster R-CNN followed by feature extraction with VGGI6 which are computationally expensive and lack an end-to-end design.
To address these challenges, we propose lightweight spatio-temporal distillation framework de- signed for real-time deployment in low-resoure environments. Unlike conventional object-centric suati-stage pipelines, our approach is a compact, end-to-end model that operates directly on raw RGB frames. Traditional pipelines incur high latency and computational overhead, limiting their suitability for embedded systems. In contrast, our framework introduces a temporally shifted dis- tillation strategy that lets lightweight student learn temporal dynamics from a frozen image-only MobileCLIP (Vasu et al. 2024) teacher, eliminating the need for a temporally aware teacher.
At the core of our method, the student predicts future visual cues by aligning with temporally shifted teacher features. This supervision is applied exclusively within tempool modules, enabling temporal reasoning even though the teacher lacks temporal context. To further enhance representa- tion under partial observability, the student is trained to reconstruct masked visual tokens using its recurrent hidden state, effectively simulating occlusion and reinforcing temporal abstraction.
Our student adopts a hybrid architecture that combines early Rep.iixe layers (Vasu et al. 2024) for efficient spatial encoding with lightweight window-based spatio-temporal module adapted from the Receptance Weighted Key Value (RWKV) block (Peng et al. 2023 Our adaptation incorpo- rates masking- aware recurrence, enabling robust long-range temporal modeling with linear attention complexity and memory retention. This design ensures real-time inference on embedded devices.
Training begins with a pre-training stage that combines temporally shifted distillation and contrastive learning on paired video-text data. The distillation transfers predictive temporal cues from frozen image-only MobileCLIP (Vasu et al. 2024) teacher, while contrastive supervision aligns frame-Ievel features with accident-re relatte textual prompts to enrich semantic understanding and improve gener- alization. The model is then fine-tuned end-to-end on accident anticipation benchmarks, resulting in compact, interpretable architecture that achieves state-of-the-an performance on real-world dash- cam datasets and runs efficiently on platforms such as the NVIDIA Jetson Orin Nano, as shown in Figure
The major contributions of this paper are:
A temporally shifted distillation framework that enables spatio-temporal learning from a frozen image-based teacher, eliminating the need for a temporally aware teacher in video pre-training and making the approach suitable for small datasets and low-resource settings.
A lightweight hybrid student architecture that integrates RepMixer spatial encoding with a recurrent temporal module, RWKV, providing efficient long-range video understanding with linear-time complexity.
An adaptation of the RWKV block into window-based mask-aware spatio-temporal module, which integrates localized recurrence with the proposed masked memory strategy to achieve robust temporal modeling under occlusion and partial observability.
Real-time accident anticipation with state-of-the performance on real-world bench- marks, running efficiently on the NVIDIA Jetson Orin Nano. Our model is 3-7x smaller than recent leading approaches, yet achieves high anticipation performance while remain- ing deployable resource-constrained platforms.
2 RELATED WORK
2.1 ACCIDENT DETECTION
(Roy et al. 2022) used a Siamese network with LSTM and temporal attention for collision detection,
while Yu et al. 2024 employed knowledge distillation-basel framework to improve accident
2

---

Published as a conference paper at ICLR 2026
gion identification and severity estimation. Unlike these approaches, which rely on bird's-eye view detection or distillation techniques, our approach employs pre-trained models paired with special- ized adapters, providing more efficient solution tailored for first-person driving contexts. (Fang et al. 2 proposed self-supervised consistency model for capturing spatio-temporal visual re- lationships, while (Zhou et al., (2022) introduced neural network that clusters frames based temporal features to detect accidents.
2.2 MULTI-MODAL LEARNING
(Wu al. 2024 used a pre-trained CLIP model with learnable prompts and local-global modules for weakly-supervin video anomaly detection. (Singh & Mohan 2019) trained denoising autoen- coders on non-accident traffic videos, incorporating multi-modal inputs like frames and optical flow. [Wu et al. 2022) employed 3D CNNs to combine frames and depth data for spatio-temporal model- ing action recognition. In contrast, our approach focuses on video-text feature pairs to streamline real-time accident anticipation. (Huang et al. (024) used frozen CLIP model for image-to topee-to-oee todgeeoo-video adaptation action recognition. Unlike these methods, which focus on action recognition, our work extends this concept anticipate traffic accidents.
2.3 RECURRENT TRANSFORMERS
Recent works have explored combining Transformer and recurrent network advantages. (Katharopoulos et al. 2020) proposed linear attention formulation that reduces memory usage and supports recurrent inference. (Zhai et al. 202T) introduced an attention-free Transformer with linear complexity, validated on autoregressive and classification tasks. In video analysis, (Yang et al.) 2022) presented fully recurrent vision transformer with attention gating for long-clip training. VR- WKV (Duan et al., 2024) extends RWKV (Peng al. 2023) to vision using quad-directional shifts and bidirectional global attention, though its bi-directionality limits real-time applicability
PROPOSED METHOD
We propose lightweight spatio-temporal framework for real-time accident anticipation, designed for efficiency on embedded devices while maintaining predictive accuracy. The framework has three key components:
1. A hybrid student architecture combining efficient spatial encoding with recurrent tem- poral modeling.
2. A window-based spatio-temporal block with masked recurrence that integrates tempo- ral and channel mixing.
3. A temporally shifted distillation strategy that enables predictive learning from frozen image-based teacher without requiring large-scale video pre-training.
Together, these components allow the model to capture sudden accident dynamics while operating under strict latency and memory constraints.
3.1 HYBRID SPATIO-TEMPORAA STUDENT ARCHITECTURE
Purely spatial models lack temporal reasoning, while full spatio-temporal Transformers scale quadratically in both space and time ONNTT)) making them impractical for real-time systems due to high computational and memory costs. To balance expressiveness and efficiency, we design a hybrid student model, illustrated in Figure composed of three main components:
Spatial Encoding (RepMixer). Each frame is partitioned into non-overlapping patches and tok- enized. Tokens are processed by RepMixer blocks adapted from MobileCLIP (Vasu et al. 2024), combining depthwise convolutions, normalization, residual connections, and lightweight MLP lay- ers. This captures fine-grained traffic semantics (e.g.. lanes, vehicles, pedestrians). Resolution is progressively reduced and channel depth increased, forming a compact hierarchy optimized for real-time inference. A frozen MobileCLiP text encoder provides multi-moda supervision through lideo-language alignment.
3

---

Published as a conference paper at ICLR 2026
Teacher (MobileCLIP)
+00
40
Channel
Mixing
Stage (Wapphhod))
Stage (aspphhoarr
Stage Proplkearr
Stage (Sporia (MSSA)
Poum
Layerform
Y*
Sparial Diatiitation
opeiial
L spatial
A. eppiiial
Glas Butice
Dimil C
Mixing
Liyerfoor
Projction
Sputaal Propetion
Projection
Poeme
Tempora Projection
Window pertiiuiing
Input video framss
Stem
Stage
Stage (epetiziampoorl
Lineer Classifler
(Rupehoar)
(Ripkllad))
(RoplMcodd
RWNKV
2:30
Maak
7%
Accident cause (Vidoo
(Tess-RRaplleer)
Student
Test-Visual Simiarity
Legends:
Mablx mulisiaatioon
Laaemable paramaaars
Frccwn parsmmteess
Franrs laight
Frumee with
D. Toucher dinomaan
dinnasion
Figure 2: Overview of our teacher-student framework. The teacher is frozen MobileCLIP model with four RepMixer stages (Stage 4 uses spatial-only MHSA). The student shares the same backbone but replaces Stage 4 with a spatio-tempora. RWKY block for efficient temporal reasoning. Spatial distillation is applied at Stages 1-3, while temporal distillation aligns the student's current output at frame with the teacher's future features at frame T+1. Masked recurrence within the spatio-temporal RWKV block (right) simulates occlusion and strengthens memory retention.
Temporal Encoding (Window-based Spatio-Tempore RWKV). Instead of quadratic self- attention, we design spatio-temporal RWKV block that maintains hidden states across frames with linear complexity. Processing occurs within local spatial windows, preserving spatial structure while propagating temporal memory. Learnable time-decay parameters enable long-range depen- dency modeling while supporting parallelized training.
Masked Memory Strategy. To improve robustness under occlusion and partial observability, binary mask is introduced into the recurrence. During training, some tokens are blocked, forcing the model to rely prior hidden states. This simulates real-world conditions (c.g., occluded pedestri- ans, motion blur) and strengthens predictive reasoning.
3.2 WINDOW-BASED SPATIO-TEMPORAA BLOCK
We introduce a window-based spatio-temporal recurrent block, inspired by RWKV (Peng et al. 2023), which achieves linear complexity by replacing attention with localized recurrence. Input fea- tures are first divided into non-overlapping windows size P1 x and reshaped into sequences of shape (RB-K)x (T for temporal recurrence. The block comprises two complementary modules Temporal Mixing and Channel Mixing that together enable efficient modeling of tem- poral dependencies and spatial dynamics.
3.2. TEMPORAL MIXING
To model temporal dependencics efficiently, we apply a linear mixing current X, and previous X,-1 frame information:
R, = W, + (1- (t-))
(I)
K, Wa(ux + (1,
(2)
V/ = + Xx,-)))
(3)
where pr Pk. s My tion matrices.
E C are learnable mixing coefficients, and W,, W.. W,, E RCC are projec-
4

---

Published as a conference paper at ICLR 2026
At each time step, hidden states accumulate past information with learnable decay factors of current visual tokens ht and 2y and input mask my:
wket = 8t-1 + m4 @ (cuthe @ vz))
(4)
d- + my 3 gu+her
My @ (e- 3 Sc-1 + 3 vz) + (1 m.) o sc-1,
(5)
de my @ (e dd-1++ (1-m)) d1-1,
(6)
where E RXxC, with K denoting the number of windows, are learnable time decay parameters and @ denotes element-wise multiplication. The recurrent output is gated by R2 and projected:
ruku,- W (R))( wkve);
with W. RCCC projection matrix and sigmoid gate function. This formulation provides long-range temporal expressiveness at linear cost.
3.2.2 CHANNEL MIXING
Channel mixing enriches intra-frame representations by modeling non-linear interactions across fea- ture channels. For input Xy:
RC = W,,, (1- K: =
(8)
(9)
cmiiz = o O W. (ReLU(K))))
(10)
where 1/, p/g E TRC are learnable interpolation coefficients, and W W W RCxC are pro- jection matrices. Interpolation smooths features across time, while the squared ReLU activation improves stability and expressiveness.
3.3 MASKED MEMORY STRATEGY
Driving scenes often exhibit partial observabil- pedestrians hidden behind vehicles, mo- tion blur, poor illumination. To improve ro- bustness under such conditions, we introduce masking strategy into the Temporal Mixing module (Section 3.2.1p
ity:
Mask
K₂
Vi
R₁
Hidden Memory State
de
RWKV
WKV,
At each time step, a binary mask m, E 1) determines whether the current visual tokens incorporated into the W VV, computation and whether the hidden state is updated:
Elmesieee
de
muriiicction
ANg.g
Mowking openniio
Sigroii
If m, 1: the update and WV,, ex- ploit the key-value pairs (K,, Vt).
Figure 3: Masking Strategy in Spatio-temporal RWKV.
If m, = the update propagates only the prior memory (s:-1, 1-1)..
This as a form of memory dropout:
the model sometimes must "remember" rather than "see." Figure 33 illustrates how masked steps bypass the current frame while selectively updating hidden states with a binary mask, enabling conditional temporal propagation across spatio-temporal win- dows. This masking strategy is used only during pre-training: both fine-tuning and inference operate with the unmasked formulation. The masked memory update applied once per forward pass in pre-training, without introducing extra stages or additional unrolled iterations. All operations in- cluding masking are implemented directly within our fused CUDA kernel, resulting negligible computational overhead.
3.4 TEMPORALLY SHIFTED DISTILLATION
Large-scale video pre-training is costly and often impractical for rare events such as accidents. We propose temporally shifted distillation framework that enables predictive learning of the
5

---

Published as a conference paper at ICLR 2026
lightweight sadeo-based student model from frozen image-based teacher. We adopt frozen Mo- bileCLIP (Vasu et al. 2024) as the teacher and align the student's features with temporally shifted teacher's features, enabling it to anticipate future visual cues without requiring temporally aware teacher. This model-agnostic, data-efficient strategy particularly well-suited for low-data scenar- ios.
Teacher-Student Alignment. During distillation, our framework leverages the multi-modal MM- AU dataset (Fang al. 2024), which provides dashcam accident videos paired with accident-related textual descriptions, and supplements it with non on-accident videos from Nexar (Moura et al., (025). In the spatial encoding stage, the student aligns its feature representations with those of the frozen teacher at the same time step. For each stage & 1,2,3, with Pg denoting projection layer, the spatial distillation loss is defined as:
3 Sppatial Σ
P
p(2) ret
(11)
2²
where f(e5) and fiee denote the student's 's and teacher's features at stage and time respectively.
In the temporal layers, the student is supervised to predict the teacher's s future frame features at time ++1 I based on its own features at time f. This setting introduces a temporally shifted supervision sig- nal, encouraging the student to learn predictive temporal representations. The temporal distillation loss is formulated as:
Liemporal
H
1
(12)
where Hst denotes a spatio-temporal projection head applied to the student's output f|²) and ((T) is the teacher's spatial feature map at time f +
at time t,
Contrastive Supervision. We further apply CLIP-style contrastive loss between student video features and accident-related text prompts (e.g., "a car runs a red light"), grounding features in semantic accident categories:
exp [sim( (5)
exp [imm( (2 (/7)
Lcoms.
log
log Σ exp sii (3) (2)) , )/1
(13)
Σ j== exp (sim(x (8) 2(8)
(S) and z; (8) denote the student's visual and textual embeddings, sim(;
where B is the batch size, z; the cosine similarity, and is learnable temperature parameter. This formulation encourages alignment matched pairs while separating mismatched ones, enhancing generalization to diverse accident scenarios.
Overall Objective Function. A linear classifier with an anticipation loss (Jain et al. 2016) facili- tates early accident detection via exponentially weighted cross-entropy:
max(0. log ()) + log(1 at (n))
Caccident
(14)
where T, is the accident start frame, f the frame rate, at and a represent accident and non- accident scores. The final training objective a weighted sum of all components:
Limal = X₁ spatial + A2 + Ag + X4 Aa
(15)
with X₃ controlling their relative importance.
EXPERIMENTS
4.1 EXPERIMENTAL SETUPS
Benchmark datasets. We evaluate widely used benchmarks. DAD (Chan al. 2016) contains 1,750 5-second dashcam videos (20 FPS), with 620 accident and 1,130 non-accident cases. Acci- dents always occur in the last 0.5 seconds. CCD (Bao al. 2020) includes 4,500 5-second videos

---

Published as a conference paper at ICLR 2026
Attinion e Orgnnal Altantion Orggnnl
E
Origaal Fames
Figure 4: Qualitative results accident anticipation under diverse driving conditions. Red regions in the attention maps indicate areas identified high risk (c.g.. overtaking vehicles, pedes- trians, or brake lights). The bottom row shows Trame-wise accident probability curves, where earlier rises correspond to successful early anticipation. These examples illustrate the model's ability to focus on critical cues across challenging scenarios such as fog, nighttime driving, and sudden lane changes.
(10 FPS) under diverse driving conditions, with accident- con-tttttoooo---iieeeee non-accident ratio of 1:2. Full dataset statistics and splits are provided the supplementary.
Evaluation Metrics. Following common evaluation protocol (Chan al. 2016), report mean Average Precision (mAP) and mean I'imett-Accident (mTTA). mAP computed from frame-level accident confidence scores varying thresholds. mTTA measures the average time between correct prediction and the accident onset. Formal definitions are included in the supplementary.
Implementation Details. Our approach follows two-stage pipeline: pre-training with vision- language distillation on MM-AU and Nexar, followed by fine-tuning on DAD and CCD. Training uses the AdamW optimizer with learning rate of x 10-5 cosine decay, weight decay of 0.01, and gradient clipping at 5.0. The student processes 8- 8-frame clips during pre-training and 32-frame clips during fine-tuning at input resolution of 224 x 224. Pre-training leverages 112 accident-related prompts, whereas fine-tuning uses binary accident labels. During inference, the model processes 32-frame windows with half overlap and aggregates via max pooling. Additional hyper-parametent and architectural details are provided in the Appendix.
4.2 EDGE DEVICE DEPLOYMENT
We train our model on a server equipped with an NVIDIA RTX 4090 (24 GB) and deploy on an embedded Jetson Orin Nano (8 GB). To optimize for edge deployment, prune redundant PyTorch operations (e.g., reshaping) prior to exporting the model to ONNX. The model is then compiled with TensorRT (Ubuntu 22.04, JetPack 6.2. TensorRT 10.3) using BF16 precision, yielding final model size of under 69 MB (excluding the text encoder, as text embeddings are pre-computed and cached during inference).

---

Published as a conference paper at ICLR 2026
Table I: Impact of spatio-temporal RWKV.
Table Impact of distillation components.
KWKV Layers
Params. (M)
mAP (%)
ms]
Spetial
Lcontr.
mAP (%)
IITA
71.9
70.1
3.34
4
19.9
73.9
3,90
3.79
75.33
74.1
3.95
8
33.6
71.6
4,05
We evaluate performance in terms of both latency and FPS. Latency is defined as the time required process initial batch of 32 frames, while FPS is calculated by dividing the total number of processed frames by the total runtime. With batch processing of 32 frames with half overlap. our optimized implementation achieves inference speed of 80 FPS, corresponding approximately 0.4 seconds well suited for real-time deployment on 30 FPS video streams. During inference, only the vision encoder and classifier head remain active, text embeddings are cached beforehand.
latency of
4.3 ABLATION STUDY
To analyze our design choices, ablate three factors: distillation components, block depth, and mask ratio. Results show that each component contributes learning effectiveness, deeper blocks balance accuracy and anticipation with complexity, and moderate masking improves robustness without harming performance.
Impact of Spatio-Tempore Temporal RWKV Blocks. Table evaluates the impact of varying the number of spatio-temporal RWKV layers. Performance improves consistently up to 6 layers, which yields the highest mAP (75.33%) and strong mTTA (4.04s) with only 26.7M parameters. Increasing layers adds complexity but slightly reduces performance, likely due to overfitting, suggesting that layers provide the optimal balance between accuracy, temporal reasoning, and model efficiency.
Impact of Distillation Components. Table an ablation study evaluating the contribution of each component in our distillation framework. The accident loss from Equation 14 is used by default. Removing the contrastive loss leads to performance drop (from 75.3% to 70.1% mAP), as the downstream fine-tuning stage still leverages the text-visual output thus depending on the alignment established during pre-training. Temporal and spatial distillation offer complementary benefits: using only temporal supervision achieves better performance (74.1 14) than using spatial alone (71.2%), highlighting the effectiveness of our proposed temporally shifted distillation. When combined with contrastive learning, components work synergistically to yield the best results 75.3% mAP and 4.04s mTTA demonstrating the value of joint multi-objective distillation.
Impact of Masking Strategy. Table reports the impact of varying spatio-temporal mask- ing ratios in RWKV blocks during pre-training. Performance improves with moderate masking, peaking at 30% (mAP 75.3%, mTTA 4.04s), indicating that partial input encourages robust context learning. However, higher masking ra- tios (50-75%) degrade performance, suggest- ing that excessive information removal hinders accurate anticipation.
Table 3: Impact of the masking ratio for spatio- temporal RWKV blocks.
Masking Ratio (%)
mAP (%)
mTTA (s)
No mask
74.0
4.00
30
75.3
4.04
50
71.8
3.93
75
71.0
3.94
Impact of Temporal Shift in Distillation. Table 4 analyzes how different temporal-shift values affect anntiiattanticipatioon performance. moderate shift of one frame yields the best overall trade-off, achieving the highest mAP (75.3%) together with strong mTTA of 4.04 Increasing the shift to two frames results in a slight reduction in mAP (74.7%) while offering only marginal gain in mTTA (4.06 s). A larger three-frame shift further decreases mAp (70.7%) despite small improvement in mTAA (4.13s). s). These results indicate that although larger temporal shifts promote earlier prediction, they provides the most effective balance and aligns with the short predictive horizon characteristic of accident-antin anttipatinniiiiiaiionn tasks.
at the cost of reduced accuracy. Consequently, small temporal shift
8

---

Published as a conference paper at ICLR 2026
Table 4: Impact of the various temporal shift in temporally shifted distillation.
Table 5: Impact of a video-based teacher in tem- porally shifted distillation.
Temporal Shift
mAP (%)
m (S)
Video Distillation
mAp (%)
mTTA (S)
no shift (only spatial)
71.2
3.79
VJEPA2 no shift
65.3
4.12
1
75.3
4.04
VJEPA2 shift (1)
66.0
4.21
2
74.7
4.06
3
70.7
4.13
Impact of Video Teacher. We evaluate video-based teacher using V-JEPA2 Assran et al. (2055) a recent state-of-the-ar model for spatio-tempord representation learning. As shown in Table introducing a temporal shift continues to yield meaningful benefits: the anticipation time increases from 4.12 to 4.21 s, demonstrating that our shifted distillation remains effective even when the teacher already encodes temporal dynamics. However, the overall mAP achieved with V-JEPA2 is notably lower than that obtained with the image-based MobileCLIP teacher. This performance gap expected due to architectural mismatches- V-JEPA2 employs different spatial tokenization and operates much lower spatial resolution in its shallow layers, making its spatial representations difficult for the student to align with. Consequently, the learning process is driven primarily by temporal supervision rather than spatial correspondence. While the temporal shift still improves early anticipation, the lack of compatible high-resolution spatial guidance results in reduced mAP compared to the MobileCLIP-bard -based setting.
Impact of Temporal RWKV and Distilla- tion. Table 6 highlights the complementary roles of Temporal RWKV (T-RWKV) and Tem- porally Shifted Distillation (TSD). Using T- RWKV alone with fine-tuning (FN) provides limited benefit, achieving only 39.4% mAP. In contrast, applying TSD with the purely Spa- tial RWKV (S-RWKV) blocks yields a substan- tial improvement to 55.6% mAp and mTTA of 4s, indicating that shifted supervision supplies meaningful future-aware cues without temporal recurrence. However, effective temporal mod- cling is essential for fully exploiting this signal: T-RWKV with pre-training reaches 67.7% mAP, and the combination of T-RWKV and TSD achieves the best performance 75.3% mAP with mTTA of 4.04 These results demonstrate that neither TSD nor temporal modeling alone is sufficient -strong anticipatory capability arises only when both forms of supervision are jointly employed.
Table 6: Ablation of Temporal RWKV (T-RWKV) and Temporally Shifted Distillation (TSD).
Module
mAP (%) mTTA (s)
T-RWKV + FN
39.4
3.97
S-RWKV + TSD + FN
55.6
4.00
T-RWKV + pre-train + FN
67.7
4.00
T-RWKV + TSD + FN
75.3
4.04
4.4 QUALITATIVE RESULTS
Figure iillustrates successful cases of early accident anticipation. In A), vehicle collides while overtaking into the opposite lane under foggy conditions; the model maintains focus on both the bus and the overtaking car, achieving a 4.55 early warning despite limited visibility. In B), a motorcyclist crashes into a turning vehicle; the model sharply attends to the risky motion around the motorcycle and triggers a 1.7s advance alert. C) presents a nighttime rear-end collision, where attention is concentrated on brake lights and dense traffic, allowing a 4. prediction. In D), a sudden lane switch results in a side collision; the model progressively attends to the intruding vehicle and predicts the accident 2.6s ahead of time.
4.5
SHATE-OF-THE-ART COMPARISON
All previous works rely object detection (Faster-RCNNN and external feature extraction pipelines (VGG-16), introducing significant computational overhead and latency. While MASTTA is the only other recent end-to-end approach, our method achieves higher mAp (+5.1%) and mTTA (+0.08s), while being 3.8x smaller enabling more efficient early accident prediction.
Table provides comprehensive comparison of recent methods the DAD and CCD datasets. On DAD, our model delivers the best balance between early anticipation (mTTA of 4.04s) and high precision (75.33% mAP), outperforming all prior approaches in the trade-off between these metries.

---

Published as a conference paper at ICLR 2026
Table 7: Comparison of the longest mTAA and the corresponding mAP of SOTA models on the DAD and CCD. Best balanced results in terms of model size, accuracy, and early prediction are in bold.
Dataset
Methed
Source
Inference Params. (M)
mAP(%)
m (A(S)
Cha
2016
ACCVI6
18.1
1.31
adaL.EA
Suzuki of
2018
CVPRIS
180
52.3
3.43
GCRNN Bao et
ACMMM20
275
53.7
3.53
Falama
ICPR21
78
49.8
3.76
DRIVEDBR Ban al
20XI
ICCV21
140
62.8
2.78
I-RA Zeng
CVPR21
185
49.1
3.04
DSTA Karm of
2022
TITS22
180
50.
3.66
TIV23
DAA GNN Song el
2124
PR23
183
70.6
1.59
MASTIA MIO QL
2022
TCSVT25
99
70.2
3,96
CCAF-Na
20A5
NEURO25
191
71.8
4.15
Ours
26
75.3
4.04
ACCVI6
GCRNN Bio et a
MIRA
ACMMM20
275
99.5
4.74
99.6
4.52
CCD
MASTIA
TCSVT25
99.9
4.95
CCAF-NETLLN ef 21AS
NEURO22
191
93.9
4.94
4.95
Notably, this achieved with only 26M parameters 7x fewer than DAA-GNN (183M), 8.3x fewer than CCAF-Net (191M), and 3.8x fewer than MASTTA (99M). On CCD, our model achieves highest mAP (99.9% (9.9%) and the longest mTTA (4.95s), matching MASTTA and exceeding other SOTA, again using fraction of the parameters.
Table compares prior methods on the DAD dataset from high mAP perspective. We additionally
Table 8: Comparison of the highest mAP and corresponding mTTA of prior models on DAD. Best balanced results in terms of model size, accuracy, and early prediction are in bold.
incorporate several purely Transformer based ap-
proaches Fan et al. (202T); [iu al. (2022); Li et (202)) as well as CNN-based models Carreira & Zisserman (017); Tran et al. (2018); Lin et al. (2019), all of which are widely used in video under- standing. Notably, these architectures underperform on the accident anticipation task, with none exceed- ing 70% mAP. Although CCAF-Net and MASTTA slightly outperform in mAP (81.3% and 80.8%, re- spectively), our model attains a competitive 79.61 % mAP with the smallest parameter count (26M) and favorable mTTA of 3.41s outprrforming MASTTA by 0.09s and trailing CCAF-Net by only 0.34s. These results underscore our model's strong effi- ciency-accuracy trade-off, making it well-suited for real-time accident anticipation.
Params, (M) mAP(S)
mTTA(5)
13D R(2+1)D TSM
21
68.0
2.99
31
19.7
3.57
43
53.0
3.39
GCRNN
275
72.2
1.33
I-RAT
185
51.4
3.01
MVTTv2
51
64.4
VideoSwin
88
65.4
UniFormer V2
115
65.2
DAA-GNN
183
75.2
1.47
MASTTA
99
80.8
3.32
CCAF-Net
191
81.3
3.75
Ours
26
79.6
3.41
CONCLUSIONS
This work presents lightweight spatio-temporal framework for early accident anticipation, de- signed for real-time operation in resource-constrained environments. By combining temporally shifted distillation strategy with a hybrid architecture that integrates convolutional spatial encoding and recurrent temporal modeling, our approach enables efficient long-range reasoning without the need for video-trained teachers. The framework achieves strong accuracy on limited data with low latency and memory usage, making it practical for deployment in real-world autonomous driving systems and adaptable to broader traffic safety applications.
ACKNOWLEDGMENTS
This work was supported by the National Science and Technology Council, R.O.C., under contract 133-2221-E-0111-32222
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
All architectural details and parameter settings are described in the Implementation Details section, with further configuration and training specifics provided in the supplementary material in Appendix facilitate reproducibility. The project page and code available at https::/ https::/pptpatera. - Anticiatiiionnn
github
The manuscript's 's writing has been polished using ChatGPT.
REFERENCES
Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Am- mar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025.
Wentao Bao, Qi Yu, and Yu Kong. Uncertainty- based Traffic Accident Anticipation with Spatio- Temporal Relational Learning. in Proceedings of the 28th ACM International Conference Multimedia, pp. 2682-2690, 2020.
Wentao Bao, Qi Yu, and Yu Kong. DRIVE: Deep Reinforced Accident Anticipation with Visual Explanation. in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7599-7608, 2021. doi: 10. 10.109/CCCV499222221..7777
Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp- 6299-6308, 2017.
Fu-Hsiang Chan, Yu-Tine Yu-Ting Chen, Yu Xiang, and Min Sun. Anticipating Accidents in Dashcam Videos. in Asian Conference on Computer Vision, pp. 136-153, 2016.
Yuchen Duan, Weiyun Wang, Zhe Chen, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Hongsheng Li, Jifeng Dai, and Wenhai Wang. Vision-rwkv: Efficient and scalable visual perception with rwke-like architectures. arXiv preprint arXiv:2403.02.000 2024.
Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer. Multiscale Vision Transformers. in Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition (CVPR), pp. 6824-68355 2021.
Jianwu Fang, Jiahuan Qiao, Jie Bai, Hongkai Yu, and Jianru Xue. Traffic Accident Detection via Self-Supervised Consistency Learning in Driving Scenarios. IEEE Transactions on Intelligent Transportation Systems, 23(7):9601-661 2022. doi: 1TT./T/TT.002..11111.1
Jianwu Fang, Lei-lei Li, Junfei Zhou, Junbin Xiao, Hongkai Yu, Chen Lv. Jianru Xue, and Tat- Seng Chua. Abductive ego-view accident video understanding for safe driving perception. in Proceedings of the IEEE/VV Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22030-220444 2024.
Mishal Fatima, Muhammad Umar Karim Khan, and Chong-Min Kyung. Global Feature Aggre- gation for Accident Anticipation. in Proceedings of the International Conference on Pattern Recognition (ICPR), pp. 2809-2816, 2021. doi: 10.1 10.110//CP448800.20222 2021..4121.999..
Xiaohu Huang, Hao Zhou, Kun Yao, and Kai Han. FROSTER: Frozen CLIP is Strong Teacher for Open- Vocabulacy Action Recognition. in Proceedingy of the International Conference on Learning Representations, pp. 1-16, 2024.
Ashesh Jain, Avi Singh, Hema S Koppula, Shane Soh, and Ashutosh Saxena. Recurrent Neural Networks for Driver Activity Anticipation via Sensory-fusion Architecture. in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 3118-3125, 2016. doi: 10. N0..1199/CRAA.216..7477777
Muhammad Monjurul Karim, Yu Li, Ruwen Qin, and Zhaozheng Yin. A Dynamic Opatial-Temporal Attention Network for Early Anticipation Traffic Accidents. IEEE Transactions on Intelligent Transportation Systems, (11)99999-90000 2022. doi: 10.1
11

---

Published as a conference paper at ICLR 2026
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers mnns: Fast autoregressive transformers with linear attention. In International conference on ma- chine learning, pp. 5156-5165, PMLR, 2020.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Limin Wang. and Yu Qiao. UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding, in Proceedings the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1632-1643, 2023.
Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE/CVF international conference computer vision, pp. 7083-7093, 2019.
Wei Liu, Yafei Li, Tao Zhang, Yixiang Gao, Longsheng Wei, and Jun Chen. CCAF-Net: Cas-
cade Complementarity- Aware Fusion Network for traffic accident prediction in dashcam videos.
Neuroccoputting 624: 129285, 2025. ISSN 0925-2312. doi: https///oii.rr//// 1ttp://...011///..ccceoo
2024.129285. URL https: http:://www..cc...... wtw.sc//eeeedaeeec..ccc////eeeee
$092523122022563
Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu, Video Swin Trans- the former. in Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR), pp. 5202-3211 2022.
Sachin Mehta and Mohammad Rastegari. Mobilevit: light-weight, general-purpose, and mobile- friendly vision transformer. arXiv preprint 10.02,7, 2021.
Daniel C. Moura, Shizhan Zhu, and Orly Zvitia. Nexar dashcam collision prediction dataset and challenge, 2025. URL htttss://axxiii http://axxi...or/aa//////// 03848
Patrik Patera, Yie- Tir-Tarng Chen, and Wen-Hsien Fang. A Multi-modal Architecture with Spatio- Temporal-Ter Text Adaptation for Video-based Traffic Accident Anticipation. IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-15, 2025. doi: 10. 10.1109/CCSV.222555552999
Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, the transformer era. arXiv preprint arXiv:2305.13048 2023.
Rwkv: Reinventing mns for
Debaditya Roy, Tetsuhiro Ishizaka, C. Krishna Mohan, and Atsushi Fukuda. Detection Collision- Prone Vehicle Behavior at Intersections Using Siamese Interaction LSTM. IEEE Transactions Intelligent Transportation Systems, 23(4): 2022. doi: 10. 2029T.TS.2000.3330
Dinesh Singh and Chalavadi Krishna Mohan. Deep Spatio-Temporal Representation for Detection of Road Accidents Using Stacked Autoencoder. IEEE Transactions on Intelligent Transportation Systems, 20(3):879-888) 2019. doi: 10.1 10.1109/TTTS,20 288.2T1T,20018.2333333...
Wenfeng Song, Shuai Li, Tao Chang, Ke Xie, Aimin Hao, and Hong Qin. Dynamic Attention Augmented Graph Network for Video Accident Anticipation. Pottern Recognition, 147:1 10071, 2024. ISSN 0031-3203.
Tomoyuki Suzuki, Hirokatsu Kataoka, Yoshimitsu Aoki, and Yutaka Satoh. Anticipating Traffic Accidents with Adaptive Loss and Large-Scale Incident DB. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3521-3529, 2018. doi:
10.1100//VPR.2211.0003
Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri. A closer look at spatiotemporal convolutions for action recognition. Proceedings of the IEEE conference Computer Vision and Puttern Recognition, pp. 6450-6459, 2018.
Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vemulapalli, and Oncel Tuzel. Mobileclip: Fast image-text models through multi-modal reinforeed training. in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 15963-15974, 2024.
12

---

Published as a conference paper at ICLR 2026
Tianhang Wang, Kai Chen, Guang Chen, Bin Li, Zhijun Li. Zhengfa Liu, and Changjun Jiang. Gsc: A graph and spatio-temporal continuity based framework for accident anticipation. IEEE Transactions Intelligent Vehicles, 9(1):2249-2D) 2261, 2024. doi: 10. 00.1199/TI.2023.2557769
Hanbo Wu, Xin Ma, and Yibin Li. Spatiotemporal Multimodal Learning With 3D CNNs for Video Action Recognition. IEEF Transactions Circuits and Systems for Video Technology, 32(3):
1250-1261, 2022. doi: 1..11099TCSVTT.221..377777
Peng Wu, Xuerong Zhou, Guansong Pang. Lingru Zhou, Qingsen Yan. Peng Wang, and YANNING ZHANG. VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection. in Proceedings of the AAAI Conference on Artificial Intelligence, 38(6):6074 6082,
2024. doi: 10. 16.9//aaa... 00009//aii.3316.284448888 v8866.28423.
Jiewen Yang, Xingho Dong, Liujun Liu, Chao Zhang, Jiajun Shen, and Dahai Yu. Recurring the transformer for video action recognition. in Proceedings of the IEEE/CVF Conference Com- puter Vision and Pattern Recognition (CVPR), pp. 14063-14073, 2022.
Hongyang Yu, Xinfeng Zhang, Yaowei Wang, Qingming Huang, and Baocai Yin. Multiple-Level Accident Detection. IEEE Transactions on Circuits and Sys-
Distillation for Video tems for Video Technology, 34(6):4445 4457, 2024. doi: 10. 10.1109/TCSVT2222333333388
Kuo-Har Zeng, Shih-Han Chou, Fu-Hsiang Chan, Juan Carlos Niebles, and Min Sun. Agent-Centre Centric Risk Assessment: Accident Anticipation and Risky Region Localization. in Proceedingy of the IEEE/CVF Conference on Compuler Vision and Putlern Recognition (CVPR), pp. 1330-1338,
2017. doi: 10.1109/CVPR:2017..2
Shuangfei Zhai, Walter Talbott, Nitish Srivastava, Chen Huang, Hanlin Goh, Ruixiang Zhang, and Josh Susskind. An Attention Free Transformer. arXiv preprint arXiv:2105. 14103, 2021.
Zhili Zhou, Xiaohua Dong, Zhetao Li, Keping Yu, Chun Ding, and Yimin Yang. Temporal Feature Encoding for Traffic Accident Detection in VANET Environment. IEEE Transactions on Intelligent Transportation Systems, 23(10):19772-1788,, 2022. doi: 3147826.
A SUMMARY OF SUPPLEMENTARY CONTRIBUTIONS
This supplementary material provides additional details supporting our main submission. We de- scribe the proposed spatio-temporal masking strategy along with ablation results, present the com- plete architectural configurations of both the student and teacher models, and outline the train- ing protocols and deployment setup. We also include qualitative examples that highlight model limitations, clarify the dataset composition, and explain the use of coarse- coarse-grained text labels dur- ing fine-tuning. The project page and code available at TSD-Acc dent-Anti catiion
gith.u io/
B ANALYSIS OF CCD PERFORMANCE
To further address concerns regarding potential overfitting on the CCD dataset, we provide addi- tional analyses and metrics beyond the mAP and mTTA reported in the main paper. CCD is a well-known saturated benchmark where numerous prior works (c.g.. DSA, GCRNN, MASTTA) routinely achieve 99.0-99.994 mAP under the standard protocol, and our results follow this estab- lished trend. To verify robustness, T9 ROC-AUC, which reaches a similarly high value of 85.0%, indicating strong separability between accident and non-accident trajectories. We also evaluate smaller student variant (XS) with only 13 million parameters; despite its lower capacity, it preserves the same mAp of 99. 99.9% with only minor decrease in mTTA (4.92 s), suggesting that CCD performance does not depend on fragile model capacity. Furthermore, we include TTAs0% the time-to-acee -cccideet 80% recall which remains consistently high at 4.84 s. Collectively, these results confirm that our model's CCD performance stable and reflects the dataset's ' inherent case rather than overfitting.
13

---

Published as a conference paper at ICLR 2026
Table 9: Comparison of smaller network variant on CCD with additional metries.
Model
AUC. (%)
mAP (%)
m mTTA (8)
T'As0%
Params (M)
Model smaller (XS)
85.0
99.9
4.92
4.84
13
Model small (S)
86.9
99.9
4.95
4.86
26
C ANALYSIS OF SHIFTED-TEACHER ALIGNMENT
We further assess the effectiveness of Temporally Shifted Distillation (TSD) by examining the dif- ference in cosine similarity between two pairs signals: the student's features X aligned with the teacher's same-time features X/" and the student's s features X² aligned with the teacher's future features X To further quantify this effect, we introduce the Temporal Alignment Ratio (TAR),
cos cos(X;, X CO8 cO8 co8(X; x +
TAR;
(16)
which measures how strongly the student aligns with the teacher's future state relative to its present state. Values greater than indicate that the student preferentially aligns with future representa- tions, reflecting true anticipatory capability rather than static feature matching. Empirically, TAR remains consistently above and increases over time, further confirming that TSD induces mean- ingful temporal forecasting behavior. As shown in Fig. 5 decreasing blue TAR curve (smooth) would indicate that TSD provides no benefit, suggesting that the student relies primarily on spatial correlations rather than learning to anticipate future representations. In contrast, the consistently increasing trend demonstrates that the student becomes progressively better aligned with the future teacher embeddings.
1.4
Temporal Alignment
1.2
features alignment
1.0
Features same dominates
0.8
Smoothed Curve
0.4
Epoch
Figure 5: Temporal alignment comparison showing that the student aligns more strongly with the teacher's, future features. TAR (blue curve) > further confirms TSD induces anticipatory learning.
D ARCHITECTURAL DETAILS
D.1 STUDENT VISION ENCODER
Our Student Vision Encoder adopts a lightweight yet expressive RepMixer (illustrated in Figure 6 spatial encoding on the MobileCI.IP Vasu et al. (2024) with our lightweight window-based spatio- temporal module adapted from the RWKV Peng al. (2023), optimized for efficient spatio-temporal modeling in accident anticipation. As detailed in Table I0, the encoder processes input video of shape T x x W through four hierarchical stages, each progressively reducing spatial resolution while expanding feature dimensionality. The stem consists of two convolutional layers standard 3 x convolution followed by a MobileOne-stye 3 3 convolution with stride 2 - yielldnng 48-dim features.
Stage begins with a patch embedding layer using a x MobileOne-style convolution with stride 2 generate tokens at H! W. resolution, followed by 2 RepMixer blocks with 96-dim features 7
14

---

Published as a conference paper at ICLR 2026
Table 10: Student Vision Encoder. Architectural configuration of the proposed lightweight spatio- temporal model variant.
Stage
#Tokens
Layer Spec.
Description
Dim.
3 x 3, stride 2
Stem
T'x H W
Conv
48
x 3 MobileOne Style, stride 2
Patch Embed.
x MobileOne Style, stride 2
H W T x 4
RepMixer
96
4
Blocks
2x
Patch Embed.
x MobileOne Style, stride 2
T x
RepMixer
128
Blocks
6x
Patch Embed.
MobileOne Style, stride
T
W
RepMixer
256
Blocks
10x
Patch Embed.
7 x 7 MobileOne Style, stride 2
8 T
W
Spatio-temporal RWKV
512
32
32
Blocks
6x
Parameters (M)
26
for local spatial modeling. Stage 2 continues this pattern, reducing the resolution to W. and H! applies 6 RepMixer blocks with 128 channels. Stage 3 further downsamples 16 x and employs 10 RepMixer blocks with 256-dim features to capture mid-level semantic representations.
Stage 4, spatial resolution reduced to H W. 32
32 resulting in 7 x 7 spatial tokens with input of 224 x 224. Here, 3D condi- tional positional encoding is applied, and tem- poral modeling is introduced using 6 Spatio- temporal RWKV blocks with 512-dim tokens. These RWKV blocks allow efficient linear- complexity modeling across frames, enabling the encoder to capture long-range temporal de- pendencies without sacrificing latency. The complete Vision Encoder comprises only 26 million parameters.
(okip caneecec cornectio))
Batch Normalization
3x3 DW Convolution
ConvFFN
D.2 STUDENT TEXT ENCODER
The text encoder adopts the Text- RepMixer chitecture from MobileCLIP, maintaining ar- chitectural consistency with the vision encoder while being specifically tailored for language processing. It employs hybrid design that combines ID convolutions with self-attention layers. Unlike purely convolutional approaches - which were found to underperform -Text-RepMixer ef- fectively balances local and global context modeling by integrating convolutional token mixing with attention- based reasoning. For efficient inference, skip connections and normalization layers are reparameterized, and the feed-forward layers are enhanced with depthwise 1D convolutions, form- ing ConvFFN blocks that can be fused for optimized execution. The complete model, including both the Vision and Text Encoders, contains a total of 69 million parameters.
Figure 6: RepMixer Block from MobileCLIP.
D.3 TEACHER VISION ENCODER
The Teacher Vision Encoder shares the overall architecture the Student, detailed in Fig- ure [I] including the hierarchical stage structure and patch embedding. However, it is deeper, with 12, 24, 4] blocks and larger embedding dimensions of [80, 160, 320, 640] across stages to
15

---

Published as a conference paper at ICLR 2026
Table 11: Teacher Vision Encoder. Architectural configuration of the large image-based model variant serving as the teacher.
Stage
#Tokens
Layer Spec.
Description
Dim.
3 x 3, stride 2
Stem
T'x H W
Conv
48
x 3 MobileOne Style, stride 2
Patch Embed.
x MobileOne Style, stride 2
H T x 4
W
RepMixer
80
Blocks
4x
Patch Embed.
x MobileOne Style, stride 2
T x
RepMixer
160
Blocks
12x
Patch Embed.
x MobileOne Style, stride
T
W
RepMixer
320
Blocks
24x
Patch Embed.
7 x MobileOne Style, stride 2
T
8 32
W 32
Multi Head Self-Attention
640
Blocks
4x
Parameters (M)
36
4. While the Student introduces lightweight spatio-temporal RWKV blocks in Stage 4, the Teacher retains conditional positional encoding and standard spatial multi-head self-attention (MHSA) for purely spatial modeling. The Teacher is initialized with the official pre-trained weights from Mobile- CLIP Vasu et al. (2024) and kept entirely frozen during pre-training, providing stable, high-capacity supervision for the student's spatio-temporal adaptation via our proposed temporally shifted distil- lation.
Hyper-parameter
Pre-training
Fine-tuning
Input resolution
224x224
224x224
Input #frames
32
Frame sample interval
8
3
Random Resize Crop
[0.1, 1.0]
[0.1, 1.0]
Random Horizontal Flip
0.5
0.5
Color Jitter
V
A1. X2, As Aa
10.3, 0.4, 0.5, 1.0]
Mask ratio
30%
Train epochs
70
50
Warmup epochs
5
0
Batch size
70
12
Optimizer
AdamW
AdamW
Peak learning rate
8e-5
8e-6
LR decay schedule
cosine
cosine
Weight decay rate
0.01
0.01
Gradient clipping
5
Mixed precision
BF16
BF16
Table 12: Hyperparameters for Pre-training and Training.
E EXPERIMENTAL SETUPS
We conducted all experiments using a single NVIDIA RTX 4090 GPU with GBGB memory for both pre-training and fine-tun fine-tuning, while the final model deployed NVIDIA Jetson Orin Nano with GB shared memory for real-time inference. The pre-training and fine-tuning phase have utilized the dataset described in the Datasets section of this supplementary materials. Table T2
16

---

Published as a conference paper at ICLR 2026
summarizes the hyper- parameerr configurations used for both pre-training and fine-tuning. In both settings, the input resolution is fixed at 224 x 224. During pre-training, the model processes clips of 8 frames sampled every frames, while fine-tuning uses 32-frame clips sampled every 3 frames to improve temporal precision for accident anticipation. For input data augmentation, both phases employ random resize cropping (scale range 0.1 1.0), horizontal flipping with probability of 0.5, and color jitter is applied to promote visual robustness.
The pre-training phase additionally incorporates a 30% masking ratio applied the input frames and employs a multi-task loss with weighting coefficients A1, X2, λa, A = [0.3, 0.4, 0.5, 1.0]. whereas fine-tuning relies solely on the classification loss. Both phases are trained for 50 epochs, with warm--up up epochs. Due to memory constraints, the batch size is set to 70 during pre-training and 12 during fine-tuning. Optimization performed using Adam with a peak learning rate of 8 x 10-5 and cosine learning rate decay schedule. A weight decay rate of 0.01 and gradient clipping threshold of are consistently applied throughout both phases. Both pre-training and training were performed under PyTorch mixed precision BF16. 6. Training across multiple random seeds, mAP remains stable within +0.3% and mTTA within =.008.
Fine-tuning uses the same objective function Laceideu as defined in Equation (14) of the main manuscript for text-visual features, along with a standard cross-entropy loss for the visual classi- fier.
F EVALUATION METRICS
Average Precision (AP). Each method produces confidence score per frame representing the like- lihood of a future accident. A prediction is counted as True Positive (TP) if the score exceeds threshold q before the accident onset; otherwise, is treated as False Negative (FN). Any pre- diction in non-accident video is considered a False Positive (FP). By varying q. precision-recall curve obtained, and AP is computed as the area under this curve. Precision (TP/(TP- + FP)) and recall (TP/(TP + FN)) are evaluated at each threshold, and the mean Average Precision (mAP) is calculated as their average across all evaluated thresholds.
Time to Accident (TTA). TTA is defined as T--T, where T is the accident start frame and r is the first frame where the confidence exceeds The mean TTA (mTTA) averages TTA over true positives across thresholds.
Oura
MASTTA
GORAN
G PERFORMANCE
Figure illustrates the trade-off between model size and early accident anticipation ability across state-of-the-ar methods. The x-axis represents the number of inference parameters (in millions), and the y-axis denotes the mean Time-to-Acciden (mTTA) in seconds. Each point corresponds to a competing approach. Our method, highlighted in green and marked with a red arrow, achieves the highest m'TA (4.04s) while using only 26M parameters-the smallest among all models. This result demon- strates that our approach delivers superior early anticipation despite being significantly more lightweight than existing baselines, offering clear advantage for efficient, real-world deployment.
JAA-CNN
OGA
Inferenc Paramss (M)
Figure 7: Inference Parameters vs. Early Pre- diction (mTTA). Our method (green) achieves real-time performance with the smallest model size, surpassing larger baselines in terms of mTTA. This demonstrates favorable trade-off between efficiency and temporal anticipation, ideal for resource-limited deployment.
H IMPLEMENTATION DETAILS
We implement model and all training/testing pipelines using PyTorch 2.5.0. To enable efficient spatio-temporal modeling, our RWKV-based module is implemented with custom CUDA kernel that supports parallel computation over video spatio-temporal input sequence, compiled with Ninja
17

---

Published as a conference paper at ICLR 2026
Accident description: "Vehicles run red light"
SUP GRASHES TIME
Accident description: 'The truck is out of control"
EBR
2016-11228 16:00
Accident description: "Pedestrian moves the motorway"
MIEAPTT 12
27amth 201 B01801213 1011310
Figure 8: Example of accident scenes from MM-AU with corresponding textual descriptions used during pre-training.
1.10.2.1 library. This design significantly accelerates both training and inference, particularly when handling long video sequences. To prepare window-based inputs for the RWKV block, we utilize PyTorch's built-in Unlold d operation window-hased extract spatio-temporal patches, and nn. Fn.Fold to reconstruct the processed output back to the original input shape. This approach en- sures computational efficiency while maintaining compatibility with the rest of the architecture. The similar unfolding window-based pattern can be found in another methods such as SWIN- Transformer Liu al. 2022) and Mobile VIT Mehta & Rastegari (20TT).
DATASETS
The training of framework involves both pre-training and fine-tuning phases. We pre-train the model on the recently released MM-AU Fang et al. 2024) dataset, which includes only accident videos paired with textual descriptions detailing 112 distinct accident scenarios. To compensate for the absence of negative samples, we incorporate non-accident videos from Nexar Moura et al. (2025). DAD(Cha et al. (2166), and CCD et al. al. (2020). This setup offers key beneñts: (1) MM- textual descriptions provide strong inductive bias, helping the model learn causal relationships beyond visual saliency; (2) pre-training on separate dataset supports domain generalization and reduces overfitting; and (3) exposure to varied accident types, viewpoints, and contexts enhances robustness. MM-AU is not used for evaluation due to its lack of negative samples and standard- ized test splits with evaluation protocol. Instead, we leverage it solely for pre-training to exploit its rich text-video annotations. Figure [ shows example accident scenes from MM-AU with their corresponding textual descriptions used during pre-training.
18

---

Published as a conference paper at ICLR 2026
During fine-tuning and inference, only coarse- -aanine text labels "Normal traffic situa- tion" and "A traffic accident" due to the absence of detailed textual annotations in the downstream datasets. These labels provide semantic conditioning, enabling class-aware alignment even without fine-grained textual supervision. Fine-tuning performed on two widely used benchmark datasets for traffic accident anticipation DAD and CCD ensure fair comparison with prior state-of- the-art methods.
Traffic accident
Original Frens
A]
Adbendonn Maga
G
Fountion MOS
Figure 9: Qualitative analysis of model limitations. Examples of false negatives (A-B) and false positives (C-D) are shown. Red regions in the attention maps indicate high-risk areas, while the bottom row displays frame-wise accident probability curves.
J LIMITATIONS
Figure presents qualitative results across four diverse driving scenarios, illustrating both false negatives (A-B) and false positives (C-D) in traffic accident anticipation. Subfigures A) and B) depict cases where the model fails to anticipate accidents. In A), the accident occurs in the far background with only small visual footprint, making it difficult for the model detect early warning cues. Despite partial attention over relevant regions, the subtlety and scale of the event lead missed prediction. In B), the camera is mounted on motorbike helmet, causing severe camera shake and unstable motion that hinder consistent spatio-temporal reasoning. Although the attention maps correctly highlight the road and nearby vehicles, the predicted accident probability remains low throughout, likely due the accident being heavily obscured by another vehicle in front.
In contrast, C) and D) represent false positives, where the model erroneously predicts an accident in videos where no incident occurs. Both sequences are captured with helmet-mounted cameras, introducing natural shakiness. C) features dense daytime traffic, with a tightly packed group of motorbikes driving in close proximity. The attention maps strongly focus surrounding vehicles, causing the model to interpret the scene as hazardous, despite the absence an accident. D), recorded at night under low visibility, involves high-speed driving with bright headlights and road
19

---

Published as a conference paper at ICLR 2026
reflections. These challenging lighting conditions, combined with motion from the selmet-mounted camera, result in persistently high accident probabilities, even though no collision occurs.
Overall, these visualizations underscore key challenges in accident anticipation, including motion instability, small-scale accident cues, traffic density, and adverse lighting. They highlight the impor- tance of incorporating spatio-temporal consistency, scale awareness, and robustness to visual noise into model design.
20


Reviews:
[{"review_id": "oDiT9zxpU9", "reviewer": "Reviewer_rGQA", "summary": "The paper proposes a lightweight framework for accident anticipation using temporally shifted distillation (TSD).\nA student model learns to predict future cues by aligning its current features with a teacher’s representation from later frames.\nThe design combines a RepMixer backbone with an RWKV-based temporal block for efficiency.\nExperiments on DAD and CCD show improved early anticipation and real-time performance on edge devices.", "questions": "Have the authors tried using a video-based teacher that already has temporal understanding? It would be interesting to see whether the proposed shift is still necessary in that setting.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tThe idea of learning temporal prediction through shifted distillation is original and intuitively appealing.\n\n2.\tThe method achieves a good balance between accuracy and efficiency, which makes it suitable for real-world deployment.\n\n3.\tThe paper is clearly written, and the model architecture and ablation design are well presented.", "weaknesses": "1.\tThe analysis of the temporally shifted distillation is limited.\nTable 2 only reports overall performance, so it is difficult to tell whether the model actually learns to anticipate future events or only captures static correlations.\nAdditional experiments showing temporal behavior, such as feature alignment across time, would strengthen the claim.\n\n2.\tThe mechanism itself is not deeply analyzed.\nIt would be useful to evaluate how different time shifts affect learning or whether the model improves its understanding of future frames during training.\n\n3.\tSince the teacher model processes only single frames, it is unclear how much of the predictive ability comes from the distillation process compared with the student’s temporal module.\nMore explanation or comparative evidence would clarify this point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2oLXJdIwOM", "reviewer": "Reviewer_7qEU", "summary": "This paper proposes a lightweight dash-cam accident-anticipation model that distills future-aware temporal cues from a frozen MobileCLIP teacher using a windowed RWKV student and masked memory. It addresses the problem of anticipating accidents several seconds earlier than prior work while staying real time on edge devices with only RGB inputs. Experiments show that the porposed method obtains 75.3% mAP and 4.04s mTTA on DAD with 26M params, beating heavier SOTA (in Table 4).", "questions": "- Please respond to the weaknesses above.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The reviewer found the idea of distilling from an image-only MobileCLIP into a temporal RWKV student for accident anticipation to be interesting.\n- Interestingly, temporal-only distillation outperforms spatial-only (74.1% vs 71.2% mAP in Table 2), showing future supervision actually drives anticipation.\n- Experiments are comprehensive: Table 1 (RWKV depth), Table 2 (distillation parts), Table 3 (mask ratio), Figure 4 (qualitative attention) and writing is mostly clear and structured.", "weaknesses": "- The temporally shifted distillation still does not show a direct comparison to distilling from a video teacher., so it is unclear how much is due to the shift itself.\n- Experiments miss an ablation on modern, purely temporal CNN/Transformer baselines that work on DAD/CCD datasets even though Table 4 already cites many frame-level systems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OGrkFQA5kD", "reviewer": "Reviewer_bqJU", "summary": "The paper introduces a lightweight spatio-temporal framework for real-time traffic accident anticipation, particularly suited for deployment on resource-constrained edge devices. The key contributions are: (1) a novel temporally shifted distillation strategy that enables a student model to learn predictive temporal dynamics from a frozen image-based teacher, eliminating the need for a video pre-trained teacher and making it ideal for small datasets and low-resource settings; (2) a hybrid architecture that combines RepMixer spatial encoding with an RWKV-inspired recurrent module for efficient long-range temporal reasoning, achieving low computational complexity; (3) a masked memory strategy that simulates occlusions and partial visibility to improve robustness under real-world conditions.", "questions": "See the weakness.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper innovatively redefines temporal learning through temporally shifted distillation, removing the need for video-pretrained teachers and enabling real-time performance under constraints.\n2. Comprehensive experiments and ablation studies show improvements in mAP and mTTA with fewer parameters.\n3. Clear structure, well-explained components, and visual aids make the paper accessible, though some concepts remain high-level.", "weaknesses": "1. The stitching method of the spatio-temporal scanning pattern seems somewhat arbitrary. There might be continuous information between feature sub-blocks that is not utilized. Is this an existing design approach?\n\n2. In lines 231-236, I don't understand why only KV is involved in the calculation and not the Q parameter.\n\n3. In lines 241-260, can the masked part be considered multi-stage, requiring multiple training iterations? Does this introduce higher training costs compared to other models? Is this reflected in the paper?\n\n4. Lines 264-269: Are there applications on larger-scale datasets? I would like to know how this lightweight model performs under larger-scale conditions. Even if the performance isn't strong, can you provide the model's applicability boundary?\n\n5. Table 4: The performance on the CCD dataset seems close to overfitting. Has the lightweight version been applied to other methods? Also, are there additional evaluation metrics? Are the two metrics used sufficient?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)