Title: EPA: Boosting Event-based Video Frame Interpolation with Perceptually Aligned Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Event-Based Vision

Deep Learning

Computer Vision

Video Frame Interpolation
Abstract: Event cameras, with their capacity to provide high temporal resolution information between frames, are increasingly utilized for video frame interpolation (VFI) in challenging scenarios characterized by high-speed motion and significant occlusion. However, prevalent issues of blur and distortion within the keyframes and ground truth data used for training and inference in these demanding conditions are frequently overlooked. This oversight impedes the perceptual realism and multi-scene generalization capabilities of existing event-based VFI (E-VFI) methods when generating interpolated frames. Motivated by the observation that semantic-perceptual discrepancies between degraded and pristine images are considerably smaller than their image-level differences, we introduce EPA. This novel E-VFI framework diverges from approaches reliant on direct image-level supervision by constructing multilevel, degradation-insensitive semantic perceptual supervisory signals to enhance the perceptual realism and multi-scene generalization of the model's predictions. Specifically, EPA operates in two phases: it first employs a DINO-based perceptual extractor, a customized style adapter, and a reconstruction generator to derive multi-layered, degradation-insensitive semantic-perceptual features ($\mathcal{S}$). Second, a novel Bidirectional Event-Guided Alignment (BEGA) module utilizes deformable convolutions to align perceptual features from keyframes to ground truth with inter-frame temporal guidance extracted from event signals. By decoupling the learning process from direct image-level supervision, EPA enhances model robustness against degraded keyframes and unreliable ground truth information. Extensive experiments demonstrate that this approach yields interpolated frames more consistent with human perceptual preferences. *The code will be released upon acceptance.*


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses video frame interpolation (VFI) using event signals, highlighting that existing event-based methods often overlook blur and distortion in keyframes and ground-truth data, leading to suboptimal performance. To mitigate this, the authors propose EPA, a novel framework that leverages multilevel, degradation-insensitive semantic-perceptual supervision using a DINO-pretrained vision backbone. The approach aims to enhance perceptual realism and experiments show that EPA achieves state-of-the-art results in perceptual metrics such as LPIPS.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper is well-organized and clearly written.\n\n- The model design is technically sound.\n\n- Using pretrained vision features for perceptual-level supervision is a reasonable.\n\nWeaknesses:\n\n- No analysis of computational cost is provided (e.g., model size, speed, training overhead).\n\n- No video results are shown, which limits the evaluation of temporal consistency—a key factor in VFI tasks.\n\n- Recent methods capable of handling large motion and ambiguous supervision, such as classic models with novel indexing [1] and diffusion-based approaches [2], are not compared. The examples shown seem to involve relatively moderate motion; a qualitative comparison on more challenging sequences would help justify the use of event signals.\n\n- The related work section lacks coverage of prior efforts on unsupervised event-based VFI [3] for degraded inputs, and those further exploiting the hidden temporal information in degraded inputs [4–6].\n\nMinor Issues:\n\nThere are typos such as line 111: “losse” → “losses”\n\nReferences:\n\n[1] Clearer frames, anytime: Resolving velocity ambiguity in video frame interpolation, ECCV 2024\n\n[2] Framer: Interactive frame interpolation, ICLR 2025\n\n[3] Unsupervised Event-Based Video Reconstruction, WACV 2024\n\n[4] Event-Based Frame Interpolation with Ad-hoc Deblurring, CVPR 2023\n\n[5] Unifying Motion Deblurring and Frame Interpolation with Events, CVPR 2022\n\n[6] Event-Guided Frame Interpolation and Dynamic Range Expansion of Single Rolling Shutter Image, ACM-MM 2023\n\nQUESTIONS:\nWhy is there no computational cost analysis?\n\nHow does the model perform in terms of video-level consistency?\n\nHow does it compare with recent large-motion-capable VFI methods [1,2]?\n\n(Refer to the “Weaknesses” section)\n\nLIMITATIONS:\nA clear analysis of computational costs and more comprehensive video-based comparisons would strengthen the paper’s claims.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an event-based method for arbitrary-time video frame interpolation between two degraded images. Compared with existing flow-based or multi-task (restoration + interpolation) methods, the proposed framework claims to avoid such trade-offs without introducing additional complexity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\nThe paper presents a new pipeline to learn robust features from vision foundation models and guide reconstruction under image degradation.\n\nThe proposed method is extensively evaluated on both synthetic and real datasets, demonstrating superior performance over existing approaches.\n\nWeaknesses\n\nThe proposed framework lacks a natural connection between motion-induced image degradation (e.g., blur) and event generation, and the two core modules are relatively loosely coupled.\n\nThe DINO-based feature extraction module (Sec 3.1) is primarily used to learn degradation-robust features from images and is unrelated to events. Similar strategies have been adopted in prior work. The authors should clarify the unique contribution of this component in their pipeline.\n\nUnlike prior event-based optical flow methods where events contribute to both motion estimation and image synthesis, the proposed approach uses events only for alignment, without directly enhancing semantic features. The rationale for this design should be discussed.\n\nThe ablation study is limited to toggling modules on or off. A deeper analysis is needed, such as replacing DINO with other backbone models or combining other backbones with the proposed supervision to verify the generality of the approach.\n\nQUESTIONS:\nHow is the off-the-shelf reconstruction generator [41] aligned with features extracted by M_f (DINO), especially to support the claim of compatible to multi-level feature input for restoring the image?\n\nInconsistently with HS-ERGB, the proposed method achieves better perceptual metrics (e.g., LPIPS) but worse traditional ones (e.g., PSNR) on BS-ERGB and EventAid datasets. An in-depth analysis for this phenomenon is needed, including more significant visual comparison analyses.\n\nThe paper should clarify differences between training and inference stages and report the inference cost.\n\nDoes the claim of L82 apply to all existing methods? For example, does EVDI [46] have this problem?\n\nminor: \n\n111 losse\n\nsupp 46 Limition\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel event-based video frame interpolation framework, designed to address image distortions in complex scenarios, where existing methods often fail. By introducing multi-level, degradation-insensitive, semantic-aware supervision signals, the proposed method enhances the perceptual realism and cross-scene generalization of the interpolated frames. Experimental results demonstrate that the proposed method excels in both perceptual quality and generalization capability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n(1) The paper proposes an effective feature-level supervision strategy that addresses the limitations of existing methods when dealing with degraded images.\n\n(2) The experimental results not only show excellent performance on traditional metrics (e.g., PSNR, SSIM) but also significantly outperform prior works on perceptual quality metrics (e.g., LPIPS, DISTS).\n\nWeaknesses:\n\n(1) The proposed method employs a generator-based architecture to produce the final image, a technique inherently known to boost the perceptual quality of generated outputs. Consequently, it is unclear whether the final improvement in perceptual quality stems more from the generator architecture itself or from the proposed feature-level supervision. The authors should provide more extensive ablation studies or analysis to disentangle these two effects.\n\n(2) The paper initially discusses \"motion blur and image degradation\" in broad terms, but the subsequent experiments primarily demonstrate effectiveness on blur and camera noise. Image degradation encompasses a much wider range of phenomena, such as lens distortion, rolling shutter artifacts, compression artifacts, low resolution, rain, and fog. It is questionable whether the proposed method maintains its robustness against these other types of degradation.\n\n(3) The proposed method requires a two-stage training process, but its training efficiency is not discussed. Furthermore, the computational efficiency of the method during inference remains unclear, and it is not specified whether it can meet real-time requirements.\n\n(4) The robustness of DINO features is not guaranteed in specialized domains, such as medical microscopy videos or materials analysis. Since the proposed method relies heavily on DINO features for semantic alignment, its broad applicability and performance in such out-of-distribution scenarios are uncertain.\n\nQUESTIONS:\nSee Weakness for details.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of event-based video frame interpolation (E-VFI), particularly in challenging scenarios where input keyframes may suffer from degradation like motion blur and noise. The authors argue that conventional methods, which rely on direct image-level supervision, are hampered by these artifacts, limiting their perceptual realism and generalization. To overcome this, they propose EPA, a novel framework that shifts the learning objective from the image domain to a semantic-perceptual feature space. The method operates in two stages: first, it trains a generator to reconstruct high-fidelity images from \"degradation-insensitive\" semantic features extracted by a frozen DINO-based model. Second, it introduces a Bidirectional Event-Guided Alignment (BEGA) module to align the features of the input keyframes to predict the features of the intermediate frame, guided by event data. The final interpolated frame is then synthesized from these aligned features. Experiments on several synthetic and real-world datasets show that EPA produces results with higher perceptual quality compared to state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n- The paper identifies a significant and practical challenge in video frame interpolation.\n- The authors conduct a comprehensive evaluation on a variety of synthetic (Vimeo90k, GOPRO) and real-world (HS-ERGB, BS-ERGB, EventAid-F) datasets.\n- The visual comparisons provided in Figures 4, 5, and 7 demonstrate the proposed method's strength.\n\n\nWeaknesses\n-  The central thesis of the paper is to move away from degraded image-level details by operating in a \"degradation-insensitive\" semantic feature space. However, the proposed Style Adapter (A_s) directly contradicts this motivation. The adapter is designed to \"compensate for this deficiency [of the VFM]... with essential low-level details\" by taking features from a lightweight image encoder that processes the (potentially degraded) input images and fusing them with the semantic features. This re-introduces the very low-level, potentially corrupted information that the framework ostensibly seeks to avoid. This creates a conceptual inconsistency at the heart of the proposed method. The paper fails to justify how this component does not simply propagate the input degradation.\n- The entire framework is predicated on the assumption that semantic features from a VFM like DINO are largely insensitive to image degradation. The evidence provided for this critical assumption is thin. Figure 8(a) shows that the L2 distance between features of a clean and a blurred image is smaller than the L2 distance between the images themselves. This is expected, as feature extraction is a dimensionality-reduction process that inherently discards some pixel-level variance. However, \"lower distance\" does not automatically equate to \"insensitivity\" or a more reliable supervisory signal. The features are still visibly different, and the paper provides no analysis to prove that these feature differences are less harmful to the learning process than the original image-level differences. This assumption requires much stronger empirical and theoretical support.\n- The core ideas underpinning EPA are not new. The use of feature-space supervision via pre-trained networks is the principle behind perceptual losses (e.g., LPIPS ), which have been a staple in image synthesis for years. The paper attempts to distinguish its work by performing explicit feature alignment and generation in this space, rather than just using it as a loss function. However, this is more of an architectural choice than a paradigm shift. The components themselves—a DINO feature extractor, deformable convolutions for alignment , and a flow-based generator —are all existing technologies. The contribution amounts to a novel combination of these parts, which is an engineering contribution.\n- While EPA excels in perceptual metrics, its performance on fidelity metrics (PSNR/SSIM) on the noisy BS-ERGB dataset is alarmingly poor compared to baselines. In Table 2, for 1-skip interpolation, EPA scores 27.943 in PSNR, whereas CBMNet and TLXNet achieve 29.257 and 29.298, respectively. This is a very significant drop. The authors argue that PSNR is not the goal, but a method truly robust to noise should not suffer such a precipitous decline in reconstruction fidelity. This result suggests that the method may be learning to generate perceptually plausible textures that deviate substantially from the ground truth structure, which calls into question its overall robustness.\n\nQUESTIONS:\n- Could you please elaborate on the rationale for re-injecting low-level features from the input keyframes via the Style Adapter? Given that the paper's primary motivation is to build resilience to defects in these keyframes, this appears counter-intuitive. Is there a mechanism that prevents the adapter from simply passing on the input noise or blur, and if so, how was it validated?\n\n- How do you interpret the significant drop in PSNR/SSIM on the noisy BS-ERGB dataset? Does this suggest that in the presence of heavy degradation, the feature-level supervision prioritizes perceptual \"plausibility\" to such an extent that it sacrifices structural fidelity to the ground truth? How can a user be confident that the generated output is faithful to the scene's motion and not a well-textured hallucination?\n\n- The paper claims to move away from \"direct image-level supervision\" , yet the final training stage employs the L_cg loss, which includes LPIPS and Laplacian loss on the output image. Could you clarify why this is not considered direct image-level supervision?\n\n- Beyond the L2 distance comparison in Figure 8, have you performed any other analysis to more rigorously quantify the \"degradation-insensitivity\" of DINO features? For example, how does the signal-to-noise ratio of the features compare to that of the input images under varying levels of degradation?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}