A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking
Authors: Zixiang Zhao
Haowen Bai
Bingxin Ke
Yukun Cui
Lilun Deng
Yulun Zhang
Kai Zhang
Konrad Schindler
Venue: NeurIPS 2025 spotlight ()
Keywords: image fusion
video fusion
low-level vision
Primary area: applications
TL;DR: We present UniVF, a unified video fusion framework with multi-frame learning; VF-Bench, the first benchmark covering four video fusion tasks; and a unified spatial–temporal evaluation protocol with a new temporal consistency loss.

Abstract: The real world is dynamic, yet most image fusion methods process static frames independently, ignoring temporal correlations in videos and leading to flickering and temporal inconsistency. To address this, we propose Unified Video Fusion (UniVF), a novel and unified framework for video fusion that leverages multi-frame learning and optical flow-based feature warping for informative, temporally coherent video fusion. To support its development, we also introduce Video Fusion Benchmark (VF-Bench), the first comprehensive benchmark covering four video fusion tasks: multi-exposure, multi-focus, infrared-visible, and medical fusion. VF-Bench provides high-quality, well-aligned video pairs obtained through synthetic data generation and rigorous curation from existing datasets, with a unified evaluation protocol that jointly assesses the spatial quality and temporal consistency of video fusion. Extensive experiments show that UniVF achieves state-of-the-art results across all tasks on VF-Bench. Project page: [vfbench.github.io](https://vfbench.github.io).

A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking
Zixiang Zhao'
Haowen Bai²
Bingxin Ke
Yukun Cui² Konrad Schindler!
Lilun Deng²
Yulun Zhang
Kai Zhang'
ETH Zürich
X Jiaotong University
Shanghai Jiao Tong University *Nanjing University
zixiang zhaoBethz.ch
Abstract
The real world is dynamic, yet most image fusion methods process static frames independently, ignoring temporal correlations in videos and leading to flickering and temporal inconsistency. To address this, we propose Unified Video Fusion (UniVF), novel and unified framework for video fusion that leverages multi- frame learning and optical flow-based feature warping for informative, temporally coherent video fusion. To support its development, we also introduce Video Fusion Benchmark (VF-Bench), the first comprehensive benchmark covering four video fusion tasks: multi-exposure, multi-focus, infrared-visible and medical fusion. VF-Bench provides high-quality, well-aligned video pairs obtained through syn- thetic data generation and rigorous curation from existing datasets, with unified evaluation protocol that jointly assesses the spatial quality and temporal consistency of video fusion. Extensive experiments show that UniVF achieves state-of-the-art results across all tasks on VF-Bench. Project page: vfbench.github.i
VF
Dataset: VF-Bench
Similarity Loss
Tempural Loss
BISWE
mnllii ficcts Saiin
worp
prodiction
SOUTEe STHADOS
Gradient Loss
prodiction
optical
exposure
grafient
warpped prol
Frsim
Model: UniVF
Losses
Metrics
Figure Overview our main contribution this paper.
Introduction
Image fusion has long been key research direction in computer vision and image processing. enables the combination of complementary information from multiple source images into single,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

more informative and perceptually enhanced result [1] A variety of fusion techniques, such as
melti-exposure 7 7&, multi-focus (9 4 infrared-visible [T2, and medical image fusion [TO] I..
have proven valuable in applications, such as low-light enhancement and exposure correction [2
131, extended depth-of-field imaging and generation of full-focus scenes 4 target recognition in
adverse conditions [14], and improved diagnostic support in clinical imaging [15]. These approaches
effectively overcome the limitations of individual sensors or imaging configurations, improving image interpretation by both human observers and machine vision systems. While image fusion has been extensively explored, transitioning to video fusion is a natural next step, as videos provide a continuous and temporally coherent view of dynamic scenes, including object and camera motion, transient events, and contextual variations [16]. With recently advanced imaging hardware and the increasing amount of video data, it has become feasible and necessary to extend image fusion to the temporal domain. The goal is to combine complementary information from multiple input videos into a single, temporally consistent output that offers a more complete representation of the scene.
However, the step from image to video domain introduces several new challenges beyond simply applying image fusion hame-by-framme (i) Leveraging temporal information: Processing frames independently ignores the inherent temporal continuity of videos, leading to flickering and motion discontinuities. Effective video fusion must incorporate information from adjacent frames, not only improving per-frame quality but also ensuring temporal coherence. (ii) Limited dataset scale: Compared to paired images, collecting perfectly aligned, temporally synchronized, and diverse video pairs is a lot more challenging and expensive, limiting benchmarking and development for data-driven fusion approaches. (iii) Lack of evaluation protocols: Existing evaluation metries are designed for images, while ignoring consistency along the temporal axis.
To tackle these challenges, we propose a Unified Video Fusion framework (UniVF) that explicitly incorporates multi-frame learning to exploit spatial-temporal information, thereby producing infor- mative and temporally consistent fused videos. Specifically, UniVF adopts a Transformer-based [17] encoder-decoder architecture and employs optical flow [8] to warp features from adjacent frames to the current one, effectively capturing temporal dependencies and integrating spatio-temporal relations. A dedicated temporal consistency loss further complements the standard fusion loss based on spatial similarity to suppress flickering and promote temporal continuity across frames.
We then propose a comprehensive Video Fusion Benchmark (VF-Bench) that covers four video fusion tasks: multi-exposure video fusion, multi-focus video fusion, infrared-visite video fusion. For the first two tasks, where paired videos are difficult to acquire directly, we propose novel data generation paradigms: To create multi-exposure data, we utilize 10-bit high dynamic range (HDR) videos, convert the encoded video signals into the linear light domain via the Electro-Optical Transfer Function (EOTF), and perform exposure adjustments to generate diverse exposure pairs; For the multi-focus case, we leverage advances in video depth estimation to simulate the optical focusing process, thereby creating realistic multi-focur video pairs from standard videos. For the latter two tasks, where realistic data synthesis is infeasible, we carefully curate existing datasets by defining objective selection criteria and conducting manual sereening to ensure data quality and sufficiently accurate alignment. Moreover, we develop a comprehensive suite of evaluation metrics that cover both the (per-frame) spatial quality and the (frame-to-frame) temporal consistency of fused video, providing a more holistic evaluation protocol.
fusion, and medical
Our main contributions can be summarized as follows, with an illustrative overview in Fig. !
We propose a novel Unified Video Fusion framework, UniVE, that explicitly incorporates multi- frame learning and cross-frame feature warping to exploit apatial-tempore information, producing informative and temporally consistent videos.
We construct the first comprehensive Video Fusion Benchmark, VF-Bench, by carefully designed data generation strategies and rigorous selection from existing datasets. VF-Bench provides well- aligned, high-quality video pairs across four representative video fusion tasks (multi-exposure, multi-focus, feffaare-visiblle and medical video fusion).
To train UniVF on VF-Bench, we introduce a temporal consistency loss alongside the conventional
image fusion losses, to suppress flickering and ensure smooth frame transitions in fused videos. We establish a comprehensive evaluation protocol for video fusion, integrating both spatial quality and temporal consistency metrics for a thorough assessment.
Experiments on VF-Bench demonstrate that our UniVF achieves state-of-the-a (SOTA) video fusion performance across all four sub-tasks, setting strong baseline for future research in video fusion.

---

W( Out1ae
optical flow estimation
--1
optical flow
E3
W( W
141-11
W( Sopeee
r conatatenation feature warping
W
W
0
Figure Detailed illustration of our UniVF architecture.
2
Related Work
In the era of deep learning, neural networks are frequently used in image fusion to extract source
features, merge features, and reconstruct the fused image [19 1 4 Image fusion algorithms are
commonly categorized into two categories: discriminative [20 and generative [24] 3
Discrimi-
native models, utilizing feature extractors such as CNNs 305 and Transformers 33 29 employ
model-driven (30 or data-driven 32 [3] approaches to obtain features from source images in
the image domain, frequency domain, or feature space [4 36) After information interaction and
fusion within the feature space, these models ultimately learn mapping from the source images to the fused image. On the other hand, generative fusion methods, like GANs 24 37 and Diffusion [3] models, perform modeling of the latent space manifold to minimize the distributional gap between source and fused images, providing more details in the fused results. Additionally, upstream image registration contributes to robust performance if inputs are misaligned [38] Guidance from downstream tasks [42 such as object detection [3 45 46) and semantic segmentation [47450], allows the model to learn more semantically relevant information. Furthermore, unified fusion models while meta-learning can help to better adapt the loss function and the feature extractor [54] 43 46 Vision-language models, through their more explicit semantics, offer more flexible guidance [56, (57.. Recently, video fusion, as a further advancement of image fusion, has been demonstrated for infrared and RGB inputs
leverage inter-task synergy [51555),
[58] 59). Moreover, multi-exposure sequences with alternating exposure times enable HDR video reconstruction [30) However, a unified video fusion framework and benchmark are still lacking.
3 UniVF: A Unified Video Fusion Framework
Overview. Given a pair of video sequences V1 = 14 and 1/2 = [17) where T is the total number of frames, the goal of video fusion is to generate fused video VF (4 ) that integrates complementary information from both inputs. In the following, we introduce our Unified Video Fusion framework, UniVE, and describe how utilizes spatial information within each frame and temporal dependencies between adjacent frames to produce temporally coherent fused videos.
3.1 UniVF Details
The proposed Vn framework is made up of four key components: feature extractor, optical flow estimator, feature warping module, and a feature decoder, which are responsible for extract- ing frame-wise features, estimating their displacements from frame to frame, aligning them, and reconstructing the fused frames, respectively. An illustration of the architecture is shown in Fig. 2
Feature Extraction. The goal of this component is extract domain-specifie and spatially rich features from each source video stream. Given a pair (V1, V2), for each time step t, we extract a snippet of three consecutive frames from each source: [7/_1, 1,,, The T where & E {1, 2). Each video
stream has dedicated encoder Ex(:..) consisting of several Restormer blocks [17], which is shared

---

across the three frames of the same source:
T E+1 = Ex TV, 1tt,,, h E {1,2}.
Optical Flow and Feature Warping. The difference between video fusion and single-frame image fusion lies in the ability to jointly reason over multiple consecutive frames. By exploiting information from preceding and succeeding frames, video fusion can capture dynamics and enhance feature extraction in the current frame. Thus, inspired by [64 651, we explicitly estimate dense optical flow to align features from adjacent frames to the current time step. Specifically, given two consecutive frames 1h and 1 (s E [t 1. + 1}), SEA-RAFT S(..)) [18). SOTA optical flow estimator, predicts the bidirectional flow 0
o .- S(1 h. E [1,2), E - 1,++)
Each optical flow field represents the motion of pixels from one frame to another, where each flow vector indicates the displacement of a pixel to corresponding location in the neighboring frame. We choose SEA-RAFT [18] for its combination of simplicity, efficiency, and accuracy, which suits our video fusion scenario. Then, temporally align features, UniVF performs feature warping based on these estimated flows via (differentiable) bilinear sampling. The bidirectional flow fields Ok s- are used to warp the deep features from adjacent frames to the current time step:
k: T-
W Oz,,), k E {1, E -t- 1,+
(3)
where W(- O) denotes warping according to the flow field O. Warped features - are temporally aligned with the target frame and serve as motion-compensann
inputs for subsequent fusion.
Fusion and Reconstruction. The 3 x 2 feature maps from both sources, warped to common reference, are concatenated along the channel dimension and fed to the Restormer- based [17 decoder D(-), which is tasked with modeling long-range dependencies in both the spatial and temporal dimensions:
きh = Concat (0). $2.
T1 I-
T1 8+1
T2 1111- 8e (.) I. D
(4)
Finally, the per-frame fusion results If" are reassembled into fused video sequence.
4 VF-Bench: A Video Fusion Benchmark
Overview.
To advance the development and evaluation of video fusion techniques and to promote further research into the topic, we have put together a Video Fusion Benchmark, VF-Bench, comprehensive benchmarking suite that includes four different video fusion scenarios: multi-exposure, multi-focus, infrared-visible, and medical video fusion. Examples are shown in Fig. [ The dataset offers large collection of paired video sequences with good quality, and precisely aligned to support both model training and testing. In the following we describe our data generation pipeline and the selection criteria. For additional visualizations, as well as further details about data preparation, please refer to Appendices and
Multi-Expasure Video Fusion. To construct multi extosure video pairs, we propose novel data processing pipeline with which we synthetically generate different exposure levels from 10-bit HDR source videos by adjusting exposure parameters, see Fig. aa. The use of 10-bit HDR videos advantageous because preserves a wide dynamic range, ensuring that even after exposure adjustments and potential quality degradation, details are retained that would be lost with 8-bit SDR sources. We start from the YouTube-HDD HDDR Dataset [66), a large-scale collection of short-form 10-bit HDR videos sourced from YouTube. From >200 candidates, with an average of 150 frames, choosing those with rich visual content, vivid colors, and free from watermarks video effects. These were further divided into 450 for training and 50 for testing.
manually curated 500 scenes
Since exposure depends approximately linearly on scene radiance, is essential to perform exposure adjustment in the linear light domain. In this way one accurately simulates radiometric changes and avoids distortions introduced by non-linear gamma-encoding. Therefore, we first convert the encoded video signals with the Electro- Optical Transfer Function (EOTF) and transform the video into a linear color space. Exposure adjustments are then applied in this linear domain by +3 EV (exposure value), simulating over-exposed and under-exposed conditions. To produce 8-bit videos,

---

BOTF
Adjust Exposure +3EV
80% quantile
focal depch
Input HDR
Lincar Light
Video Depth
Blar Radius (Far-focus))
Hlurred Video (fir- (Grr-ocuu))
BT.2020-9BT.00
Istimator
OETF
Input Video
Video Depth
near depth
Lineu->SDR
Blur Radius (neur focus)
Blurred Video (near- focus)
Outpurs
80-bit
sementic mask
nearest object
(8)
(b)
The proposed data generation paradigms for (a) multi-exposure video pair and (b) multi-focus vidco pair our VF- yF-Bench.
Figure 3:
then perform gamut mapping from the 10-bit BT.2020 color space the 8-bit BT.709 color space. The adjusted videos are mapped to 8-bit SDR video pairs using the BT.709 standard Gamma Opto-Electronic Transfer Function (OETF) [67]. to obtain paired over- and under-exposed video sequences. The described process closely resembles real-world multi-exposuree video capture, where single scene is recorded at different exposure settings, while ensuring consistent color mapping and precise spatial spatial-tempon temporal alignment across exposure levels. More details about this process are provided in Appendix(A.. A.I
Multi-Focus Video Fusion. Existing multi-focue image fusion datasets primarily rely on light field cameras 68 669, manually labeled focal masks 770 71 blur simulation based on foreground- background semantic segmentation masks (72). The former two approaches are expensive and difficult to scale up, whereas the last one does not follow the physical process: focal planes and associated circles of confusion (CoC), by definition, depend on continuous scene depth 73 rather than semantic labels.
Therefore, we propose to utilize depth maps to construct a multi-focus video dataset, by estimating the per-pixel blur radius, see Fig. b). We use the DAVIS dataset [74] video source for experiments. Specifically, 150 videos are split into 120 training scenes and 30 test scenes, with an
average length of 70 frames. We run single-view video depth estimator [75] them to obtain dense (inverse) depth. Given a focal depth, the CoC for pixel can be calculated as:
CoC' = Af
- D')/ f)1/D' = d d' /dylo.
(5)
with the aperture, I the focal length, D, the focal depth, and D' the depth of scene pixel To account for unknown camera metadata, the CoC is approximated by the estimated normalized inverse depth d', the given normalized focal depth dy, and a constant blur strength factor a. Further details on the derivation of Eq. (5) can be found in Appendix A.2
We select two normalized focal depth values, dfar and dycar, from the first frame of each video, representing the background and foreground focus, respectively. The background focal depth is the th percentile of the inverse depth values. To find the foreground focus depth. we compute an average depth value for each segmented object according to the DAVIS masks (74] and select the closest of them. Finally, a Gaussian blur is applied each pixel with the kernel size proportional to the calculated CoC, thereby generating paired multi-focus video sequences.
Infrared- Visible Video Fusion. Unlike the previous tasks, paired infrared-visible video datasets can be obtained from RGBT tracking benchmarks, whereas they are difficult to realistically simulate. To construct our video fusion dataset,
start from the VTMOT 76 dataset and implement three-stage filtering process to ensure data quality, accurate alignment, and diversity of content. First, infrared video frames are evaluated using three objective metries: Image Entropy, Global Contrast, and Dark Area Proportion. Frames that exhibit low entropy, insufficient contrast or excessive dark regions are considered low-quality and discarded, thus removing uninformative scenes. Second, for the RGB modality, we adopt Retinex theory to decompose each frame into illumination and reflectance components (77). RGB frames with high illumination values, indicating sufficient ambient lighting and limited need for infrared data, are excluded, thus retaining only video pairs where the infrared and visible channels provide complementary information. Finally, we perform frame-wiss fusion of the remaining video pairs using SOTA image fusion algorithms like CDDFuse II and FMMA 78), followed by manual inspection to validate the alignment and eliminate pairs affected by miistrggsttationn or ghosting artifacts. Further details of the complete selection criteria can be found
5

---

in Appendix A.3
Through this process, we curate a total of 90 video scenes with, on average, 300 frames. These are randomly split into 75 training scenes and 15 testing scenes.
Medical Video Fusion. For medical video fusion, we rely on the Harvard Medical dataset [79 as source, treating consecutive slices of MRT and corresponding CT, PET or SPECT volumes as video sequences. A filtering strategy similar to the one used for infrared-visible fusion is adopted to ensure data quality. Specifically, frames with large invalid regions or poor visual quality are removed, preserving only meaningful, interpretable sequences with rich visual details. As result, we curate a total of 57 scenes with 27 frames on average, which are divided into 49 for training and for testing.
5 Experiments
We now evaluate our UniVF on VF-Bench for all four fusion scenarios: multi-exposure fusion (MEF), multi-focus fusion (MFF), infrared-visible fusion (IVF), and medical video fusion (MVF). We first describe the experimental setup, with particular focus on the newly proposed temporal consistency term the training loss, as well as the temporal consistency evaluation metrics that we add to the single-frame evaluation protocol. Then, we discuss the results, which highlight that our approach already constitutes a strong baseline. Finally, we conduct ablation studics to validate our design choices. Further experimental results are shown in Appendix due to space limitations.
5.1 Setup
Loss Function. To jointly optimize spatial fidelity and temporal consistency, we adopt a compound training loss with three terms:
Eoputiial 01 Egrat + OrgCeemp:
where {ai, .2) are weight parameters, set to {10, 2). {1, 0.5). 2) and {1, 1) for MEF, MFF, IVF and MVF tasks respectively, such that the terms have comparable magnitudes. The three losses are (i) Spatial similarity: the spatial loss Cspanial measures per-pixel reconstruction error. Specifically, for IVF and MVF tasks, following (1.. Lspatial # max 41 12) For MEF tasks, following 55 we set Lypatial
Lint + LMFF-SSIM, where Lint - H 1 mean( 12)1l,, and EMEF-SSIM borrows from 80 For MFF task, we set Lupatial = Lintt Hw 7 mean 7t, T2) (ii) Gradient preservation: to preserve image structures and edges, following (56. we introduce a dedicated gradient loss Lar = =l
V [ V denotes the Sobel gradient operator. (iii) Temporal consistency: To suppress flickering and ensure smooth transitions across frames, we introduce a temporal consistency loss that explicitly enforces hame-to-frame consistency, by penalizing misalignments between adjacent frames:
uump BpE My (T - W (F (p)]
+ EpEM, [1 - W (f (p)],]
(7)
where 1," is the fused video sequence from Eq. (4) and W(: O) denotes warping with the optical flow field O. M and Mien are validity masks at time step that indicate regions with reliable flow estimates, as detailed below. This loss term implements the strong temporal continuity of videos with reasonable frame rates, by enforcing consistency between the current fused frame and its two warped neighbors, so as to reduce flickering and abrupt changes.
Validity masks. To improve the robustness of the temporal consistency loss term Lump in Eq. and avoid unreliable gradients, we introduce validity masks ( [Mr preu. Mi to identify well-aligned, non-occluded regions between adjacent frames. Each mask derived via consistency check: given the forward flow O,-r+1 and backward flow Oc+1-st, the latter is first warped onto the coordinate space of frame as O (p) W (O++1-tt O (p)). Since the backward flow is defined in the pixel space of frame L+1 while the forward flow originates from frame 1, warping enables both flows to be compared in the same coordinate space. The consistency error
flow
and the binary mask is defined as M(p) = 1 if (pp) < otherwise O. is a predefined threshold (set to 1.0 in our implementation). Intuitively, this verifies whether pixel can move forward and then return along the estimated flow paths with minimal deviation. Large inconsistencies typically indicate occlusions, motion boundaries, or flow
is computed as A(p) O (p) + Ō++1-

---

Previous Frame
Current Frame
Next Frame
M pezv
Maca
Previous Frame
Current Frame
Next Frame
M peev
M
Figure 4: Previous, current, and next frames with their corresponding validity masks Mine and M Black regions denote invalid unreliable areas, corresponding poorly aligned occluded pixels that excluded from temporal consistency computation.
Under
Over
CUNet
DDMEF
HoLoCo
CRMEF
TC-MoA
FILM
ReFussoo
Ours
Figure 5: Qualitative comparison of fusion outcomes for multi-exposure video fusion.
ambiguities, By restricting the temporal consistency loss to these valid regions, UniVF effectively reduces flickering and enforces smooth temporal transitions. Visual examples of [Mpeev Miea) shown in Fig.
Training details. We RTX 4090 GPU. The loss is minimized with Adam, starting with a learning rate of 10-4 that decays exponentially 1% of its initial value the course of 20k iterations. Training 32, with gradient accumulation. As our network architecture, we adopt Restormer blocks [17] in both the encoder (-(-) and decoder D(+) components. Each block contains 8 attention heads and has feature dimension of 32. Both the encoders and decoder are configured with 4 stacked blocks.
experiments machine equipped with single NVIDIA GeForce
batch size of
Metrics. (i) Spatial domain evaluation metrics: We adopt four widely used quantitative metrics: VIF (Visual Information Fidelity), SSIM (Structural Similarity Index), MI (Mutual Information), and QAB/F These indicators measure perceptual fidelity, structural similarity, mutual information content, and edge preservation, respectively. In all cases, a higher value means better fusion of the complementary information from the two sources. For further details, see [81]. (ii) Temporal domain consistency metrics: To assess the temporal consistency and motion smoothness of fused videos, proposed two complementary evaluation metrics.
Bi-Directional Self- Warping Error (BiSWE): quantify
As a reference-fiee free metric, BiSWE is designed to temporal alignment errors within video sequence. Given video clip [1-1. Ie. I.1). we compute the optical flow fields Oz-st = S(I,, It). {t - 1, 1} with SEA-RAFT -(..) [8]. Validity masks { preve Mirx) are applied to exclude unreliable regions
based on forward-backwart consistency. The BiSWE value is computed as
BiSWE MP [T(()) W (T,-1, 0,11-1)) (p)h,
+ Ep (1((p) - W O4411x)411)))00 (p)li),
(8)
7

---

Near
Far
CUNet
IFCNN
EPT
RFL
FILM
ReFusion
TC-MoA
Ours
Figure 6: Qualitative comparison of fusion outcomes for multi-focus vidco fusion.
Infrared
Visible
CDDFuse
EMMA
TC-MoA
FILM
RcFusion
Text-DiFuse
TDFusion
DCEvo
Ours
Figure 7: Qualitative comparison of fusion outcomes for infrared-visible video fusion.
where W(: O) is the warping function guided by the flow O, and E denotes averaging over valid pixels. Lower BiSWE indicates improved temporal alignment.
Motion Smoothness with Dual Reference Videos (MS2R): To assess the naturalness and consistency of motion transitions, MS2R evaluates the coherence of flow changes in the fused video and two reference sequences. The flow change is defined as the difference between consecutive flows within clip. Intuitively, compares the accelerations of objects projected onto the image plane. The MS2R score is defined
MS2R Ep (p) AOR: (p)1.] + E. | (p) RO (p)l.),
where AOF = of - Of and Of are also obtained from S(.,)) 18 R R. computed similarly from the two reference sequences. lower MS2R indicates smoother and more natural motion trajectories in the fusion video.
5.2 Video Fusion Experiments
Multi-Exposure Video Fusion. We ran experiments on the MEF branch of VF-Bench. Tested
methods include CUNet (9) DDMEF [82). HoL-Coo [3]. CRMEF 84 TC-MoA (55), FILM [56]
and ReFusion [54]. As illustrated in Tab. and Fig. VniVF consistently achieves superior

---

MRI
SPECT
CDDFuse
FMMA
FILM
ReFusion
Text-DiFuse
BSAFusion
C2RF
Ours
Figure 8: Qualitative comparison of fusion outcomes for medical vidco fusion.
Table 1: Quantitative evaluation results for the MEF and MFF task. The red and blue highlights indicate the
highest second- -iihhsst scores.
VF-Bench Multi- Exposure Video Fusion Branch
VF-Bench Multi-Focus Video Fusion Branch
VIFt
SSIM
MI
Qubft
BiSWEL
MS2R.
VIFt
SSIM?
MI1
Qabf'
BiSWEJ
MS2R_
CUNet
0.58
0.67
2.26
0.39
7.12
0.42
CUNet
0.56
0.88
3.74
0.53
6.79
1.47
DDMEF
0.72
0.94
2.71
0.65
10.06
1.04
IFCNN
0.69
0.89
4.93
0.70
6.25
1.38
HoLoCo
0.39
0.78
CRMEF
0.64
0.95
2.61
0.61
8.68
0.42
EPT
0.77
0.90
6.31
0.78
5.97
1.14
TC-MoA
0.76
0.71
7.78
0.34
TC-MoA
0.77
5,46
1.13
FILM
0.78
0.98
4.39
0.71
8.27
0.34
FILM
0.76
0.89
5.02
0.75
6.32
1.28
ReFus
0.75
0.97
0.70
7.95
ReFus
0.73
4.95
0.73
5,80
1.28
Ours
0.82
0.99
4.45
0.72
6.40
0.33
Ours
0.79
0.90
6.32
0.79
5.95
1.08
quantitative and qualitative performance, effectively balancing dynamic range expansion, contrast enhancement, and image quality preservation across multiple exposure levels.
Multi-Focus Video Fusion. For the experiments on the MFF branch of VF-Bench, the tested methods
include CUNet (9. IFCNN 855 RFL [86], EPT 877, TC-MoA (55]. FILM [56). and ReFusion [54].
As shown in Fig. 6 and Tab. our UniVF baseline again achieves the best performance both qualitatively and quantitatively, accurately identifying focused regions and producing sharp fusion
results free of artifacts, across both the foreground and background.
Notably, both training and testing for the MEF and MFF branches were conducted on the 2K- resolution version of the dataset. Considering potential computational resource constraints, we also report results low-resolution version in the Appendix for reference.
Infrared-Vistee Video Fusion. We conducted experiments on the IVF branch of VF-Bench, with CDDFuse 88 EMMA [14, TC-MoA [55]. MRFS [50, FILM (56), ReFusion [54] Text-
DiFuse (57. TDFusion
4 and DCEvo 42). The results once more show superior performance of
UniVE, in terms of both visual quality and quantitative metries. As illustrated in Fig. 7 the method faithfully preserves critical thermal and structural details, enhances object visibility and reduces noise in low-light conditions. Quantitative comparisons in Tab. further confirm that UniVF consistently has an edge in most metrics, highlighting its robustness across diverse scenes and object types.
Medical Video Fusion. On the MVF branch of VF-Bench, we evaluate range of methods including CDDFuse 88). EMMA 14]. FILM (56). ReFusion [54]. Text-DiFuse [57], BSAFusion [89 and C2RF 38 As can be seen in Fig. g and Tab. 3 UniVF once more effectively preserves fine-grained
textures from MRI images, while simultaneously enhancing and maintaining the salient intensities of
the CT, PET SPECT modality. The fused results exhibit clear anatomical details and clean tissue boundaries from the MRI source, along with distinct color distributions originating from the SPECT images support clinical diagnosis.
Ablation Studies. To explore contribution of each key component within UniVE, we conducted ablation studies the IVF task, with results summarized in Tab. In Exp. I, we removed the feature warping module, i.e., multi-fran falti-frame features are directly concatenated along the channel dimension and fed into the decoder, without optical flow correction. In Exp. II. both the feature warping module

---

Table 2: Quantitative evaluation for the IVF task.
Table 3: Quantitative evaluation for the MVF task.
VE-Bench Infrared-Vislae Fusion
VF-Bench Medical Video Fusion Branch
VIF
SSIM
MI
Qubi
BiSWE
MS2R 1
VIF
SSIM
1
Qahi
BiSWE:
EMMA
CDDF
0.37
0.37
0.63
0.61
2.41
2.01
0.54
0.58
4.79
5.12
0.37
0.37
CDDF
0.29
0.76
1.80
0.59
26.33
1.34
TC-MoA
205
EMMA
0.29
0.68
1.73
0.60
30.00
1.98
0.36
32.04
FILM
0.40
0.83
2,05
0.64
4.78
0.37
ReFus
0.31
0.32
1.74
0.67
32.85
1.74
Refas
0.42
0.54
2.27
0.67
4,64
0.36
Text-D
0.52
34.09
Text-
0.30
0.60
1.64
0.39
10.63
0:40
BSAF
0.28
0.63
1.69
0.58
34.73
1.66
TDFusion
DCEvo
4.57
0.37
C2RF
0.30
0.73
1.75
0.59
32.67
2.06
Ocas
0.44
0.64
2,47
0.68
3.34
0:35
Ours
0.35
0.76
2,00
0.68
29.61
1,30
Table 4: Ablation experiments results, with red representing the best values.
Configurations
Matrics
Descriptions
feature warping
multi-inputs
Dump
VIF
SSIM 7
MI 1
Qabf 1
BiSWE
MS2R
Exp. I: w/o fenture warping
0.40
0.63
2.44
0.66
4.18
0.36
Exp. II: w/o warping & multi inputs
0.38
0.61
2,07
0.64
4.46
0.37
Exp. III: w/o Cump
0.42
0.65
2.38
0.65
5.79
0.39
UniVF (Ours)
0.44
0.64
2.47
0.68
3.94
0.35
and multi-frame inputs were removed, reverting the model to conventional numee-b-ffram fusion scheme. To ensure a fair comparison, we increased the number of Restormer blocks to maintain an equivalent total parameter count. In Exp. III, while retaining the original multi-frame fusion structure with feature warping, we switched off the temporal consistency loss Limp during training.
Taken together, the ablation experiments demonstrate the necessity of each component in our scheme. Specifically, multi-frame feature warping enhances temporal coherence and overall fusion quality, while the temporal consistency loss further ensures smooth transitions across consecutive frames. Their combination yields superior video fusion compared simplified frame-wise baselines.
6 Conclusion
We have presented UniVF, a unified framework for video fusion that explicitly leverages multi-frame learning and optical flow-based feature warping exploit both spatial and temporal information. To ensure temporal coherence of the fused results, we introduce custom temporal consistency loss that suppresses flickering and enforces smooth frame transitions. Additionally, we have introduced a comprehensive VF-Bench, to our knowledge the first benchmark for video fusion. It covers four representative tasks with carefully constructed, paired video datasets and a holistic evaluation protocol, including dedicated temporal consistency metrics. Extensive experiments demonstrate that UniVE, with its straightforward design, achieves SOTA performance across all four tasks. We hope that our benchmark, together with the strong baseline of our fusion framework, encourages further research into video fusion and lays a solid foundation for it.
Acknowledgments
This work was supported by Huawei Technologies Oy (Finland), and by SwissAI Compute Grant.
References
[1] Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin. Radu Timofte, and Luc Van Gool. Cddfuse: forrelation-drives dual--ranch feature decomposition for multi-modality image fusion. In IEEE Conf. Comput. Vis. Pattern Recog,, pages 5906-5916. 2023.
[2] Zixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang. Shuang Xu, Yulun Zhang Kai Zhang, Deyu Meng. Radu Timofte, and Luc Van Gool. Ddffm: Denoising diffusion model for multi-modality image fusion. In Inl. Conf. Compul. Vis., pages
2023.
[3] Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu. Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-awart aware dual adversarial learning and scetarccenario modai-yo benchmark to fuse infrared and visible for object detection. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7792-5801, 2022.
10

---

[4] Xingchen Zhang. Deep learning-based multi-focus image fusion: A survey and comparative study, IEEE Trans. Pattern Anal. Mach. Intell., 449))4819-4488 2021.
[5] Zhengxue Wang. Zhiqiang Yan, Jinshan Pan, Guangwei Gao, Kai Zhang, and Jian Yang. Dornet: A degradation oriented and regularized network for blind depth super-resolution. In IEFE Conf. Comput. Vis. Pattern Recog, pages 18813-1522... 2025.
[6] Zeyu Xiao and Xinchao Wang. Event-based video super-resolution via state space models. In JEEE Conf. Comput. Vis. Pattern Recog-, pages 12564-12574, 2025.
[7] Kede Ma, Zhengfang Duanmu, Hanwei Zhu, Yuming Fang, and Zhou Wang. Deep guided learning for fast expoi-exposure image fusion. IEEE Trans. Image Process., 29:2808 2819, 2020.
[8] Kede Ma, Hui Li, Hongwei Yong. Zhou Wang, Deyu Meng, and Lei Zhang. Robust multi-exposuuee image fusion: A structural patch decomposition approach. IEEE Trans. Image Procesx. 26(5):2519-22 2017.
[9] Xin Deng and Pier Laigi Dragotti. Deep convolutional neural network for multi-modal image restoration and fusion. IEFE Trans. Pattern Anal. Mach. Intell., 43(10))333333344 2020.
[10] Alex Pappachen James and Belur V. Dasarathy, Medical image fusion: A survey of the state of the art. Inf. Fusion, 19:4-19, 2014.
[11] Han Xu and Jiay Ma. Emfusion: An unsupervised enhanced medical image fusion network. Inf. Fusion, 76:177-186, 2021.
[12] Mingde Yao, Menglu Wang. King-Man Tam. Lingen Li, Tianfan Xue, and Jinwei Gu. Polarfree: Polarization-based reflection-free free imaging. In JEEE Conf. Comput. Vis. Pattern Recog, pages 10890-10899, 2025.
[13] Zixuan Chen, Yujin Wang, Xin Cai, Zhiyuan You, Zheming Lu, Fan Zhang, Shi Guo, and Tianfan Xue. Ultrafusion: Ultra high dynamic imaging using exposure fusion. In IEEE Conf. Comput. Vis. Puttern Recog.. pages 16111-16121, June 2025.
[14] Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Kai Zhang, Shuang Xu, Dongdong Chen, Radu Timofte, and Lue Van Gool. Equivariant multi-modality image fusion. In IEEE Conf. Comput. Vis. Pattern Recog., pages 25912-25921. IEEE, 2024.
[15] Jiawei Li, Jinyuan Liu, Shihua Zhou, Qiang Zhang, and Nikola K. Kasabov. Gesenet: A general semantic- guided network with couple mask ensemble for medical image fusion. IEEE Trans. Neural Networks Learn. Syst. pages 1-14, 2023.
[16] Zhuoyuan Li, Junqi Liao, Chuanbo Tang, Haotian Zhang, Yuqi Li, Yifan Bian, Xihua Sheng, Xinmin Feng, Yao Li, Changsheng Gao, et al. USTC-TD: A test dataset and benchmark for image and video coding in 2020s. IEEE Trans. Multimedia, 2025.
[17] Syed Waqas Zamir, Aditya Arora. Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- -ssun Yang. Restormer: Efficient transformer for high-resolution image restoration. In IEEE Conf. Comput. Vis. Pattern Recog,, pages 5718-5715 0729,, 2022.
[18] Yihan Wang, Lahav Lipson. and Jia Deng. SEA-RAFT: simple, efficient, accurate RAFT for optical flow. In Eur. Conf. Comput. Vis., volume 15065, pages 36 54. Springer, 2024.
[19] Julius Erbach, Dominik Narnhofer, Andreas Dombos, Bernt Schiele, Jan Eric Lenssen, and Konrad Schindler. Solving inverse problems with flair. arXiv preprint 2xXiv:2506.0266 2025.
[20] Shuang Xu, Jiangshe Zhang, Zixiang Zhao, Kai Sun, Junmin Liu, and Chunxia Zhang. Deep gradient projection networks for pan-sharpening. In IEEE Conf. Comput. Vis. Puttern Recog., pages 1366-1375. 2021.
[21] Zixiang Zhao, Shuang Xu, Jiangshe Zhang, Chengyang Liang, Chunxia Zhang, and Junmin Liu. Efficient and model-based infrared and visible image fusion via algorithm unrolling. IEEE Transactions on Circuits and Systems for Video Technology, 32(3):1 1ese1186-11 1196, 2022.
[22] Hui Li, Tianyang Xu, Xiaojun Wu, Jiwen Lu, and Josef Kittler. Lrmet: A novel representation learning guided fusion network for infrared and visible images. IEEE Trans. Pattern Anal. Mach. Intell., 45(9): 1040- 1052, 2023.
[23] Zixiang Zhao, Jiangshe Zhang, Shuang Xu, Zudi Lin, and Hanspeter Plister. Discrete cosine transform network for guided depth map super-resolution In IEEE Conf. Comput. Vis. Pattern Recog., pages 5687-5697, 2022.
11

---

[24] Jiayi Ma. Wei Yu, Pengwei Liang, Chang Li, and Junjun Jiang. Fusiongan: A generative adversarial network for infrared and visible image fusion. Inf. Fusion, 48:11-26, 2019.
[25] K Ram Prabhakar, V Sai Srikar, and R Venkatesh Babu. Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. Int. Conf. Comput. Vis., pages 4724-4732, 2017.
[26] Dong Han, Liang Li, Xiaojie Guo, and Jiayi Ma. Multi-exposure image fusion via deep perceptual enhancement. Inf. Fusion, 79:248-262, 2022.
[27] Fangyuan Gao, Xin Deng, Mai Xu, Jingyi Xu, and Pier Luigi Dragotti. Multi-modal convolutional dictionary learning, IEEE Trans. Image Process., 31:1325- 1111325-3399 2022.
[28] Odysscas Bouzos, Toannis Andreadis, and Nikolaos Mitianoudis. A convolutional neural network-hased conditional random field model for structured multi-focus image fusion rohust to noise. IEFE Trans. Image Process. 32:2915-29338 2023.
[29] Yuanshen Guan, Ruikang Xu, Mingde Yao, Lizhi Wang, and Zhiwei Xiong. Mutual-guided dynamic network for image fusion. In ACM Int. Conf. Multimedia, pages 1779-1788, 2023.
[30] Xin Deng. Yutong Zhang. Mai Xu, Shuhang Gu, and Yiping Duan. Deep coupled feedback network for joint exposure fusion and image super-resolution. IEEE Trans. Image Process. 30:3098 3112, 2021.
[31] Linhao Qu. Shaolei Liu. Manning Wang, and Zhijian Song. Transmef: A transformer-based multi-exposure image fusion framework using self-supervised multi-task learning. In AAAI, pages 2126-2134, 2022.
[32] Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, Jiangshe Zhang, and Pengfei Li. DIDFuse: Deep image decomposition for infrared and visible image fusion. In IJCAI, pages 970-976, 2020.
[33] Hui Li and Xiao-Jun Wu. Densefuse: A fusion approach to infrared and visible images. IEEE Trans, Image Process., 2865):2614-2623, 2018.
[34] Haoyu Ma. Qingmin Liao, Juncheng Zhang, Shaojun Liu, and Jing-Hao Xue. An a-matte boundary defocus model- based cascaded network for multi-focus image fusion. IEEE Trans. Image Process., 29:8668 8679, 2020.
[35] Zeyu Wang. Xiongfei Li. Haoran Duan, and Xiaoli Zhang. A self-supervised residual feature learning model for multifocus image fusion. IEEE Trans. Image Process., 31:4527-4542. 2022.
[36] Zeyu Wang, Xiongfei Li, Libo Zhao, Haoran Duan, Shidong Wang, Hao Liu, and Xiaoli Zhang. When multi-focus image fusion networks meet traditional edge-preservation technology. Int. J. Comput. Vis., pages 1-24, 2023.
[37] Jiayi Ma, Han Xu, Junjun Jiang, Xiaoguang Mei, and Xiao-Ping (Steven) Zhang. Ddegan: A dual- discriminator conditional generative adversarial network for multi-resolution image fusion. IEFE Trans. Image Process., 29:4980-4995 2020.
[38] Linfeng Tang, Qinglong Yan, Leyuan Fang Xinyu Xiang, and Jiayi Ma. C2rf: Bridging multi-modal image registration and fusion via commonality mining and contrastive learning. Int. J. Compia. Vis., 2025.
[39] Di Wang, Jinyuan Liu, Xin Fan, and Risheng Liu. Unsupervised misaligned infrared and visible image fusion via modallty image generation and registration. In IJCAI, pages
2022.
[40] Zhiying Jiang, Zengxi Zhang, Xin Fan, and Risheng Liu. Towards all weather and unobstructed multi- spectral image stitching: Algorithm and benchmark. In ACM Int. Conf. Multimedia, pages 3783-3791, 2022.
[41] Han Xu, Jiayi Ma, Jiteng Yuan, Zhuliang Le, and Wei Liu. Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion. In IEEE Conf. Comput. Vis, Pattern Recog., pages 19647-19656, 2022.
[42] Jinyuan Liu, Bowei Zhang, Qingyun Mei, Xingyuan Li, Yang Zou, Zhiying Jiang, Long Ma, Risheng Liu, and Xin Fan. Deevo: Discriminative cross- dimensional cvolutionary learning for infrared and visible image fusion. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2226-2235. 2025.
[43] Haowen Bai, Jiangshe Zhang, Zixiang Zhao, Yichen Wu, Lilun Deng. Yukun Cui, Tao Feng, and Shuang Xu. driven image fusion with learnable fusion loss. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7457-7468, 2025.
12

---

[44] Haowen Bai, Zixiang Zhao. Jiangshe Zhang, Baisong Jiang, Lilun Deng, Yukun Cui, Shuang Xu, and Chunxia Zhang. Deep unfolding multi-modal image fusion network via attribution analysis. IEEE Trans. Circuit Syst. Video Technol. 35(4):3498 2025.
[45] Yiming Sun, Bing Cao, Pengfei Zhu, and Qinghua Hu. Detfusion: A detection-driven infrared and visible image fusion network. In ACM Inf. Conf. Multimedia, pages 4003-4011, 2022.
[46] Wenda Zhao, Shigeng Xie, Fan Zhao, You He, and Huchuan Lu. Mctafusion: Infrared and visible image fusion via meta- feature embedding from object detection. In IEEE Conf. Comput. Vis. Pattern Recog.. pages 13955-13965. IEEE, 2023.
[47] Linfeng Tang, Yuxin Deng, Yong Ma, Jun Huang, and Jiayi Ma. Superfusion: A versatile image registration and fusion network with semantic awareness. IEEE/CAA Journal of Automatica Sinica, 9(12):112-21) 2022.
[48] Linfeng Tang, Jiteng Yuan, and Jiayi Ma. Image fusion in the loop high-level vision tasks; A semantic- aware real-time infrared and visible image fusion network. Inf. Fusion, 82:28-42, 2022.
[49] Jinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma, Risheng Liu, Wei Zhong, Zhongxuan Luo, and Xin Fan. Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation. In Int. Conf. Comput. Vis., pages 81 15-8124, 2023.
[50] Hao Zhang, Xuhui Zuo, Jie Jiang, Chunchao Guo, and Jiayi Ma. Mrfs: Mutually reinforcing image fusion and segmentation. In JEEE Conf. Comput. Vis. Puttern Recog., pages 26974-26983, 2024.
[51] Chunyang Cheng, Tianyang Xu, Zhenhua Feng, Xiaojun Wu, Zhangyong Tang, Hui Li, Zeyang Zhang, Sara Atito Ali, Muhammad Awais, and Josef Kittlr. One model for ALL: low-level task interaction is a key to task-agnostic image fusion. In IEEE Conf. Comput. Vis. Pattern Recog., pages 28102-28112 12, 2025.
[52] Han Xu, Jiayi Ma, Junjun Jiang, Xiaojie Guo, and Haibin Ling. U2fusion: A unified unsupervised image fusion network. IEEE Trans. Pattern Anal. Mach. Intell., 44(1):502 518, 2022.
[53] Pengwei Liang. Junjun Jiang, Xianming Liu, and Jiayi Ma. Fusion from decomposition: A self-supervised decomposition approach for image fusion. In Eur. Conf. Comput. Vis., pages 719-735, 2022.
[54] Haowen Bai, Zixiang Zhao, Jiangshe Zhang, Yichen Wu, Lilun Deng, Yukun Cui, Baisong Jiang, and Shuang Xu. Refusion: Learning image fusion from reconstruction with learnable loss via meta-learning. Int. J. Comput. Vis., (3555225577- 2025.
[55] Pengfei Zhu, Yang Sun, Bing Cao, and Qinghua Hu. Task-customizze mixture of adapters for general image fusion. In JEEE Conf. Comput. Vis. Pattern Recog., pages 7099-7108. IEEE, 2024.
[56] Zixiang Zhao, Lilun Deng, Haowen Bai, Yukun Cui, Zhipeng Zhang, Yulun Zhang, Haotong Qin, Dongdong Chen, Jiangshe Zhang, Peng Wang, and Luc Van Gool. Image fusion via vision-language model. In Int. Conf. Mach. Learn., 2024.
[57] Hao Zhang, Lei Cao, and Jiayi Ma. Text-difuse: An interactive multi-modal image fusion framework based on mext-modulated diffusion model. In Adv Neural Inform. Process. Syst., 2024.
[58] Linfeng Tang. Yeda Wang. Meiqi Gong, Zizhuo Li, Yuxin Deng, Xunpeng Yi, Chunyu Li, Han Xu, Hao Zhang, and Jiayi Ma. Videofusion: A spatio-temporal collaborative network for mutii-modal video fusion and restoration. arXiv preprint arXi::2503.23359, 2025.
[59] Housheng Xie, Meng Sang, Yukuan Zhang, Yang Yang. Shan Zhao, and Jianbo Zhong. Revs: unified registration and fusion framework for video streams. IEEE Truns. Multimedia, 2024.
[60] Gangwei Xu, Yujin Wang, Jinwei Gu, Tianfan Xue, and Xin Yang. Hdrflow: Real-time hdr video reconstruction with large motions. In IEEE Conf. Comput. Vis. Pattern Recog., pages 24851-24860, 2024.
[61] Guanying Chen, Chaofeng Chen, Shi Guo, Zhetong Liang, Kwan-Yee Wong, and Lei Zhang. Hdr video reconstruction: A coarse-to-fine network and a real-world benchmark dataset. In Int. Conf. Comput. Vis., pages 2502 2511, 2021.
[62] Yong Shu, Liquan Shen, Xiangyu Hu, Mengyao Li, and Zihao Zhou Towards real-world hdr video reconstruction: A large-scale benchmark dataset and two-stage alignment network. In IEEE Conf. 2024.
Comput. Vis. Pattern Recog,, pages
[63] Haesoo Chung and nam IK Cho. Lan-hdr: Luminance- based alignment network for high dynamic range video reconstruction. In Int. Conf. Comput. Vis., pages 12760 12769, 2023.
13

---

[64] Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Basicvsr++: Improving video super-resolution with enhanced propagation and alignment. In IEEE Conf. Comput. Vis. Pattern Recog.. pages 5972-5981, 2022.
[65] Jingyun Liang. Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. VRT: A video restoration transformer. IEFE Trans. Image Process, 33:2171-2182, 2024.
[66]
Yilin Wang. Joong Gon Yim, Neil Birkbeck, and Balu Adsumilli. Youtube SFV+HDR quality dataset. In IEEE Int. Conf. Image Prucess., pages 96-102. IEEE, 2024.
[67] BT.21000 Image parameter values for high dynamic range television for use in production and international programme exchange. https::/www itu. int/rec/Rttitei R- RE--REE--- BT. 2100 2018. Accessed: 2021-02-02.
[68] Mansour Nejati, Shadrokh Samavi, and Shahram Shirani. Multi-focus image fusion using dictionary- based sparse representation. Inf. Fusion, 25:72-84, 2015.
[69] Juncheng Zhang. Qingmin Liao, Shaojun Liu, Haoyu Ma. Wenming Yang. and Jing Hao Xue, Real-mff. A large realistic fullii-oouus image dataset with ground truth. Pattern Recognition Letters, 138:370-377, 2020.
[70] Shuang Xu, Xiaoli Wei, Chunxia Zhang, Junmin Liu, and Jiangshe Zhang. Mffw: A new dataset for multi-focus image fusion. arXiv preprint mrxXii:2002.047700 2020.
[71] Hao Zhang, Zhuliang Le, Zhenfeng Shao, Han Xu, and Jiayi Ma. MFF-GAN: an unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion. Inf. Fusion, 66:40-53, 53,40-53222 2021.
[72] Xiaopeng Guo, Rencan Nie, Jinde Cao, Dongming Zhou, Liye Mei, and Kangjian He. Fusegan: Learning to fuse oulti-focus image via conditional generative adversarial network. IEEE Trans. Multimedia, 118811998,, 2019.
[73] R. Fernando. GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics. Addison- Wesley Professional, 2004.
[74] Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine- Sorkine-Horuung, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675, 2017.
[75] Bingxin Ke, Dominik Narnhofer, Shengyu Huang, Lei Ke, Torben Peters, Katerina Fragkiadaki, Anton Obukhov, and Konrad Schindler. Video depth without video models. In IEEE Conf. Comput. Vis. Pattern Recog,, 2025.
[76] Yabin Zhu, Qianwu Wang. Chenglong Li. Jin Tang, Chengjie Gu, and Zhixiang Huang. Visible thermall multiple object tracking: Large-scale video dataset and progressive fusion approach. Pattern Recognit., 161:1113300 2025.
[77] Zhenqi Fu, Yan Yang, Xiaotong Tu, Yue Huang, Xinghao Ding, and Kai-Kuang Ma. Learning a simple low-light image enhancer from paired low-light instances. In IEEE Conf. Comput. Vis. Pattern Recog., pages 22252-22261. IEEE, 2023.
[78] Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Kai Zhang, Shuang Xu. Dongdong Chen, Radu Timofte, and Lue Van Gool. Equivariant multi-modality image fusion. In IEEE Conf. Comput. Vis. Pattern Recog., pages 1091222592... JEEE, 2024.
[79] BKeith A. Johnson and J. Alex Becker. Harvard medical website. http://wwes med. harvard. edu/ AANLIB/home html
[80] Kede Ma. Kai Zeng, and Zhou Wang. Perceptual quality assessment for multi-exposure image fusion. IEEE Trans, Image Process., 24(1 (4(11):3345-33 3356,,3362220 2015.
[81] Jiay Ma, Yong Ma, and Chang Li. Infrared and visible image fusion methods and applications: A survey. Inf. Fusion, 15:1537178, 2019.
[82] Xiao Tan, Huaian Chen, Rui Zhang, Qihan Wang, Yan Kan, Jinjin Zheng, Yi Jin, and Enhong Chen. Deep multi-exposure image fusion for dynamic scenes. IEEE Trans. Image Process., 02:5310-5325, 2023.
[83] Jinyuan Liu, Guanyao Wu, Junsheng Luan. Zhiying Jiang, Risheng Liu, and Xin Fan. Holoco: Holistic and local contrastive learning network for multi nulti-exposure image fusion. Inf. Fusion, 95:237-244 2023.
14

---

[84] Zhu Liu, Jinyuan Liu, Guanyao Wu, Zihang Chen. Xin Fan. and Risheng Liu, Searching a compact architecture for robust multi-exposure image fusion. IEEE Trans. Circuit Syst. Video Technol., 34(7):6224- 6237, 2024.
[85] Yu Zhang, Yu Liu, Peng Sun, Han Yan, Xiaolin Zhao, and Li Zhang. Ifenn: A general image fusion framework based on convolutional neural network. Inf. Fusion, 54:99-118. 118. 2020.
[86] Zeyu Wang. Xiongfei Li, Haoran Duan, and Xiaoli Zhang. A self-supervised residual feature learning model multifocus image fusion. IEEE Trans. Image Process. 31:4527-4542, 2022.
[87] Zeyu Wang. Xiongfei Li, Libo Zhao, Haoran Duan, Shidong Wang, Hao Liu, and Xiaoli Zhang. When multi-focus image fusion networks meet traditional pdge-preservation technology. Int. J. Comput. Vis., 131(10):2529-2555 2023.
[88] Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. Cddfuse: Correlation-driver dual-branch feature decomposition for multi-modality image fusion. In IEEE Conf. Comput. Vis. Pattern Recog., pages 5906-5916 5916. IEEE, 2023.
[89] Huafeng Li, Dayong Su, Qing Cai, and Yafei Zhang. Bsafusion: A bidirectional stepwise feature alignment network for unaligned medical image fusion. In AAAI, pages 4725-4733, 2025. 2025.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have clearly stated the contributions and scope in the abstract and introduc- tion. The claims made are consistent with both the theoretical discussion and experimental results presented in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
We have discussed the limitations of our work in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
15

---

The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: There new theoretical results in paper, which focuses mainly method design, benchmark establishment, and empirical evaluation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have detailed the experimental settings, including datasets, model configu- rations, and evaluation metrics, in the main paper and Appendix ensure reproducibility. Additionally, our code has been made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility also be provided via detailed
16

---

instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibilityy the case of closed-source models, may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have provided instructions for reproduction details in the main paper and Appendix. The dataset, code and pretrained models publicly available to ensure faithful reproduction of the results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
sealiiccguies///// httee//uuddee/eeeuiiiiiesee CooteSummissionool111
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.
ssinioliccyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: All necessary details, including data splits, hyperparameters, and optimizer settings, provided in Section 5.
17

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: do not report error bars, but provide extensive experimental results across multiple benchmark datasets to demonstrate consistency and robustness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have details this in the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp////errpp...cppp////ttttta
18

---

Answer: [Yes]
Justification: Our research does not involve sensitive data, human subjects, or potential ethical concerns, and conforms fully with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper basic study on video fusion and does not directly involve societal impacts to the best of our knowledge.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups). privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
We have not released any data or models that may have a risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
19

---

We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited all original papers that produced the code packages and datasets used in our work, and respected their licenses.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papprssii thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have publicly released all models, datasets, and pretrained weights, along with detailed documentation and example scripts to ensure reproducibility and ease of use.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects, no IRB approval is required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: We used LLM models only for writing and formatting assistance, which did not affect the core methodology or research outcomes.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should not be described.
21

---

A Further Details of Data Preparation in VF-Bench
A.1 Explanation of Key Terms in Multi- Multi-Exposure Video Fusion
BT.709: A widely adopted International Telecommunication Union Radiocommunncation Sector (TTU-R) standard for high-definiiion television (HDTV) video, specifying parameters such color primaries, transfer characteristics (OETF), and color space for 8-bit SDR (Standard Dynamic Range) video.
BT.2020: An ITU-R recommendation defining parameters for ulrra-high-dffiition television (UHITT)), including wider color gamut, higher bit-depth (typically 10-bit or 12-bit), and enhanced color reproduction capabilities compared to BT.709,, It is used for HDR (High Dynamic Range) video content.
Elletro-OOtiial Transfer Function (EOTF): A mathematical function that defines how digital signal values are converted into visible light by a display. It transforms non-linear, gamma-encoded video signals into linear light domain, accurately representing scene radiance. ВT.2020 (HLG encoding formar))
( EOTF is defined as:
S V 0.5
L
(10)
+0.28477
0.8788
0.5 < V S
where V the normalized video signal value and L is the corresponding linear luminance.
Opto-Electronic Transfer Function (OETF): The inverse of EOTF, this function defines how light captured by camera sensor converted into digital video signal values. It typically applies gamma curve to map linear scene radiance into non-linear encoding space suitable for storage and broadcast. The OETF for BT.709 is specified as:
4.5L
0 I. < 0.018
V
(1I)
[10099 0.099
0.018 L 1
where is the normalized linear light level and is the video signal value.
Linear Color Space: A color space where the numerical values of pixel intensities directly proportional to the physical light intensity in the real world. In this domain, exposure adjustments and radiometric operations can be performed accurately, opposed to gamma-encoded, non-linear spaces where such operations would introduce distortions.
A.2 Circle of Confusion Derivation and Approximation (Eq. (5)) in Multi-Focus Video Fusion
The blur level in optical imaging systems is characterized by the size of the Circle of Confusion (CoC), which determines the degree defocus blur for each pixel. larger CoC corresponds to stronger blur. Based on the thin lens equation:
1
(12)
D
7'
where v is the image distance, D is the object distance, and f is the focal length of the lens. The image distance for object depth D is given by:
fD
(13)
D D
Assuming focus distance Dy, the corresponding image distance is:
fD, Vy = D
(14)
As illustrated in Fig. the CoC pixel can then be computed as:
CoC'
(15)
HHybidd Log Gamma (HLG) is widely used high dynamic range (HDR) encoding format, employed in the YouTube-HHD HDR dataset (66).
22

---

Focal Point
Plane
in Focus
Figure Illustration of the optical geometry for Circle of Confusion.
where A is the aperture diameter.
Substituting the expressions for of and vy, and simplifying, obtain:
D D' Di(D, ク
Coc = Af
(16)
To facilitate practical implementation without requiring precise camera metadata, we approximate the CoC based estimated normalized inverse depth. Let d' 1/D and d, 1/Dj. Assuming < D', D (a valid assumption most video capture scenarios), and noting that the focus distance Dy remains fixed within specific video, we approximate Eq. (I6) as:
1 CoC' x DDD))/D d' dg
d.
(17)
Further simplifying yields:
1- df
CoC' x
(18)
dj
By introducing global blur strength factor to account for unknown camera parameters, the CoC for pixel is approximated by:
d'
Coc = d3
o.
(19)
dj
This approximation is justified as it preserves the relative CoC values across pixels, which critical for simulating realistic defocus blur patterns in the absence of explicit optical parameters. Further- more, since inverse depth typically correlates linearly with perceived defocus in monocular video sequences, this approximation remains perceptually valid. The scaling factor absorbs the unknown optical constants and ensures consistent blur strength across frames. We take o = 0.025, which is approximated by using common camera setup.
Finally, for frame with the longer edge length pixels, we calculate the Gaussian blur kernel size kernel' for each pixel from the calculated CoC values by:
kernel' = CoC I.
(20)
A.3 Selection Criteria for Infrared- Visiblee Video Fusion
A.3.1
Objective Assessment for Infrared Frames
To ensure the quality of infrared-visible video fusion, an objective assessment is performed on infrared frames prior to fusion. Three quantitative metrics - Image Entropy, Global Contrast, and Dark Area Proportion used to evaluate each frame. Frames that do not meet predefined thresholds are
23

---

discarded to exclude low-quality or uninformative content. In the following, we present more details of the metrics, computation methods, and thresholds.
Image Entropy. Image Entropy quantifies the information content and textural complexity of a grayscale image. Higher entropy indicates richer pixel intensity distribution, while lower values indicate less informative content. It is computed as:
253
H = Σ (()) p(),
(21)
where p(i) is the normalized histogram value of intensity i. The entropy of each infrared frame is calculated from its normalized histogram via Eq, (2I). Frames with > 6 are retained while those below this threshold are excluded due to insufficient information.
Global Contrast. Global Contrast is assessed by the standard deviation of pixel intensities, reflecting the overall contrast distribution in the image. A higher standard deviation indicates stronger contrast and clearer object boundaries, which essential highlight thermal patterns. It computed as:
1 N
Σ
24 1
(22)
where n; is the intensity of pixel j the mean intensity, and N the total number of pixels. Each frame's contrast is evaluated from its grayscale intensities. Frames with o > 30 are retained; those below are discarded due insufficient structural and thermal contrast.
Dark Area Proportion. Frames containing excessive dark regions often reflect poor capture con- ditions or insufficient thermal signals. To quantify this, we compute the Dark Area Proportion D, which represents the proportion of pixels whose intensity falls below a predefined threshold T 10:
1
D
Σ S(, N
(23)
where N is the total number of pixels in the frame and 1(-) is the indicator function:
1, if 1 ST
1(4 (T) = (o otherwise
(24)
Frames with a dark ratio under 5% are retained while those exceeding this value are discarded for lacking sufficient thermal content and visibility.
Overall Quality Scoring. After filtering based on each of the three individual metrics, we further perform comprehensive selection of scenes using a weighted normalized score that combines the three metrics:
H Hmax
σ
Score Wi
w2
w3 (1 D).
(25)
Tmax
Hmax and Tmax denote the maximum values of and a over the entire dataset, respectively. Weights W1, Wa, and W' satisfying Wi + Wy + W3 = 1. Equal weights (w) This composite score offers an intuitive measure for evaluating overall frame quality. Based on this score, we discard the bottom 10% of scenes.
1/3) are used here.
A.3.2 Exclusion of Ilgmiiilumnnntoonn RGB Frames
Retinex theory decouples a natural image (4) into an illumination component (L) and a reflectance component (R), mathematically expressed as:
- L R.
(26)
Here, represents the intensity of illumination incident upon the scene, which varies with lighting conditions. Conversely, R signifies the intrinsic reflectance properties of the scene, which remain invariant to changes in illumination. This Retinex decomposition allows for the effective extraction of the scene's 's illumination information, facilitating subsequent filtering processes. In our work, we leverage the model proposed in [77] extract the scene's illumination component. Following this, frames with illumination levels ranked in the top 25% based on mean (L) values are excluded.
24

---

Table 5: Quantitative evaluation results for the low-resolution MEF (540p) and MFF (480p) task. The red and
blue highlights indicate the highest and second-highest
VF-Bench Multi- Exposue F Fusion Branch (540p)
VF-Bench Multi-Focus Fusion Branch (480p)
VIFt
SSIM
Qabft
BiSWEL
VIFt
SSIM
MI1
Qubft
BiSWE,
MS2R.
CUNet
0.50
0.85
1.85
0.39
7.55
0.20
CUNet
0,53
0.36
3.52
0.68
10.23
0.42
DDMEF
0.71
0.95
2.96
0.66
8.99
0.71
IFCNN
0.68
0.87
4.85
0.73
9.37
0.38
HoLoCa
0.50
0.86
2.56
0.42
8.22
0.19
RFI.
0.77
0.90
6.31
0.78
8.46
0.28
CRMEF
0.62
0.94
2.60
0.63
8.72
0.19
EPT
0.76
0.90
6.33
0.78
8.50
0.29
TC-MoA
0.74
0.72
7.82
0.16
TC-MoA
0.75
5.27
8.39
0.28
FILM
0.77
0.99
4.35
0.72
8.28
0.17
FILM
0.75
0.89
5.06
0.78
8.61
0.33
ReFus
0.74
0.72
7.63
ReFus
0.73
0.32
Ours
0.79
0.99
4.38
0.73
6.96
0.16
Ours
0,77
0.90
6.34
0.79
8.29
0.27
B Visualizations of the Four Branches in VF-Bench
We present visualizations of some part of the video pairs from the four branches in VF-Bench as follows:
Dataset visualizations for Multi-exposure Video Fusion branch in VF-Bench are shown in Fig. 10
Dataset visualizations for Multi-focus Video Fusion branch in VF-Bench are shown in Fig. 11
Dataset visualizations for Infrared-Vistee Visible Video Fusion branch in VF-Bench are shown in Fig.
Dataset visualizations for Medical Video Fusion branch in VF-Bench are shown in Fig. 13
Our VF-Bench provides high-quality data in diverse scenes, serving as strong benchmark for future video fusion tasks.
C Quantitative Results on Low-Resolution MEF and MFF Branches
Considering that inference on full-resolution videos may not be suitable for small devices scenarios with limited computational resources, we additionally conduct experiments on low-resolution versions of the MEF (540p) and MFF (480p) datasets in Tab. S While the primary training and evaluation of both branches are performed on the 2Krresolution datasets in the main paper, the low-resolution results presented here serve complementary evidence to assess the consistency of performance across different input scales. The supplementary results further demonstrate that our model can consistently produce high-quality fused videos.
D Additional Qualitative Fusion Comparison Results
We present more fusion visualization results in the figures below and on the project homepage videos:
More qualitative comparisons for Multi-exposuuee Video Fusion results are shown Fig.
More qualitative comparisons for Multi-focus Video Fusion results shown in Fig. 15
More qualitative comparisons for Infrared- Visible Video Fusion results are shown in Fig. 16
More qualitative comparisons for Medical Video Fusion results are shown in Fig.
The visual comparisons support our observation and conclusions: our UniVF method effectively preserves fine details and texture from the source images, while comprehensively integrating informa- tion from different settings or modalities produce richly informative fused images. The videos further show that our results exhibit superior temporal consistency, with significantly less flickering and motion incoherence.
E
Limitations
We assume well alignment between modalities in the input videos, and we have accordingly filtered the videos VF-Bench. However, there are, although rarely, still a few frames (1 out of 300 frames) that not aligned. With this assumption and trained on our well-filtered data, the model may
25

---

produce artifacts or ghosting in the fused video when encountering misaligned frames, as shown in Fig. [8 In future work, we plan to incorporate alignment modules into our UniVF to improve the robustness of the fusion process against misaligned input frames while maintaining the model's efficiency and fusion quality.
26

---

Raui Ai Taatttal Tanth Phrihsall
Figure 10: Dataset visualizations for Multi-exposure Video Fusion branch VF-Bench. Columns 1-5 and 11-15 correspond ander-exposed video sequences, while columns 6-10 and 16-20 correspond their respective over-exposed video sequences.
27

---

Figure 11: Dataset visualizations for Multi-focus Video Fusion branch in VF-Bench. Odd rows correspond to the far-focus video sequences and even rows correspond to the respective near-focus video sequences.
28

---

Dataset visualizations for Infrared- Visible Video Fusion branch in VF-Bench. Columns 1-5 and 11-15 correspond to infrared video sequences, while columns 6-10 and 16-20 correspond to their respective visible vidco sequenees.
Figure 12:
29

---

Figure 13:
Dataset visualizations for Medical Video Fusion branch in VF-Bh Columns 1-10 correspond to
MRI video sequences, while columns 1 1-20 correspond to their respective CT, PET or SPECT video sequences.
30

---

Fit ReFusion
Under
Over
CUNet
DOMEF
HoLoCo
CRMEF
IC-MoA
FILM
Ours
Figure 14: Visualization comparison of the fusion results in the multi-exposure video fusion task.
Near
Far
CUNet
IFCNN
EPT
RFL
FILM
ReFusion
TC-MoA
Ours
Figure 15: Visualization comparison of the fusion results in the multi-focus video fusion task.
Infrared
Visible
CDDFuse
EMMA
TC-MoA
MRFS
FILM
ReFusion
Text-DiFuse
TDFusion
DCEvo
Ours
Figure 16: Visualization comparison of the fusion results in the infrared-visible video fusion task.
31

---

MRI
SPECT
CDDFuse
FMMA
FILM
ReFusion
Text-DiFuse
BSAFusion
C2RF
Ours
Figure Visualization comparison fusion results medical video fusion task.
SA 931
11A- 9.7%
9n
Infrared
Visible
Fused Result
Figure 18: Artifacts in fused video caused by misaligned input frames.
32


Reviews:
[{"review_id": "6mG7tPJFc8", "reviewer": "Reviewer_Wqk9", "summary": "The proposed method addresses the issue of flickering artifacts in video fusion, which arise from neglecting temporal correlations. The authors propose UniVF, a unified framework for video fusion that explicitly models spatial-temporal dependencies through multi-frame learning and optical flow-based feature warping to ensure temporally coherent outputs. To facilitate research and evaluation, the authors introduce VF-Bench, the first comprehensive benchmark for video fusion, covering four distinct tasks. It is accompanied by a unified evaluation protocol that jointly assesses spatial quality and temporal consistency. The authors report that UniVF achieves state-of-the-art performance across all tasks on this new benchmark. This work provides a much-needed benchmark and a strong baseline for future research in temporally coherent video fusion.", "questions": "The technical content is solid, and the paper is novel and well-organized. However, as noted in the weaknesses:\n1.Foundational concepts like warping are assumed known. \n2.It is unclear whether the two encoders in Fig. 2 share weights\n3.Temporal loss only considers adjacent frames without modeling longer dependencies. \n4.The rationale for choosing SEA-RAFT is lacking. \n5.It's unclear if optical flow is frozen or jointly trained.", "limitations": "yes", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1.VF-Bench Contribution.\nThe proposed VF-Bench is a valuable contribution, addressing the long-standing lack of high-quality, well-aligned, and diverse video pairs for fusion tasks. Its data synthesis strategy and unified evaluation protocol promote standardized and reproducible research.\n2.UniVF Framework Design.\nThe UniVF framework is well-motivated, employing optical flow-based feature warping to ensure temporal consistency. Combined with a temporal consistency loss, it presents a coherent and effective solution for video fusion.\nWeaknesses:\n1.Core operations like \"feature warping\" are mentioned but not thoroughly described. It would help readers if the paper explicitly stated how bilinear warping is used and applied.\n2.Figure 2 shows two encoders for the input streams. Are their weights shared or independent? This design choice could affect performance and needs clarification.\n3.The proposed temporal loss only uses immediate neighbors. How to balance video fusion performance with the size of the neighbors window?\n4.SEA-RAFT is used without justification. It's unclear whether this choice is optimal or arbitrary. An ablation comparing several flow methods (e.g., RAFT, Unimatch) would help isolate the contribution from the quality of the flow.\n5.A critical design decision-whether the flow network is frozen or trained end-to-end-is not discussed. Please clarify your choice.\nThese weaknesses need to be addressed with additional experiments and a deeper architectural discussion.", "was_revised": true, "final_justification": "I appreciate the author’s rebuttal, which has satisfactorily resolved my concerns. Accordingly, I have decided to increase my score.", "rebuttal": "We thank the reviewer for the encouraging comments on both VF-Bench `a valuable contribution` and the UniVF framework `well-motivated` and `coherent and effective solution for video fusion`.\n\n1. **Q1:** `Explanation of \"feature warping\"`  \n   **R1:** Feature warping in UniVF is implemented via bilinear sampling, following the common practice in optical flow-based temporal modeling. Specifically, for each input image, we estimate the bidirectional optical flows between adjacent frames, denoted as $\\\\{\\mathcal{O}\\_{t-1 \\rightarrow t}^{k}, \\mathcal{O}\\_{t+1 \\rightarrow t}^{k}\\\\}$. Each optical flow field represents the motion of pixels from one frame to another, where each flow vector indicates the displacement of a pixel to its corresponding location in the neighboring frame. These flow fields are then applied to the deep features $\\\\{\\Phi\\_{t-1}^{k},\\Phi\\_{t+1}^{k}\\\\}$ extracted by the encoder, enabling us to align them with the target frame $t$. The features are sampled at subpixel locations using differentiable bilinear interpolation to produce the warped representations $\\\\{\\widetilde{\\Phi}\\_{t-1 \\rightarrow t}^{k},\\widetilde{\\Phi}\\_{t+1 \\rightarrow t}^{k}\\\\}$, which are then fused for final prediction.\n\n2. **Q2:** `Are the encoder weights shared or independent?`  \n   **R2:** The two encoders processing the input video streams are independent. \n   * Theoretically, when we have inputs from different modalities, we need separate encoders to extract their respective private features. \n   * In practice, we demonstrate the effectiveness of using two shared encoders through ablation studies. Following ablation experiments results in Table 4 from the original paper, the IVF task results are shown in the table below.\n  \n      |               | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R |\n      | ------------- | ---- | ---- | ---- | ---- | ----- | ---- |\n      | Shared Encoder  | 0.43 | 0.64 | 2.43 | 0.65 | 3.94  | 0.36 |\n      | Independent Encoder (Ours) | **0.44** | **0.64** | **2.47** | **0.68** | **3.84**  | **0.35** |\n      \n      The ablation results validate the soundness of our algorithm design.\n   *  Notably, for video frames from the same input source, the same set of encoder parameters is used to encode each frame.\n3. **Q3:** `Window size chosen for Temporal loss`  \n   **R3:** Our current temporal loss leverages the two adjacent frames $(t-1,t,t+1)$ to enforce local temporal consistency. Here we present a comparison of the IVF task results when the temporal window size is set to 5:\n\n   |               | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R | Inference Time |\n   |--|------|------|------|------|-------|------|--|\n   | 5-frame      | **0.45** | 0.64 | 2.45 | 0.67 | **3.82**  | **0.35** | 0.44\n   | 3-frame (Ours)  | 0.44 | **0.64** | **2.47** | **0.68** | 3.84  | 0.35 | **0.21** \n\n   From the comparison results, we observed that while larger windows improve long-term temporal consistency, they also introduce alignment drift and increase inference latency. In other words, increasing the window size to 5 brought only marginal improvements in temporal metrics, but negatively affected spatial metrics and significantly increased the model’s inference time. Therefore, considering the trade-off between temporal consistency, spatial information, and inference efficiency, we decided to use a temporal window of 3 frames, i.e., the current frame as well as the previous and subsequent frames.\n4. **Q4:** `Selection of SEA-RAFT.`  \n   **R4:** SEA-RAFT was chosen for its efficiency and reasonable accuracy in optical flow estimation. Per request, we conducted ablation with an alternative flow estimator, Unimatch [1], to better isolate the impact of flow quality on overall performance. The comparative results are reported in the table below. While higher-quality flow leads to slight performance gains, UniVF consistently outperforms other fusion baselines across different flow estimators. This indicates that our framework is robust to the quality of the flow input and does not rely heavily on the accuracy of any specific flow method, confirming the value of our multi-frame fusion architecture.\n\n   |               | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R |\n   | ------------- | ---- | ---- | ---- | ---- | ----- | ---- |\n   | w/ Unimatch   | 0.43 | 0.64 | 2.45 | 0.66 | 4.09  | 0.37 |\n   | Ours          | **0.44** | **0.64** | **2.47** | **0.68** | **3.84**  | **0.35** |\n\t\n   [1] Unifying Flow, Stereo and Depth Estimation. IEEE TPAMI 2023.\n5. **Q5:** `Flow network: frozen or end-to-end?`  \n   **R5:** In our current implementation, the SEA-RAFT flow estimator is used as a frozen module (i.e., not updated during UniVF training). This decision is made to reduce training complexity and prevent degradation of flow quality due to fusion loss gradients, which are not directly aligned with flow supervision. We will also plan to experiment with fine-tuning the flow network jointly in future work."}, {"review_id": "jOjkk9SJT7", "reviewer": "Reviewer_BKiZ", "summary": "The authors propose UniVF, a unified framework for fusing video inputs from different modalities and settings. It achieves temporally coherent results via optical flow-based feature warping. Additionally, VF-Bench is introduced, a benchmark for evaluating video fusion methods in four categories. The proposed unified fusion loss guarantees the temporally coherent fusion results, and the proposed benchmark’s unified evaluation protocol jointly assesses spatial and temporal video quality. Extensive experiments show that UniVF achieves state-of-the-art results across all tasks on VF-Bench.", "questions": "1.The proposed temporal consistency is mainly supported by metric tables. A common method for visualizing temporal coherence is showing time-slice (strip) images—these would give an intuitive picture of flicker or ghosting and are currently missing.\n2.When constructing the multi-focus dataset, the blur intensity factor $\\sigma$ is a key parameter that determines the degree of blurring in the near-focus plane and far-focus plane. How is this parameter determined? Additionally, could the authors provide a more detailed explanation of how the near-focus plane and far-focus plane are defined and separated in the proposed data generation process?\n3.For multi-exposure fusion dataset, I still have some confusion regarding the data generation pipeline. For example, it is unclear why exposure adjustment needs to be performed in the linear light domain, and why a conversion from 10-bit BT.2020 color space to 8-bit BT.709 is necessary after exposure adjustment. The description in the paper is somewhat brief, and the motivations behind these steps should be explained more clearly. \n4.It would be helpful if the authors could clarify how the feature warping operation is specifically performed — is it based on a deterministic affine transformation or implemented as a learnable module? \n5.The IVF branch should include C2RF [1]—a recent strong fusion model with cross-modal alignment.\n6.Using \"image domain metrics\" vs. \"temporal domain metrics\" is potentially misleading since both are computed in the image domain. A better term might be \"spatial domain metrics\" to distinguish them clearly.\n[1] C2RF: Bridging Multi-modal Image Registration and Fusion via Commonality Mining and Contrastive Learning. IJCV 2025.", "limitations": "yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths: This is a very promising paper with a practical goal: designing a single model to solve various video fusion problems. The paper is ambitious and brings value via a large-scale benchmark and a clean video fusion framework, combined with newly proposed loss functions and evaluation protocols tailored for video fusion. \nWeaknesses: However, the verification of temporal consistency is insufficient—strip-based visualization would help. The sigma parameter in the synthetic multi-focus dataset is critical but underexplained. The motivations for the HDR data pipeline and color space conversion also lack clarity. Technical details of the warping process need to be specified. See the Questions section for more details.", "was_revised": true, "final_justification": "Based on authors responses, I have decided to raise my rating to 5. I hope the authors will incorporate the important revisions into the update version of the manuscript.", "rebuttal": "We sincerely thank the reviewer for the encouraging comments regarding the practicality, ambition, and contributions of our work.\n\n1. **Q1:** `Time-slice visualizations for temporal consistency`  \n   **R1:** We apologize for not being able to include images in the rebuttal, but we sincerely appreciate your suggestion. In the revised version, we will incorporate time-slice visualizations across all pixels of the fused video sequences, comparing UniVF with existing methods. These visualizations intuitively reveal differences in flickering and temporal smoothness. The results demonstrate that our method still achieves the best temporal consistency performance.\n2. **Q2:** `Blur strength parameter in multi-focus dataset`  \n   **R2:**\n   * The blur intensity factor $\\sigma=0.025$ is empirically selected to match realistic defocus levels based on common imaging optics and perceptual blur thresholds. We determined this via visual tuning on a validation subset to balance realism and diversity.\n   * To define the near-focus and far-focus planes, we first extract a video depth map using a monocular depth estimator. The far-focus plane is set to the 80th percentile of depth values, representing typical background regions. For the near-focus plane, we compute the average depth of each segmented object using DAVIS masks, then select the closest object and use its depth value.\n3. **Q3:** `Clarifying the HDR-based multi-exposure data pipeline`  \n   **R3:**\n   * Exposure adjustment in the linear light domain is critical because exposure is a multiplicative operation on scene radiance. Performing this in the gamma-compressed (non-linear) space would lead to physically incorrect results and color distortions. Linear light ensures radiometric fidelity.\n   * Conversion from 10-bit BT.2020 to 8-bit BT.709 is necessary because: BT.2020 is used in HDR formats and supports a wider gamut and dynamic range. However, most consumer displays and fusion models are trained in 8-bit SDR color space with BT.709 format. The down-conversion ensures compatibility and standardization while retaining realistic visual variations.\n   * We will revise the data generation section to clearly outline these motivations and justify the pipeline choices more rigorously.\n4. **Q4:** `Clarification of the feature warping operation`  \n   **R4:** \n   * The feature warping operation is deterministic and implemented using differentiable bilinear interpolation, not as a learnable module. Given optical flow from SEA-RAFT, we warp feature maps from adjacent frames to the current frame by sampling features at flow-displaced coordinates. This technique is widely used in temporal alignment and video restoration [1,2].\n   * Specifically, for each input image, we estimate the bidirectional optical flows between adjacent frames, denoted as $\\\\{\\mathcal{O}\\_{t-1 \\rightarrow t}^{k}, \\mathcal{O}\\_{t+1 \\rightarrow t}^{k}\\\\}$. Each optical flow field represents the motion of pixels from one frame to another, where each flow vector indicates the displacement of a pixel to its corresponding location in the neighboring frame. These flow fields are then applied to the deep features $\\\\{\\Phi\\_{t-1}^{k},\\Phi\\_{t+1}^{k}\\\\}$ extracted by the encoder, enabling us to align them with the target frame $t$. The features are sampled at subpixel locations using differentiable bilinear interpolation to produce the warped representations $\\\\{\\widetilde{\\Phi}\\_{t-1 \\rightarrow t}^{k},\\widetilde{\\Phi}\\_{t+1 \\rightarrow t}^{k}\\\\}$, which are then fused for final prediction.      \n   [1] BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment. CVPR 2022.  \n   [2] VRT: A Video Restoration Transformer. TIP 2024.\n5. **Q5:** `C2RF results in IVF task.`  \n   **R5:** We present additional IVF comparison results with C2RF in the table below. Our method continues to achieve state-of-the-art performance in video fusion.\n   |       | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R |\n   | ----- | ---- | ---- | ---- | ---- | ----- | ---- |\n   | C2RF  | 0.27 | 0.57 | 1.67 | 0.47 | 5.34  | 0.70 |\n   | Ours  | **0.44** | **0.64** | **2.47** | **0.68** | **3.84**  | **0.35** |\n6. **Q6:** `\"image domain metrics\" vs. \"temporal domain metrics\"`  \n   **R6:** We sincerely appreciate your suggestion. We will revise the manuscript to refer to them as \"spatial metrics\" and \"temporal consistency metrics\" for clearer distinction."}, {"review_id": "3bqEatpRZh", "reviewer": "Reviewer_QQ99", "summary": "This paper introduces UniVF, a unified video fusion framework that incorporates multi-frame learning and optical flow-based feature warping for temporally coherent video fusion. It is evaluated on a newly proposed VF-Bench, which contains datasets for four distinct fusion tasks. The authors also design a new temporal loss and evaluation metrics to support future research. Experiments show that UniVF outperforms a variety of baselines.", "questions": "Based on the Weaknesses, there are several areas that require clarification or further experiments. The ablation studies are somewhat shallow, and there are missing comparisons to several key baselines (e.g., DCINN). More detailed architectural descriptions would also help clarity. Additionally, the paper does not analyze the choice of optical flow estimator or its impact on results.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \nThe paper addresses an important video fusion problem and proposes a reasonably well-designed method with clear motivations. The architecture is sound, and the benchmark is timely. Loss and evaluation metrics are novel and reasonable. \n\nWeaknesses:\n1. Methods like DCINN [1] are absent from the infrared-visible (IVF) comparison. \n2. The framework relies heavily on SEA-RAFT for flow estimation, but the impact of this choice is not analyzed. An ablation replacing SEA-RAFT with other classical models would clarify whether the gains come from the model itself or the flow estimator.\n3. Eq. (7) introduces a validity mask M to select reliable flow regions. However, there is no explanation of how this mask is computed or applied in practice.\n4. The paper omits practical information like inference speed, model parameters, etc. Given the use of multi-frame inputs and transformer blocks, this is crucial for evaluating deployment feasibility.\n5. The framework operates on 3-frame snippets, but it is unclear how performance and speed change if more frames are used. This limits the understanding of temporal modeling capabilities beyond short-term dependencies.\n6. The model architecture (Fig. 2) is well-illustrated but lacks the notation mapping to equations. Adding symbols would help make the flow of information clearer to readers.\n\n[1] A general paradigm with detail-preserving conditional invertible network for image fusion. International Journal of Computer Vision (2024)", "was_revised": true, "final_justification": "My concerns have been addressed in the discussion above. I have decided to maintain a positive opinion of the paper.", "rebuttal": "Thank you for acknowledging our paper's idea and contribution. We are very encouraged by the comment for `a reasonably well-designed method with clear motivations`, `architecture is sound`, `benchmark is timely`, and `loss and evaluation metrics are novel and reasonable`.\n\n1. **Q1:** `DCINN results in IVF task.`  \n**R1:** We present additional comparison results with DCINN in the table below. Our method continues to achieve state-of-the-art performance in video fusion quality.\n   |       | VIF      | SSIM     | MI       | Qabf     | BiSWE    | MS2R     |\n   | ----- | -------- | -------- | -------- | -------- | -------- | -------- |\n   | DCINN | 0.38     | 0.63     | 1.95     | 0.58     | 4.23     | 0.37     |\n   | Ours  | **0.44** | **0.64** | **2.47** | **0.68** | **3.84** | **0.35** |\n1. **Q2:** `Reliance on SEA-RAFT and ablation with other optical flow methods`  \n   **R2:** \n   * We selected SEA-RAFT for its SOTA power in flow estimation and its strong balance of accuracy and efficiency. To further isolate the contribution of UniVF from the flow estimator, we added an ablation study replacing SEA-RAFT with Unimatch [1] in the table in **R4** of **Response for Reviewer Wqk9**.  \n   * While higher-quality flow leads to slight performance gains, UniVF consistently outperforms other fusion baselines across different flow estimators. This indicates that our framework is robust to the quality of the flow input and does not rely heavily on the accuracy of any specific flow method, thus confirming the value of our multi-frame fusion architecture.  \n   [1] Unifying Flow, Stereo and Depth Estimation. IEEE TPAMI 2023.\n\n2. **Q3:** `Explanation for validity mask M in Eq. (7).`  \n   **R3:** To improve the robustness of the temporal consistency loss and avoid penalizing unreliable regions, we employ occlusion masks based on forward-backward optical flow consistency. Specifically, given the forward flow $ \\mathcal{O}\\_{t \\rightarrow t+1} $ and the backward flow $ \\mathcal{O}\\_{t+1 \\rightarrow t} $, we first warp the backward flow to the coordinate system of the current frame $t$:\n   $$\\widehat{\\mathcal{O}}\\_{t+1 \\rightarrow t}(p) = \\mathcal{W}\\left( \\mathcal{O}\\_{t+1 \\rightarrow t}, \\mathcal{O}\\_{t \\rightarrow t+1}(p) \\right).$$ \nHere, the warp operator $\\mathcal{W}$ uses the forward flow $\\mathcal{O}\\_{t+1 \\rightarrow t}$ to look up the corresponding backward flow vector from frame $t+1$. In essence, for a pixel $p$ in frame $t$, we find where it lands in frame $t+1$ and retrieve the backward flow vector from that new location.\nThe forward-backward consistency error is then defined as the magnitude of the “round-trip” error vector:\n   $$\n   \\Delta(p) = \\left\\| \\mathcal{O}\\_{t \\rightarrow t+1}(p) + \\widehat{\\mathcal{O}}\\_{t+1 \\rightarrow t}(p) \\right\\|\\_2.\n   $$\nA large value of $ \\Delta(p)$ indicates a significant inconsistency, which typically occurs when the pixel $p$ is occluded in frame $t+1$ or the flow estimation is unreliable. Therefore, we can identify these unreliable points by thresholding this error. We define the binary occlusion mask $M(p) \\in \\\\{0,1\\\\} $ as:\n$$M(p) = 1 \\text{ if } \\Delta(p) < \\epsilon, \\text{ otherwise } 0$$\n   where $ \\epsilon $ is a predefined threshold (1.0 in our paper). This mask is applied to both previous and next frames (i.e., $ M_{\\text{prev}} $, $ M_{\\text{next}} $) to restrict the temporal consistency loss (Eq. 7) to only well-aligned and non-occluded regions.\n\n3. **Q4:** `Inference speed and model size.`  \n**R4:** Per request, here we compare UniVF with 17 SOTA methods in four video fusion tasks in terms of the number of parameters, inference time, and FLOPs, and present the results in the table below. All experiments were conducted in a uniform environment. Note that although UniVF provides multi-frame learning for temporally coherent video fusion, it is near the median in terms of time, complexity and parameter number among the competitors, achieving excellent fusion results without excessive computational resources.\n\n   |          | Params (M) | FLOPs (G) | Time (s) |\n   |---------:|-----------:|----------:|---------:|\n   |  CDDFuse |      1.19  |   547.74  |    0.18  |\n   |     EMMA |      1.52  |    41.54  |    0.02  |\n   | ReFusion |      0.06  |    18.21  |    0.03  |\n   |     BSAF |      9.69  |    62.96  |    0.03  |\n   |     FILM |      0.49  |   180.00  |    0.08  |\n   |      TDF |      0.06  |    18.21  |    0.02  |\n   |      CUN |      0.64  |   195.88  |    0.08  |\n   |    DDMEF |     20.57  |   226.28  |    0.01  |\n   |   HOLOCO |     17.69  |   183.93  |    0.10  |\n   |    IFCNN |      0.08  |    40.05  |    0.01  |\n   |      EPT |     15.86  |   423.16  |    3.19  |\n   |    DCEvo |      2.00  |  1825.33  |    0.25  |\n   |  TextDIF |    119.46  | 17891.10  |   15.31  |\n   |    CRMEF |      1.88  |   471.92  |    0.04  |\n   |     MRFS |    134.97  |   139.19  |    0.05  |\n   |    TCMOA |    340.35  |  8388.47  |    0.68  |\n   |      RFL |      1.90  |  2330.47  |    0.08  |\n   |    UniVF (Ours) |      9.15  |   825.20  |    0.21  |\n1. **Q5:** `More frames are used.`  \n   **R5:** We added experiments evaluating \"5-frame input\" configurations in the table in **R3** of **Response for Reviewer Wqk9**. \n   From the comparison results, we observed that while larger windows improve long-term temporal consistency, they also introduce alignment drift and increase inference latency. In other words, increasing the window size to 5 brought only marginal improvements in temporal metrics, but negatively affected spatial metrics and significantly increased the model’s inference time. Therefore, considering the trade-off between temporal consistency, spatial information, and inference efficiency, we decided to use a temporal window of 3 frames, i.e., the current frame as well as the previous and subsequent frames.\n\n2. **Q6:** `Notation mapping to equations`  \n   **R6:** We appreciate this suggestion. To improve clarity, we will revise Fig. 2 to explicitly annotate components with corresponding symbols from the equations in the revision of our paper."}, {"review_id": "KPNDU6HPx4", "reviewer": "Reviewer_LVB7", "summary": "This paper proposes UniVF, a unified framework for video fusion that incorporates multi-frame learning and optical flow-based feature warping to ensure temporal coherence and informative fusion. Alongside the method, the authors introduce VF-Bench, a comprehensive benchmark consisting of four video fusion tasks: multi-exposure, multi-focus, infrared-visible, and medical video fusion. The benchmark includes careful construction or curation of well-aligned video pairs and establishes a joint evaluation protocol combining spatial quality and temporal consistency metrics. Extensive empirical studies show UniVF achieves strong quantitative and qualitative performance across all four tasks, and ablation studies support the design components.", "questions": "I hope the author will address the relevant issues raised in the Weaknesses in the rebuttal.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n1. VF-Bench covers a diverse set of video fusion scenarios with carefully constructed or curated and well-aligned video pairs.\n2. UniVF leverages multi-frame encoding/decoding and incorporates state-of-the-art optical flow for temporal alignment, with the additional use of temporal consistency loss supplementing the objective for smooth fusion.\n3. The paper adds two temporal consistency metrics, measuring intra-video smoothness and flow coherence, giving a more holistic evaluation of the fusion problem.\n4. The main technical details for model, dataset, loss, and evaluation metrics are sufficiently described. UniVF consistently outperforms state-of-the-art fusion methods, with results contextualized in the paper.\n\n## Weaknesses\n\n1. Although the paper emphasizes the benefit of video fusion, most of the visual results shown are still static image comparisons. It would be beneficial to see a comparison of the fused videos to demonstrate consistency over time.\n2. The physical or perceptual meaning of MS2R remains vague. Could the authors explain it in more intuitive terms?\n3. The transformer backbone uses four Restormer blocks, but there is no sensitivity analysis showing whether performance saturates or improves with more/less blocks. \n4. In UniVF, after calculating the optical flow on adjacent images, the corresponding features are warped. I suggest adding ablation experiments to see the effect of warping directly on the image and then feeding it into the encoder.\n5. There is no report on inference time, model size, or real-time capability. Since the framework includes optical flow and transformer operations, its computational burden needs to be discussed.\n6. RCVS [1] are mentioned in related work but missing in task evaluations. As a baseline for RGBT video stream fusion, additional it in comparison would provide a more convincing benchmark.\n\n*[1] Xie, Housheng, et al. \"RCVS: A Unified Registration and Fusion Framework for Video Streams.\" IEEE Transactions on Multimedia 2024*", "was_revised": true, "final_justification": "The rebuttal effectively addressed my concerns, and I have decided to maintain my accept score", "rebuttal": "We sincerely thank the reviewer for the positive evaluation and appreciation of our VF-Bench benchmark, UniVF framework, and the proposed evaluation metrics. Below, we respond to each of the concerns raised in detail.\n\n1. **Q1:** `Fused video comparisons are missing`  \n   **R1:** Please kindly refer to the supplementary material, where we provide fused video comparisons to more thoroughly demonstrate the effectiveness of our method.\n2. **Q2:** `A more intuitive explanation for MS2R`  \n   **R2:** \n   * MS2R evaluates how smoothly objects move over time by measuring acceleration consistency. \n   * It compares the second-order motion (flow difference) of the fused video with those from the two reference input videos.\n   * A lower MS2R implies more natural, consistent motion, which correlates with fewer flickering or jittering artifacts.\n   We will include a brief explanation in the revised manuscript to make this clearer.\n3. **Q3:** `Analysis on the number of Restormer blocks`  \n   **R3:** We included a sensitivity analysis varying the number of Restormer blocks (e.g., 1, 2, ..., 7) to show how depth affects fusion performance in the table below. 4 blocks offered a good trade-off between quality and inference efficiency. Increasing the depth gave diminishing returns and higher latency.\n\n   | Num      | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R |\n   | -------- | ---- | ---- | ---- | ---- | ----- | ---- |\n   | 1        | 0.35 | 0.58 | 2.01 | 0.55 | 4.39  | 0.39 |\n   | 2        | 0.40 | 0.62 | 2.29 | 0.63 | 4.11  | 0.36 |\n   | 3        | 0.44 | 0.64 | 2.46 | 0.67 | 3.87  | 0.37 |\n   | 4 (Ours) | 0.44 | 0.64 | 2.47 | 0.68 | 3.84  | 0.35 |\n   | 5        | 0.44 | 0.64 | 2.47 | 0.68 | 3.85  | 0.35 |\n   | 6        | 0.45 | 0.64 | 2.48 | 0.68 | 3.84  | 0.34 |\n   | 7        | 0.45 | 0.65 | 2.49 | 0.69 | 3.85  | 0.35 |\n4. **Q4:** `Ablation: Warping features vs. warping images`  \n   **R4:** \n   * Theoretically, our design choice to warp features (rather than input images) allows alignment at a semantically rich representation level, helping the model better deal with illumination or modality shifts. \n   * Nonetheless, we perform the suggested ablation: warping input frames directly and feeding them into the encoder *vs.* our default feature warping to clarify where alignment is most effective. \n  \n      |               | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R |\n      | ------------- | ---- | ---- | ---- | ---- | ----- | ---- |\n      | Warping images | 0.41 | 0.63 | 2.45 | 0.67 | 3.97  | 0.36 |\n      | Warping features (Ours)    | **0.44** | **0.64** | **2.47** | **0.68** | **3.84**  | **0.35** |\n\n      The ablation results validate the validity of our algorithm design.\n\n5. **Q5:** `inference time, model size, or real-time feasibility`  \n   **R5:** Per request, here we compare UniVF with 17 SOTA methods in four video fusion tasks in terms of the number of parameters, inference time, and FLOPs, and present the results in the table **R4** of **Response for Reviewer QQ99**. All experiments were conducted in a uniform environment. Note that although UniVF provides multi-frame learning for temporally coherent video fusion, UniVF is near the median in terms of time, complexity and parameter number among the competitors, achieving excellent fusion results without excessive computational resources.\n6. **Q6:** `RCVS [1] missing in comparisons`  \n   **R6:** We present additional comparison results with RCVS in the table below. Our method continues to achieve state-of-the-art performance in video fusion quality.\n   |       | VIF  | SSIM | MI   | Qabf | BiSWE | MS2R |\n   | ----- | ---- | ---- | ---- | ---- | ----- | ---- |\n   | RCVS  | 0.26 | 0.51 | 1.28 | 0.33 | 4.71  | 0.37 |\n   | Ours  | **0.44** | **0.64** | **2.47** | **0.68** | **3.84**  | **0.35** |"}]

Author rebuttal:

Meta-review: This paper introduces a unified video fusion framework using multi-frame learning and optical flow-based warping and VF-Bench, which is the first comprehensive benchmark for four fusion tasks. All four reviewers strongly support acceptance, with two raising scores after the rebuttal. 

Key strengths include the valuable benchmark contribution addressing a critical research gap, a technically sound framework achieving state-of-the-art results, and novel temporal consistency metrics. Initial concerns regarding technical details, missing baselines, and computational complexity were thoroughly resolved through comprehensive experiments and clarifications provided in the rebuttal. 

As one reviewer summarized: "This is a very promising paper with a practical goal: designing a single model to solve various video fusion problems. The paper is ambitious and brings value via a large-scale benchmark and a clean video fusion framework, combined with newly proposed loss functions and evaluation protocols tailored for video fusion." 

The AC and SAC therefore recommend accepting the paper.
Decision: Accept (spotlight)