Instant Video Models: Universal Adapters for Stabilizing Image-Based Networks
Authors: Matthew Dutson
Nathan Labiosa
Yin Li
Mohit Gupta
Venue: NeurIPS 2025 poster ()
Keywords: computer vision
temporal consistency
robustness
video
denoising
image enhancement
depth estimation
semantic segmentation
Primary area: applications
TL;DR: We propose a general approach for modifying image-based models to improve robustness and temporal consistency on video.

Abstract: When applied sequentially to video, frame-based networks often exhibit temporal inconsistency—for example, outputs that flicker between frames. This problem is amplified when the network inputs contain time-varying corruptions. In this work, we introduce a general approach for adapting frame-based models for stable and robust inference on video. We describe a class of stability adapters that can be inserted into virtually any architecture and a resource-efficient training process that can be performed with a frozen base network. We introduce a unified conceptual framework for describing temporal stability and corruption robustness, centered on a proposed accuracy-stability-robustness loss. By analyzing the theoretical properties of this loss, we identify the conditions where it produces well-behaved stabilizer training. Our experiments validate our approach on several vision tasks including denoising (NAFNet), image enhancement (HDRNet), monocular depth (Depth Anything v2), and semantic segmentation (DeepLabv3+). Our method improves temporal stability and robustness against a range of image corruptions (including compression artifacts, noise, and adverse weather), while preserving or improving the quality of predictions.

Instant Video Models: Universal Adapters for Stabilizing Image-Baseds Networks
Matthew Dutson, Nathan Labiosa, Yin Li, and Mohit Gupta
University of W
ndutsonn mgg.tta37)0vvssc....
Abstract
When applied sequentially to video, Trame-based networks often exhibit temporal inconsistency- for example, outputs that flicker between frames. This problem is amplified when the network inputs contain time-varying corruptions. In this work, we introduce general approach for adapting frame-based models for stable and robust inference on video. We describe a class of stability adapters that can be inserted into virtually any architecture and a resource-efficient training process that can be performed with a frozen base network. We introduce unified conceptual framework for describing temporal stability and corruption robustness, centered sttpacc--staabiltt-bbbbsee
loss. By analyzing the theoretical properties of this loss, we identify the conditions where it produces well-behaved stabilizer training. Our experiments validate approach several vision tasks including denoising (NAFNet), image enhancement (HDRNet), monocular depth (Depth Anything v2), and semantic segmentation (DeepLabv3+). Our method improves temporal stability and robustness against a range of image corruptions (including compression artifacts, noise, and adverse weather), while preserving improving the quality of predictions.
proposed
Introduction
Video is often processed
images are passed one hy one to processing pipeline model, and each output is independent of the previous one. This design choice is often driven by practical considerations: single-frame datasets are generally more diverse and accessible than video datasets, training image-based models is far less demanding in terms of compute and memory, and improvements in single-frame performance often carry over to video-based tasks.
Unfortunately, frame-wise processing faces the inherent challenge of temporal consistency, where model predictions fluctuate over time (Figure [ Totp- (to-middle). This behavior is especially problematic in tasks such as denoising or stylization, where temporal inconsistency ceptual quality. Even in applications where the model output is not intended for human consumption, instability can impact downstream tasks. For example, inconsistent monocular depth estimates could produce erratic behavior collision avoidance system. Moreover, instability can reduce perceived reliability and thereby undermine user trust, regardless of objective performance.
significantly reduce per-
Temporal consistency closely related to corruption robustness. In field deployments, vision systems often operate in non-ideal conditions. For example, an autonomous vehicle may encounter inclement weather, sensor artifacts, or low-light noise; robustness in these circumstances is vital for safe system operation. These real-world corruptions frames (Figure [ top-right). While it may be challenging to correct such degradations with a single frame, often infer the underlying clean signal from recent context. In this case, can view robustness as a natural extension of temporal consistency.
often transient, with appearance that changes between
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Frame wise prediction
Unstable
2
Independennt
22
yi2
Denoising (NAFNet)
Monocular depth corruptions
Injecting stabilizers
Versatility
Training
(ggn, 31)
X8 (g) g22)
Denoising
6( y22
Depth
Segmentation
Stability (input unstabilized stabilized)
Snow
Rain
Elastic deformation
Impuise noise
Denoising (NAFNet)
Denoising (NAFNet)
Enhancement (IIDRNet)
Enhancement (HDDNNe))
Figure 1: Stabilizing image-based networks. (top) Applying single-image models sequentially to the frames of a video can cause unstable predictions and failures under time-varying corruptions. In the top-right example, we see that randomly dropping patches causes artifacts in monocular depth estimates. (middle) We propose a method for injecting stabilizers into existing networks and for training these stabilizers using a unified seuuracy-tabilltyyrouuu improvements in stability and robustness for various tasks, without modifying the original image-
loss. (bottom) We demonstrate
based models. Image sources: [18 56 58

---

Several prior works have proposed video-centric models with improved temporal consistency [5] 44 24
60 72.. However, these methods are often narrowly designed for one or few tasks and require costly training on large-scale video datasets. Consequently, they lack the flexibility to leverage the extensive ecosystem of frame-based imaging and perception models. Further, few explicitly address robustness to transient corruptions or other challenging conditions.
74
In this work, we improve the temporal consistency and robustness of pre-trained, image-based models across various tasks. One of our primary challenges is that increasing stability may result in over-smoothing, which can, in turn, reduce accuracy. We conceptually explore the tradeoffs between output quality, corruption robustness, and temporal consistency, and introduce unified accuracy- cesouusttessstablliii loss to balance these objectives. We provide a theoretical analysis of this loss and identify strategies to avoid "over-smoothing reality," such that there is no incentive for predictions to be smoother than the true scene dynamics.
Guided by this analysis, we propose a class of versatile stabilization adapters (Figure | middle). These adapters generate control signals, based on recent spatiotemporal context, that modulate changes to the model's features and output. By operating in both the feature and output spaces, we allow the adapters to model stability wherever it exists in the visual hierarchy. This property is important for high-level vision tasks, where stability is often best described in a feature space.
Our method offers several key benefits. First, our stabilization adapters are lightweight and modular, and do not require modifying the original model parameters. Second, our adapters operate causally; stabilized outputs depend only on current and past inputs-a feature that is critical for processing streaming video in latency-sensitive applications. Third, our approach is compatible with both low- level tasks, where stability can be described in terms of pixel values, and higher-level tasks, where stability occurs at the level of scene semantics. Finally, our method naturally enhances robustness to transient corruptions, without requiring explicit corruption modeling.
We evaluate our method on a range of tasks: denoising, image enhancement, monocular depth estimation, and semantic segmentation (Figure miiddleeiiim We also demonstrate improved ro- bustness against various transient corruptions, including noise, dropped patches, elastic deformations, compression artifacts, and adverse weather (Figure bottom)) In most cases, these improvements do not reduce accuracy-on the contrary. we often see significant improvements in task metrics. Overall, our experiments establish the flexibility and practicality of our approach.
2 Related Work
Corruption robustness. Several prior works have addressed robustness against input corruptions. Hendrycks and Dietterich (23 propose metrics for measuring the robustness of image classifiers against common corruptions (e.g., compression artifacts or weather); their metrics inspire our definitions in Section 3 In general, natural corruptions have received less attention [4]] from the 59
vision community than adversarial corruptions [ 19 46 48
61 69 71], although handful of
methods and benchmarks exist
70). Like these works, our paper emphasizes robustness to naturally occurring corruptions rather than worst-case adversarial perturbations.
Consistent image enhancement. Frame-to-frane flickering is a significant problem for low-level image enhancement models; as such, there have been several works that improve temporal consistency for these tasks [5] 33
37 77 [8]. Blind video temporal consistency methods [5] 33 37 treat the frame-level model as a black box, which allows generalization across models and applications. However, because they consider only the model input and output, these methods cannot model higher-level (semantic) stability and are prone to instability when the input is impacted by transient noise corruptions. In contrast, we model stability in both the output space and the feature space, improving robustness against corrupted inputs and supporting a wider range of tasks, including those where stability is best described in semantic terms.
Task-specific video architectures. Many works propose video-optimized architectures for specific tasks, with temporal consistency stated priority in many cases. This problem has been widely studied for ill-posed, low-level tasks where the output is intended for a human viewer; examples include video colorization 36
44
64
76
79
82
83
stylization 7 10 13 16 17 21
24
39
53
37
67 73,, and inpainting
6
30
35
65
Temporal consistency is also a concern in higher-level
tasks, including segmentation
2
5
55
60
66), object detection 4 26 42 63
72 84 and depth
3

---

81] Clockwork ConvNets [60] leverage the observation that semantic content evolves more slowly and smoothly than pixel values; we share this motivation in designing feature-domain stabilizers.
estimation [27 28
32
38
45
50
68
Our goal is not design task-specific method: in fact, expect specialized architectures to outperform our general approach on benchmarks. The appeal of our approach lies in its practicality and versatility. Our method requires minimal training and alterations the original network, and can be applied to a broad range of video inference tasks, including those where highly optimized video architectures may not exist.
3 Defining Stability and Robustness
We start by defining temporal stability. Let fo : - 3 be frame-wise predictor with parameters ø. and 6(y/, 1/2) be a metric defined on the space V. We use ý to indicate model output and y for the target output (ground truth, or features from reference model). We formulate our definition as an expectation over data distribution D containing (æ, y) sequences of duration r indexed by discrete time step (the frame index). We define the stability S as the negative expected difference between adjacent predictions, i.c.,
S = -
Σ (as)
(I)
The negation is added such that stability increases as frame-to-frane variation decreases.
This notion of stability closely related to robustness, i.c., the correctness of the model predictions under input corruptions [23] The same input corruptions can cause both temporal instability and reduced prediction accuracy; examples include sensor noise, image or video compression artifacts, rain, and snow. Hendrycks and Dietterich [23) define corruption robustness R. the expected accuracy of a classifier under a distribution & of per-image perturbation functions. We extend their definition cover arbitrary metrics ô and time series of duration T,
Re -BE Σ 5 (fo(E((2))), v1)
(2)
where e, is the fer-frame perturbation at time t. Again, is the target output. We define the corruption stability Se similarly:
S. = -B=
fole
Both R. and S. include input perturbations e. R. measures how accurately model predicts the target, while S. captures the temporal smoothness of the model's outputs. Notably, Rg and S. can be applied to both the intermediate features and the output of f.
4 Learning to Balance Stability and Robustness
We now combine R. and S. to form unified sccuracy-ttability-ohhie analyze the conditions under which this loss leads to well-hehaved training. training loss Me and
Unified eecuracy-stabilltyyoobbe
loss. We define the unified loss Ue as
Uc --RR + ASS)
(4)
Σ 8(fa(E(()))). + (tfaeela)))
(5)
where À is constant that weights stability relative accuracy.
Theoretical analysis. If can be expressed in terms of norm on V, we can derive two bounds on The first, X < 1/2, which we call the oracle bound, defines the range of À where the ground truth is

---

No stahility penalty, À 0
Below oracle hound, A= 0.4
Above collapse bound, À
Value of thard prediction in
50-
in
Bo
ye
Value of second prediction 9x
Value of second prediction
Value of second peodiction 83
Figure 2: Unified loss for one-dimensional predictions. We consider time series of duration 3 consisting one-dimensional predictions, with å defined as the distance. We assume that the first prediction ý1 is fixed (cannot be modified by stabilization adapter). We show the value of the second prediction 1/2 along the x-axis and the value of the third ja along the y-axis, with contours indicating the value of the unified loss as these predictions vary. When À = 0, the minimum occurs at the nonzero but below the oracle bound, the minimum still occurs at the ground truth, but the loss increases more slowly in the direction of stabler predictions.
ground truth 02 = 8/2 and tu Y/. When
When À exceeds the collapse bound, the global minimum is the collapse state ÿ: = f/2 y1.
the global minimizer of the loss in prediction space. Under this bound, a perfectly accurate (oracle) model will never have incentive diverge from the correct prediction to increase stability. For each training item æ, the minimum loss occurs at the ground truth implying zero gradients with respect the prediction ý. See Appendix A for proof of this and the following bound.
The second bound, > 5 1, is the collapse bound, and gives the range of À where the global loss minimizer corresponds to exact repetition of the initial prediction, regardless of scene changes. Unlike the global accuracy minimizer (the oracle state), which may difficult or impossible to reach with gradient descent, the collapse state is often easily achieved. For example, if there is an EMA stabilizer (Section 5) on the output, we can achieve collapse simply by setting its decay to zero. In experiments, confirm that setting À above the collapse bound leads to prediction collapse, provided the collapse state representable in the stabilizer parameter space.
Example. In Figure 2 plot the loss function of two one-dimensional predictions ÿ2 and ÿs. for several values of A. When the stability penalty is introduced, the loss begins to tilt toward more stable predictions. When À is within the oracle bound, the global minimizer is unchanged. When it exceeds the collapse bound, the global minimizer is repeated prediction.
Note that the oracle state is mutually exclusive with the collapse state (unless the ground truth is itself collapsed), as for any time series with 1, we have 1 > 0.5. Thus, we recommend training with x < 0.5 in general; doing ensures non-collapse and yields the correct behavior in the limiting case where the model is perfectly accurate.
Designing Stabilization Adapters
Our goal is to improve the temporal stability and robustness of a pre-trained, frame- predictor for video tasks. We assume if is realized using a deep neural network with pre-trained weights do- While the unified loss U4 allows to update Φo, this fine-tuning may be computationally expensive or require substantial training data. Instead, we consider adaptation of f. where a task-specific, lightweight adapter parametrized by Aø is learned stabilize the intermediate features and outputs of f. Under this formulation, we train only the parameters Aø of the adapter (i.e., the stabilizer). and the original weights do remain fixed.
Design principles. The following principles guide our stabilizer design. First, we consider only causal stabilizers, where the stabilized outputs at time L are computed exclusively using information from times & This constraint critical for processing streaming videos. Second, we stabilize both network activations (features) and outputs. Output-only stabilization is sufficient for some low-level
5

---

21,4
Stabilizer
22, (
Stabilizer
y/+
Fuse
Fuse
-1,A-1
22,
Legend
21,t-1
22,1-1
h2
Existing layer
Controller layer
Stabilization
9
Interpolation
Figure 3: Stabilization controllers. Starting with the existing network (red), add stabilizers (yellow) to select layers. The degree of stabilization, i.e., the decay 8, can be predicted by stabilization controller (blue). This controller consists of a shared backbone g and one head h, per stabilized layer. Stabilizers can be added to both internal layers and the model output.
operations such colorization. Feature-domain stabilization expands the potential scope of our method to include higher-level tasks; for these tasks, the inherent stability of scene is often best described feature space. Finally, we limit ourselves to designs that do not interfere with the existing network architecture. Our stabilizers are layer-level adapters with independent parameters and are designed to preserve the existing feature representation.
Exponential moving average (EMA) stabilizer. As starting point, we consider a simple temporal smoothing operation applied to individual feature values forming a one-dimensional time series. Let 2 denote an activation or feature value at time f, and let & denote the corresponding stabilized activation. The exponential moving average (EMA) stabilizer produces a linear combination of the current unstabilized output and the previous stabilized output,
|2 + (1 111-31 -)
(6)
where 3 E [0, 1] is decay-rate parameter. The recursive formulation here equivalent convolving the input time series with an infinite exponentially decaying weight kernel. The EMA stabilizer is memory-efficient (only 2,-1 retained) and differentiable with respect to 3 and z. We use the EMA stabilizer the basis for more sophisticated designs.
Stabilization controllers. Simple smoothing operations (like the EMA stabilizer) limited in both their spatial context and their ability to model complex changes in the scene. To address these limitations, we propose stabilization controller network, which considers prior context (e.g., the current and previous input frames and feature maps) and predicts the amount of stabilization that should be applied to each value.
Although the idea controller-augmentte stabilization can be applied to many stabilization mecha- nisms, focus here the EMA stabilizer due to its differentiability and low memory requirements. In this case, the controller predicts the decay 3 for each activation across layers. Our architecture, shown in Figure 3, consists of shared backbone I and stabilization head h; per stabilized layer. Tog compares the current and previous frames, offering shorteut connection from frames to features. h predicts the decay values based on the output of g (resized match the layer resolution). the current unstabilized features žt, and the previous stabilized and unstabilized features ă and 21-1- Together, the parameters of g and { form Aó.
Formally, the stabilized feature tensor 2i, for layer and time is given by
بt, = Bi, ④ 2i, + (1 -B,e) ž Bi (h,(gg xt 1), Zi,e, - zi,-1)),
(7)
(8)
where c denotes an element-wise product and zi,t is the unstabilized feature tensor.
We note that our controller is conceptually similar to selective state space models 20 although the design differs significantly. Equation can be viewed linear dynamical system defined on frame-level features with parameters conditioned on the input, current, and previous features.

---

Moderate intensity (o 0.5)
High intensity (a 0.25)
33
28
Base model
32
Outpat fixed
27-
Simple faxed
PSNR
31-
Simple learned
26
Controlled
30-
Spatial
25-
29-
28
20
30
25
35
Instability
lustability
Figure 4: Image enhancement results. Introducing a controller and spatial fusion the stabilizer significantly improves the accuracy-stability tradeoff. The spatial-fusion stabilizer reduces frame-to- frame variation by up to 35% while exceeding the quality of the base model. "Instability" here refers to negative stability (-S); see Equation [ The goal is to move toward -r (lower instability) and ty (better image quality).
Controller with spatial fusion. Often, x takes the form of 2D feature map. If and h are convolutional, the predicted decay is informed by the frames and features within a local receptive field. However, with Equation [ the weighted fusion used to compute & is still constrained to the time axis. Extending the weighted fusion spatial neighborhood can improve stabilization in the presence of motion by allowing translation of features from previous frames.
To perform spatial fusion, we modify the controller head h to predict spatial decay kernel v each pixel rather than a single decay 3. For neighborhood that contains m locations (including the central pixel), the kemel 77 contains m + 1 elements. The first m elements weight the stabilized activations from the previous time step (1.- for each location in the neighborhood. The (m + )t element weights the current unstabilized activation (z₂) for the central pixel. The kernel is cettmax-normalized The first m logits are predicted directly by the controller head, and the last is set to zero (when m 1, the softmax reduces to the sigmoid in Equation &p. See Section D.3 for more details.
The spatial fusion stabilizer can represent a recursive shift projection, where a feature vector is translated on each frame by an amount corresponding to the object motion. The maximum trackable motion in this case is determined by the spatial extent of the kernel r7.
6 Experiments
Sections 6.1 and 6.2 we test our approach on image enhancement and denoising, respectively, We ablate the components of our method and explore the tradeoff between stability and accuracy. In Section 6.3 we test our stabilizers in the presence of various image corruptions, evaluating image enhancement, denoising, and depth estimation. In Section 6.4 we consider corruptions resulting from adverse weather (rain and snow).
Some low-level details (e.g., training hyperparameters) are omitted here for brevity: see the appendices for a more exhaustive description of experiment protocols. The appendices also include results for semantic segmentation and exploration of training-free stabilizer composition.
Variants and baselines. Across our experiments, consider common of variations (ablations of our approach) and baselines. The simple learned variant adds simple EMA stabilizer (Equation 6 the output and features, with one learned value per channel. The controlled variant adds learned controller that predicts the stabilizer decay according to Equation g Finally, the spatial variation augments the controlled stabilizer by adding spatial fusion. As for baselines: the simple fixed method applies a simple EMA stabilizer to internal features and the output, with one global, hand-tuned (3; the output fixed method applies hand-tuned EMA stabilizer only the model output.
6.1 Image Enhancement
Task, dataset, and base model. We first consider image enhancement, task where perceptual quality (including stability) is primary importance. We use the HDRNet model [18), which can be trained to reproduce many low-level image transformations, Specifically, target the local
7

---

Moderate naise (o 0.1)
Strong noise (o 0.2)
Extreme nrise (o (r-00.0 0.8)
37.
34.5
29.0-
Base model
37.0
34.0-
28.5
Output fixed
Simple fixed
PSNR
36.5
28.0-
Simple leamne
Controlled
36.0
33.0
27.5
Spatial
35.5
27.0-
32.5
35.0
26.5
15
20
20
30
Instabulity
Instability
Instability
Figure 5: Denoising results. Because it attempts to stabilize an iid noise residual, naive feature-space stabilization leads to worse PSNR and worse stability. We achieve the best performance with controlled stabilizer, usually with spatial fusion.
Laplacian detail-enhancement operator 49 due to its known forward model and readily available code. We consider two effect strengths: moderate (a 0.4 and 0.5) and strong (a 0.4 and a= 0.25). We generate training pairs by applying the local Laplacian filter to each frame of the Need for Speed (NFS) dataset [29]. NFS contains 100 videos (380k frames) collected at 240 FPS; we randomly select 20 videos for validation and use the remaining 80 for training. Videos are scaled to have a short-edge length of 360. We evaluate PSNR and instability (Equation with 1|-
Experiment protocol. We fine-tune the original HDRNet local_ wtt.laappaaaaatttroooooooo st24 1024 weights for both effect strengths. After this fine-tuning, we attach a stabilizer the output of each convolution and the overall model output (because the HDRNet architecture is extremely lightweight, this does not represent an unreasonable overhead). We then freeze the fine-tuned weights and train the stabilizers using BPTT on short video snippets (r = 8). We use the unified loss with 5 || 1l2. We test several values 0.1, 0.2, 0.4, 0.8, and 8.0-for for each effect strength and model variation. The first three values are within the oracle bound, and the last exceeds the collapse bound.
Figure 4 illlustrasss how PSNR and instability change as vary the degree of stabilization (A for learned stabilizers, 3 for hand-tuned stabilizers). We observe that for static (non-controlled) stabilizers, there is no benefit to stabilizing in feature space. For the static methods (output fixed, simple fixed, and simple learned), increasing stability brings reduction in quality as we begin to over-smooth the output. In contrast, for the controlled and spatial stabilizers, there region where both PSNR and stability are improved over the base model. Spatial fusion gives significant improvement in output quality- roughly dB for the high-intensity effect. We suspect this is related to the nature of the detail-enhancement task (its sensitivity to small-scale motion). Finally, we confirm that setting X = 8 prediction). This result confirms that the global collapse minimum is easily reached, despite the non-convex nature of the optimization general.
Results.
leads to prediction collapse (instability 10-3, indicating a constant
6.2 Denoising
Task, dataset, and base model. Next, we evaluate image denoising under AWGN. Denoising highlights the utility of our method when the input is itself unstable. We use the NAFNet model ()) which employs a U-Net architecture with modified convolutional blocks (NAFBlocks). We again the NFS dataset, with the same train/validation split in Section 6. We evaluate three noise levels: moderate (a = 0.1, for float images E [0, 1),, strong (a = 0.2), and extreme (a = 0.6). See
Appendix F.4 for additional results on the DAVIS 52 dataset.
Experiment protocol. We start by fine-tuning the unstabilized model for each dataset and noise level, initializing with the nafnet_ sidd_ fiidd__iwoo dth32 weights published by the model authors. We then attach a stabilizer to the output of each NAFBlock and to the model output. As before, we freeze the fine-tuned weights and train only the stabilizer parameters, using the unified loss with d || ||a and sweeping out À 0.1, 0.2, 0.4, 0.8, and 8.0.
Results. Figure 5 shows PSNR and instability across noise levels. The "simple fixed" stabilizer gives a somewhat surprising result: adding stabilization worsens both PSNR and instability. The reason is that the network backbone predicts a noise residual, which is completely uncorrelated
8

---

Enhamcement
Demoising
Depth estimation
Corruption
Mothod
PSNR
Instability
PSNR
Instability
AbsRel (1)
Delta-1.25 (?)
Instability
drup
model
164.6
18.93
151.4
0.948
9.89
Ours
31.39
30.36
35.46
20.42
0.070
0.956
4.73
Elastic distortion
Base model
24.00
64.23
27.99
47.31
0.052
0.968
7.76
Ours
26.63
24.97
30.78
21.04
0.057
0.967
4.75
Frame drop
Base model
28.65
94.04
33.42
113.5
0.065
0.936
14.17
Ours
31.69
29.14
27.34
20.56
0.050
0.974
4.91
JPEG artifacts
Base model
24.85
42.06
29.01
39.71
0.057
0.964
7.32
32.19
20.49
0.961
4.92
Impolse noise
Base model
14.28
217.8
24.65
62.10
0.047
0.974
6.83
Ours
27.37
30.90
32.04
19.73
0.056
0.972
4.98
Table I: Corruption robustness. Our method significantly improves stability while preserving or improving prediction quality. The only notable exception is reduced PSNR for denoising with dropped frames. However, we note that PSNR (averaged over frames) does not capture the jarring perceptual effect of dropped frames; thus, the high PSNR of the base model is somewhat deceptive.
between frames. Over-smoothing this residual inhibits the noise removal, which worsens PSNR and increases frame-to-frn fo-frame variation due to unremoved noise. Fortunately, this behavior does not exist for the learned or controlled stabilizers, as they target only those features that exhibit some temporal smoothness. Controlled stabilizers improve both stability and PSNR when À S 0.4. We again confirm that setting À = 8 > F leads to prediction collapse, i.e,, instability < 10-3
In most cases, we find that the spatial fusion stabilizer outperforms the non-spatial controlled stabilizer. However, there a notable exception in the case extreme noise, where the spatial fusion stabilizer is about 6 dB worse than other methods (these points are outside the range of the plot in Figure S but are included in Table 7.. Intriguingly, this quality gap only appears when evaluating long sequences (hundreds of frames) and shrinks as we reduce In Appendix F.6
we show that this problem can be
at least partially mitigated by increasing r during training.
The supplementary material includes video files comparing the stabilized and unstabilized outputs. We encourage the reader to watch these videos for clear qualitative comparison (temporal inconsistency is much more obvious in videos than in side-by-side static frames).
6.3 Corruption Robustness
Tasks, datasets, and base models. We again consider HDRNet for image enhancement and NAFNet for denoising. We also include results for depth estimation with Depth Anything v2 (75). Depth Anything is notable for the scale of data used in its training: it would be costly to develop new video architecture from scratch, making Depth Anything a good candidate for our approach. See Appendix F22 for results on semantic segmentation with DeepLabv3+.
For depth training, videos with ground-truth depth. The dataset consists of 50 indoor scenes containing ego motion and is rendered 50 FPS. We randomly select 10 scenes for validation and use the rest for training. See Appendix for further details on this dataset.
the VisionSim framework (55 (Blender) generate dataset of simulated
Experiment protocol. For HDR Net and NAFNet, we train spatial-fusion stabilizers using the same settings as in Sections 6.1 and 6.2. We train HDRNet for the moderate effect strength (a = 0.5), and NAFNet for moderate noise (a 0.1) on NFS. Unlike other models, we do not fine-tune the base Depth Anything model; we found that naive fine-tuning on a small dataset like ours quickly led to overfitting, We add controlled stabilizers to instances of weaghhnnthhnngRasssmbb.e... eLayee oon.aturrreuiiiooerrrr....ooe and the model output.
We train and evaluate stabilized models for each of the following corruptions: (1) randomly zeroing
each 8x8 patch with probability 0.1, (2) elastic deformation, see Appendix E for details, (3) randomly
zeroing each frame with probability 0.1, (4) applying JPEG compression at quality 10/100, and (5) adding impulse noise to each channel with probability 0.05 for both salt and pepper. We set À = 0.2 when training stabilizers for corruption robustness.

---

Rain
Snow
Stabilized?
Unfrozen?
PSNR
SSIM
Instability
PSNR
SSIM
Instability
151.76
0.577
262.48
28.63
0.880
57.88
31.14
0.914
59.31
32.19
0.937
70.84
0.950
66.57
32.61
0.958
58.30
35.20
0.956
58.98
Table 2:
Adverse weather robustness on RobustSPRING. Training stabilizers with a frozen base model gives substantial improvement over the unstabilized model. We obtain the best overall results by jointly training stabilizers with the base parameters.
Results. Table shows accuracy and stability with and without stabilizers. Adding stabilizers leads to significant reductions in instability and, in most cases, improvements in per- per-fraam accuracy metrics. See Appendix Figures [I and for qualitative results.
6.4 Adverse Weather Robustness
Task, dataset, and base model. We now evaluate robustness under adverse weather conditions. Specifically, we consider the rain and snow corruptions from the RobustSpring [58] dataset. These corruptions differ from those in Section 6.3 in their spatial complexity and temporal dynamics (raindrops and snowflakes follow continuous paths, unlike simpler corruptions such as randomly- dropped patches). RobustSpring contains 10 rendered sequences (2000 total frames), each with left- and fight-frame variants. Clean ground-trute frames are provided for all sequences. We randomly select 2 videos for validation and use the remaining 8 for training. Videos are downsized to 720p. We evaluate NAFNet denoising
[ with a = 0.1, adding noise after weather effects.
Experiment protocol. We fine-tune the original nafnet_siinn _idd_width32 width32 weights with noise, but not weather corruptions. Following Section 6.2] we then append a spatial-fusion stabilizer to each NAFBlock and the model output. We train these stabilizers under noise weather using À = 0.2. Consistent with our other experiments, we train stabilizers with frozen base model.
In addition, we consider two variants with an unfrozen base model. In the first variant, we fine-tune the base model on noise weather without stabilizers. In the other, we unfreeze the base model when training on noise + weather, jointly training the stabilizer and base parameters. For both variants, we use the same hyperparameters as the frozen stabilizer training.
Results. See Table 2 and Appendix Figure 14 for results. Compared to the unstabilized baseline, stabilizers substantially improve image quality and stability. Likewise, fine-tuning the original weights (without stabilizers) with weather corruptions gives a significant improvement. Compared to stabilization, fine-tuning gives better single-image quality but higher instability. We obtain the best overall results by combining fine-tuning with stabilization. In general, we expect this joint training approach be the best choice for small- medium-sized models. For larger models where training the base model is infeasible, training only the stabilizers still gives reasonable results.
7 Discussion
Limitations. The bounds Section assume that the distance can expressed in terms of norm on the prediction space ) This condition excludes many widely used loss functions, especially more sophisticated, multi-component losses. Nonconforming may still work in practice, although they may require more careful tuning of due to the lack of theoretical guarantees.
We had some difficulty with sim-to-real generalization when training stabilizers for Depth Anything. We conjecture that real video contains subtle corruptions not present in simulated video- for example, sensor noise, compression artifacts, or optical phenomena. These "baseline corruptions" could explain some of the temporal instability we see when using apparently clean input video.
Alternate metrics. In all of our experiments, metrics may be a better choice, depending on the task. For example, a variant of the Wasserstein metric may give improved results for woo-dimensional outputs and feature maps, due to its ability to account for the spatial structure of the tensor. See Appendix B for further discussion.
simple Fuclidean norm for 8. However, other
10

---

Acknowledgments and Disclosure of Funding
Thanks to Sacha Jungerman for helping us generate VisionSim sequences for the depth estimation experiments. This research was supported by NSF CAREER award 1943149, NSF CPS grant 2333491, ARL under contract number 1911NF-202021,, and the Wisconsin Alumni Research Foundation via Research Forward Initiative.
References
[1] Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. Advances in adversarial attacks and defenses in computer vision: A survey. IEEE Access, 9:155161-155196 2021.
[2] Aharon Azulay, Tavi Halperin, Orestis Vantzos, Nadav Bornstein, and Ofir Bibi. Temporally stable video segmentation without video annotations. In Proceedings of the Winter Conference on Applications of Computer Vision (WACV), pages 3449-3458, Jamuary 20122.
[3] Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. Revisiting batch normalization for improving comuption robustness. In Proceedings of the Winter Conference on Applications of Computer Vision (WACV), pages 494-503, January 2021.
[4] Gedas Bertasius, Lorenzo Torresani, and Jianbo Shi. Object detection in video with spatiotemporal sampling networks. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Nicolas Bonneel, James Tompkin, Kalyan Sunkavalli, Deqing Sun, Sylvain Paris, and Hanspeter Pfister. Blind video temporal consistency. Transactions on Graphics (TOG), 34(6), 2015.
[6] Ya-Liang Chang, Zhe Yu Liu, Kuan-Ying Ying Lee, and Winston Hsu. Learnable gated temporal shift module for deep video inpainting. arXiv, 2019.
[7] Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, and Gang Hua. Coherent online vidco style transfer. In Proceedings of the International Conference Computer Vision (ICCV), October 2017.
[8] Liang-Chies Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv, 2017.
[9] Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. In Proceedings of the European Conference Computer Vision (ECCV), pages 17 17-33, 2022.
[10] Tian Qi Chen and Mark Schmidt. Fast patch-based style transfer of arbitrary style. arXiv, 2016.
[11] Xinghao Chen, Yiman Zhang, Yunhe Wang, Han Shu, Chunjing Xu, and Chang Xu. Optical flow distillation: Towards efficient and stable video style transfer. In Proceedings of the European Conference on Computer Vision (ECCV), pages 614-630, 2020.
[12] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benen- son, Uwe Franke, Stefan Roth, and Bemt Schiele. The Cityscapes dataset for semantic urban scene understanding. In Proceedings the Conference Computer Vision and Pattern Recognition (CVPR), 2016.
[13] Yingying Deng, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, and Changsheng Xu. Arbitrary video style transfer via multi-channel correlation. In Proceedings the AAAI Conference Artificial Intelligence (AAAI), pages 1210-1217, May 2021.
[14] Nathan Drenkow, Numair Sani, Ilya Shpitser, and Mathias Unberath. systematic review of robustness in deep learning for computer vision: Mind the gap? arXiv, 2022.
[15] Gongtan Fang. Pretrained DeepLabv3 and DeepLabv3+ for Pascal VOC and Cityscapes. https:// Cectuu..com//aanF/ httma//aann/eeeeaaaaaaaaaaaaaara 2022.
[16] Chang Gao, Derun Gu, Fangjun Zhang, and Yizhou Yu. ReCoNet: Real-time coherent video style transfer network. In Proceedings the Asian Conference on Computer Vision (ACCV), pages 637-653, 2019.
[17] Wei Gao, Yijun Li, Yihang Yin, and Ming- -suan Yang. Fast video multi-style transfer. In Proceedings of the Winter Conference on Applications of Computer Vision (WACV). March 2020.
[18] Michaël Gharbi, Jiawen Chen, Jonathan T. Barron, Samuel W. Hasinoff, and Frédo Durand. Deep bilateral learning for real-tine time image enhancement. Transactions Graphics (TOG), 36(4), July 2017.
11

---

[19] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. arXiv, 2015.
[20] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling, 2024.
[21] Agrim Gupta, Justin Johason, Alexandre Alahi, and Li Fei-Fei. Characterizing and improving stability in neural style transfer. Proceedings of the International Conference on Computer Vision (ICCV), October 2017.
[22] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
[23] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
[24] Haozhi Huang, Hao Wang, Wenhan Luo, Lin Ma, Wenhao Jiang. Xiaolong Zhu, Zhifeng Li, and Wei Liu. Real-time neural style transfer for vidcos. In Proceedings of the Conference on Computer Vision and Puttern Recognition (CVPR). July 2017.
[25] Sacha Jungerman. VisionSim: A modular and extensible framework distributed simulations with rich pixel- sxel-peefect ground truth annotations and realistic sensor emulation. http:///iiionsii..raattteeeeee 2025. Accessed 2025-05-15.
[26] Kai Kang, Hongsheng Li, Tong Xiao, Wanli Ouyang, Junjie Yan, Xihui Liu, and Xiaogang Wang. Object detection in videos with tubelet proposal networks. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
[27] Kevin Karsch, Ce Liu, and Sing Bing Kang. Depth transfer: Depth extraction from video using non- parametric sampling. Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 36(1 1):2144- 2158, 2014.
[28] Bingxin Ke, Dominik Narnhofer. Shengyu Huang, Lei Ke, Torben Peters, Katerina Fragkiadaki, Anton Obukhov, and Konrad Schindler. Video depth without video models. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pages
June 2025.
[29] Hamed Kiani Galoogahi, Ashton Fagg, Chen Huang, Deva Ramanan, and Simon Lucey. Need for Speed: A benchmark for higher frame rate object tracking, In Proceedings of the International Conference on Computer Vision (ICCV), October 2017.
[30] Dahun Kim, Sanghyun Woo, Joon- Young Lee, and In So Kweon. Recurrent temporal aggregation framework for deep video inpainting. Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 42(5): 1038-102,, 2020.
[31] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv, 2017.
[32] Johannes Kopf. Xuejian Rong, and Jia-Bin Huang. Robust consistent video depth estimation. Pro- ceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pages 161 1-1621, June 2021.
[33] Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, and Ming- Ming-Hsuan Yang. Learning blind video temporal consistency. In Proceedings of the Earopean Conference on Computer Vision (ECCV), September 2018
[34] Alfred Laugros, Alice Caplier, and Matthieu Ospici. Are adversarial robustness and common perturbation robustness independant attributes? In Proceedings of the International Conference on Computer Vision (ICCV) Workshops. October 2019.
[35] Sungho Lee, Scoung Wug Oh, DaeYeun Won, and Scon Joo Kim. Copy-and-pant paste networks for deep video inpainting. In Proceedings of the International Conference on Computer Vision (ICCV), October 2019.
[36] Chenyang Lei and Qifeng Chen. Fully automatic video colorization with regularizaziinnnn and diversity. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[37] Chenyang Lei, Yazhou Xing, and Qifeng Chen. Blind video temporal consistency via deep video prior. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), volume 33, pages 1083-1095, 2020.
12

---

[38] Siyuan Li, Yue Luo, Ye Zhu, Xun Zhao, Yu Li, and Ying Shan. Enforcing temporal consistency in video depth estimation. In Proceedings of the International Conference on Computer Vision (ICCV) Workshops. pages 1145-1154, October 2021.
[39] Xucting Li, Sifei Liu, Jan Kautz, and Ming-Hsuun Yang. Learning linear transformations for fast image and video style transfer. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[40] Jingyun Liang, Jiczhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. VRT: A vidco restoration transformer. Transactions on Image Processing (TIP), 33:2171 2182, 2024.
[41] Chang Liu, Yinpeng Dong. Wenzhao Xiang, Xiao Yang, Hang Su, Jun Zhu, Yuefeng Chen, Yuan He, Hui Xue, and Shibao Zheng. A comprehensive study robustness of image classification models: Benchmarking and rethinking. International Journal of Computer Vision (IJCV), 133:567-589, 2025.
[42] Mason Liu and Menglong Zhu, Mobile video object detection with temporally aware feature maps. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[43] Songhua Liu, Tianwei Lin, Dongliang He, Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, and Errui Ding. AdaAuN: Revisit attention mechanism in arbitrary neural style transfer. In Proceedings of the International Conference on Computer Vision (ICCV), pages 6649-6658, October 2021.
[44] Yihao Liu, Hengyuan Zhao, Kelvin C. K. Chan, Xintao Wang. Chen Change Loy. Yu Qiao, and Chao Dong. Temporally consistent video colorization with deep feature propagation and self-regularization learning. Computational Visual Media, 10:375-395, 2024.
[45] Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. Transactions on Graphics (TOG), 39(4), August 2020.
[46] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proceedings of the International Conference on Learning Represemtations (ICLR), 2018.
[47] Norman Mu and Justin Gilmer. MNIST-CE C: A robustness benchmark for computer vision. arXiv, 2019.
[48] Tianyu Pang, Kun Xu, Yinpeng Dong, Chao Du, Ning Chen, and Jun Zhu. Rethinking softmax cross- entropy loss for adversarial robustness. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
[49] Sylvain Paris, Samuel W. Hasinoff, and Jan Kautz. Local Laplacian filters: edge-aware image processing with Laplacian pyramid. In SIGGRAPH, 2011.
[50] Vaishakh Patil, Wouter Van Gansheke, Dengxin Dai, and Luc Van Gool. Don't Ion't forget the past: Recurrent depth estimation from monocular video. Robotics and Automation Letters, 5(4):6813-6620, 2020.
[51] Federico Perazzi, Anna Khoreva, Rodrigo Benenson, Bernt Schiele, and Alexander Sorkine Horiinne-- Learning video object segmentation from static images. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
[52] Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Lue Van Gool, Markus Gross, and Alexander Sorkine-Homunng. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the Conference Computer Vision and Puttern Recognition (CVPR), June 2016.
[53] Gilles Pay and Patrick Perez. A flexible convolutional solver for fast style transfers. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[54] René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(3):1623 1637, 2022.
[55] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr,, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. SAM 2: Segment anything in images and videos. Xiv, 2024.
[56] Stephan Richter, Zeeshan Hayder, and Vladlen Koltun. Playing for benchmarks. In Proceedings of the International Conference Computer Vision (ICCV), pages 2232-2241, October 2017.
13

---

[57] Manuel Ruder, Alexey Dosovitskiy, and Thomas Brox. Artistic style transfer for videos. In Proceedings of the German Conference on Pattern Recognition, pages 26-36, 2016.
[58] Jenny Schmalfuss, Victor Oei, Lukas Mehl, Madlen Bartsch, Shashank Agnihotri, Margret Keuper, and Andrés Bruhn. RobustSpring: Benchmarking robustness to image corruptions for optical flow, scene flow and steren, arXiv, 2025.
[59] Sanchari Sen, Balaraman Ravindran, and Anand Raghunathan. EMPIR: Ensembles mixed precision deep networks for increased robustness against adversarial attacks. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
[60] Evan Shelhamer. Kate Rakelly, Judy Hoffiman, and Trevor Darrell. Clockwork Convnets for video semantic segmentation. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pages 852-868, 2016.
[61] Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv, 2014.
[62] Matias Tassano, Julie Delon, and Thomas Veit. FastDVDnet: Towards real-time deep video denoising without flow estimation. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[63] Subarna Tripathi, Zachary C. Lipton, Serge Belongie, and Truong Nguyen. Context matters: Refining object detection in video with recurrent neural networks. arXiv, 2016.
[64] Ziyu Wan. Bo Zhang. Dongdong Chen, and Jing Liao. Bringing old films back to life. In Proceedings of the Conference Computer Vision and Puttern Recognition (CVPR). pages 17694-17703, June 2022.
[65] Chuan Wang, Haibin Huang, Xiaoguang Han, and Jue Wang. Video inpainting by jointly learning temporal structure and spatial details. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 5232-5239,, July 2019.
[66] Hao Wang, Weining Wang, and Jing Liu. Temporal memory attention for video semantic segmentation. In Proceedings of the International Conference on Image Processing (ICIP), pages 2254-2258, 2021.
[67] Wenjing Wang. Shuai Yang, Jizheng Xu, and Jiaying Liu. Consistent video style transfer via relaxation and regularization. Transactions on Image Processing (TIP), 29:9125-9139, 2020.
[68] Yiran Wang, Min Shi, Jiaqi Li, Zihao Huang, Zhiguo Cao, Jianming Zhang, Ke Xian, and Guosheng Lin. Neural video depth stabilizer. In Proceedings of the International Conference Computer Vision (ICCV), pages 9466-9476, October 2023.
[69] Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma. and Quanquan Gu. Improving adversarial robustness requires revisiting misclassified examples. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
[70] Eric Wong and J. Zico Kolter. Learning perturbation sets for robust machine learning. arvi,, 2020.
[71] Chang Xiao, Peilin Zhong, and Changxi Zheng. Enhancing adversarial defense by wwinners-take-at In Proceedings of the International Conference on Learning Representations (ICLR). 2020.
[72] Fanyi Xiao and Yong Jac Lee. Vidco object detection with an aligned spatial-temportt memory, In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
[73] Kai Xu, Longyin Wen, Guorong Li, Honggang Qi, Liefeng Bo, and Qingming Huang. Learning self- supervised space-time enn for fast video style transfer. Transactions on Image Processing (TIP), 30:2501- 25,2, 2021.
[74] Rui Xu, Xiaoxiao Li, Bolei Zhou, and Chen Change Loy. Deep flow-guide video inpainting. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[75] Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Anything V2. In Proceedings of the Conference on Neural Information Processing Systems (NeurlPS), volume 37, pages 21875-21911, 2024.
[76] Yixin Yang, Jinshan Pan, Zhongzheng Peng, Xiaoyu Du, Zhulin Tao, and Jinhui Tang. BiSTNet: Semantic image prior guided bidirectional temporal feature fusion for deep exemplar based video colorization. Transactions Pattern Analysis and Machine Intelligence (TPAMI), 46(8):5612-666 5624, 2024.
based
14

---

[77] Chun-Han Yao, Chia-Yang Chang, and Shao-Y Yi Chien. Occlusion-awwrre video temporal consistency. In Proceedings of the International Conference Multimedia, pages 777-785, 2017.
[78] Yanhong Zeng. Jianlong Fu, and Hongyang Chao. Learning joint spatial-temporal transformations for video impainting. In Proceedings of the European Conference Computer Vision (ECCV), pages 528-543. 2020.
[79] Bo Zhang, Mingming He, Jing Liao, Pedro V. Sander, Lu Yuan, Amine Bermak, and Dong Chen. Deep exemplar-base based video colorization. In Proceedings of the Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
[80] Fan Zhang, Shaodi You, and Ying Fu. Learning temporal consistency for low light video enhance- ment from single images. In Proceedings of the Conference on Computer Vision and Puttern Recognition (CVPR), pages 4967-4976, June 2021.
[81] Haokui Zhang, Chunhua Shen, Ying Li, Yuanzhouhan Cao. Yu Liu, and Youliang Yan. Exploiting temporal consistency for real-time video depth estimation. In Proceedings of the International Conference on Computer Vision (ICCV), October 2019.
[82] Yuzhi Zhao, Lai-Man Po, Kangcheng Liu, Xuchui Wang, Wing- Yin Yu, Pengfei Xian, Yujia Zhang, and Mengyang Liu. SVCNet: Scribble-based video colorization network with temporal aggregation. Transactions on Image Processing (TIP). 32:4443 4458, 2023.
[83] Yuzhi Zhao, Lai-Man Po, Wing- Yin Yu, Yasar Abbas Ur Rehman, Mengyang Liu, Yujia Zhang, and Weifeng Ou, VCGAN: Video colorization with hybrid generative adversarial network. Transactions on Multimedia, 25:3017-3053 --3222, 2023.
[84] Xizhou Zhu, Yujie Wang. Jifeng Dai, Lu Yuan, and Yichen Wei. Flow-guided feature aggregation for video object detection, In Proceedings of the International Conference on Computer Vision (ICCV), October 2017.
15

---

A Proofs
This section contains proofs of the oracle and collapse bounds from Section
A.1 Oracle Bound
We assume that å takes the form ò(a, b) = 5(a b) where is norm on R. We define
u( v) Σ 8(0e, v1) +
8(g,. ic+1)
(9)
Σ <(u - yi) + Σ S(ii - D(+1)
(10)
as the bracketed expression in Equation using the shorthand ÿ. = f(E(()))) Our objective is to show that for any 1 # y. there exists some y" + ÿ such that u(y y) < u(ū. y)-
There are two possible cases: (1) U. # yt for at least one of the endpoints, meaning ÿ1 # yı or
ÿr * yr: and (2) ÿ: = y, at both endpoints.
We start with the first case. We assume that ÿ1 + yı, with the proof being symmetric for ÿ. + yr. We propose yt where y| y1 and y = - for t > 1. Starting with the definition of u,
" y) =C(yn - yn) + >C(y (2) +...
(11)
AC(111 (2) +...
(12)
u Ú y) C(ύ) - v1) A( 8/2) +...
(13)
By the triangle inequality and absolute homogeneity of C.
u(y' y) = AC(1/1 - (22) +
(14)
AC((11 91) (g) (22) +
(15)
S AC - 31) + AC(zi - (2)) + +...
(16)
AC(21 41) + AC 92) +
(17)
Assume that x < 1. Then,
u(y S AC(i) yi) + AC (/2) +
(18)
((1) - vi) + A( 92) +
(19)
=u(ū y)
(20)
Therefore, u(y', y) < u(l. y).
To summarize, we have shown that if A < 1, any ÿ where 9 # cannot minimize u. The same is true for g. Hyr- yr-
Now we consider the second case, where ý 7 y but their endpoints are the same. this case, we must have ÿ. # y, for some < 8 < T. We propose y = y: and y! Dr for o 8. Starting again with the definition of u.
u( uly' v)
Xc(rr- 1 yr) + ((y) y.) + Xc(() ÿr+1) +...
(21)
( v) = ..... AC - yr) + AÇ(U. - 9(+1) +...
(22)
AC( úi) + C(): y1) + -C(0r- û1t1)
(23)
By the triangle inequality and absolute homogeneity of 6.
uly'. y) = .... + X((2-1 yt)) + XC 1(+1) +...
(24)
_... AC((Er= - -9t) + (ūt vt)) + XC((1r 91) + (ū. Dt+1)) +...
(25)
5..+ AC( - -9.) AC(r. y.) + +XC(ye ÿ.) + XC(x. D +...
(26)
-... AÇ( -(.) + +22((U - v.) + AÇ(T. D.+1) +...
(27)
16

---

Assume that A < 1/2. Then,
u(y'. 1) S ... Ac út) + 2AÇ(0: ye) + >Ç( -(+1)) t...
(28)
< 2... + Ac 9i) C(úe - yt) AC(s (1+1)) +...
(29)
=u(ù, y)
(30)
Therefore, u(y',3) y) < u(u y).
We have now shown that if À 1/2, any y where y. * ye for 1 < cannot minimize u. Combining this with the result from the first case, we have shown that for X < 1/2, ú = is the unique global minimizer of u.
A.2 Collapse Bound
To prove the collapse bound, first prove the convexity of We then show that, if ÿ1 is fixed
(cannot be modified by the stabilizer), D. = d1 for 1 < s S is a local minimizer of u for X > T 1. Because is convex, this makes y. i the global minimizer.
Define q as the concatenation (ú. v). i.e., the vector input to u. Consider two such inputs q and q² satisfies the triangle inequality;
u(q + q¹) = Σ
((): - y: + vl -v!) - x Σ Clu: D1+1 + D! - vl.1)
(31)
t=1
==1
<
((ú) - vi) + Σ
C(ú: 11+1) + Σ clo - -vt) + +
<(U
0441) (32)
t=1
t=1
t=1
u(() + u(q");
(33)
and is absolutely homogeneous;
u((q)= Σ
C(ch: cy.) + Σ ((es: - cG++1)
(34)
1=1
|c|
Σ
<(D: -1) + 1c/
Σ
((. 91t1)
(35)
1=1
t=1
= lc/u(q):
(36)
implying is convex;
u(rg (1 (1-1)) -r)aa)) S u((r) +(((1 )))q1) |r|u(q) 1 +-rl(((q)) = ru(g) + +11 (1-r)u(q²).
(37)
(38)
(39)
where 0 S S 1. Because is jointly convex on q, it is also convex with respect to (92.... ........ 9-) when y and Ú, are fixed.
Our goal is now show that, assuming y and タ are fixed, ú. タ1 for < S is local minimizer of u. That is, we want to find the À regime where any small movement away from this point increases the value of u. Assume perturbed prediction
yt -31 Pl, 21 + P2. .2..... タ1 + pp),
(40)
17

---

where O. Starting with the definition of 2,
u(ÿ. y) Σ ((i ye) +
(3) 91)
(41)
1=1
[=1
Σ C(0) - ye)
(42)
u .)) = Σ ((9 + + - yy) + A
<((n +p:) - + -P++1))
(43)
Σ t=1
C(ú + v1) + A
S(p: P(+1)
(44)
t=1
Let
0 argmax((p.)
(45)
1E1:7
be the time step with the aargest-magiitule perturbation, and let é = ((ps) be the magnitude of this perturbation. Applying the reverse triangle inequality to the first summation in Equation 44 (the accuracy term), we have
((i + P. yi)
((()) -y:) (-p::)
(46)
(S y1) ((-p:))
(47)
Σ c(9) - y.) Σ ((P)
(48)
Σ ((ú) Ui) Σ ((P))
(49)
C(u yy) (T 1)
(50)
So, introducing the perturbation reduces the accuracy term by most (T 1)d. Now considering the second summation (the stability term),
C(D: P++1) 2 A Σ <(p: Pc+1)
(51)
[=l
2 AC(Pi - -po)
(52)
A((pe)
(53)
Ad.
(54)
That is, introducing the perturbation increases the stability term by at least Ag.
Therefore, if X > T- - 1, the overall change in u is positive, and we have shown that 9. = 9) for 1 <<s s < 7 local minimizer of u. By convexity of u, this point is also global minimizer.
B Transport Metric
In this section, we propose alternate metric for with unified loss. Specifically, describe a variant of the Wasserstein metric that accounts for the spatial structure of an image or feature tensor.
Let 21, 22 E k be two image feature map channels, and let a the transport distance T(a) = ((a) = 8(21, 22) as the minimum cost of a linear optimization. Intuitively, this optimization finds the shortest correspondence from to zero. The correspondence
Z2- We define
18

---

can employ three mechanisms: (1) mass movement from positive region to a negative region, with cost proportional mass and distance, (2) mass destruction, with cost proportional mass, and (3) mass creation, also with cost proportional to mass. The cost to create destroy unit of mass is 7-
Formally, solve the following optimization:
T(a) = min
Σ 1
digmi + + +c)
(55)
m.p.C.
subject to
a, + P: - Σ Mij =0 Vi
(56)
p > 0 Vi
(57)
G > 0 Vi
(58)
mij > 0 Vi,j
(59)
where de is the distance (Euclidean) from pixel i pixel j. myj is the mass moved from pixel to pixel j. and pj and o are the mass production and consumption at pixel i. Both and j are in the range 1...h w.
At first glance, this problem may seem computationally infeasible because the number of diy pa- rameters equals the number of pixels squared, e.g., trillion parameters for one-megapixel image. Fortunately, we can reduce the number of parameters to ~ (h x w) by pruning all edges where diz 2y. Intuitively, if dij > 2 the cost to destroy unit of mass at location and recreate it location j is less than the cost to move it from i to j.
Despite pruning, this optimization remains somewhat impractical. Finding solution takes secondss using the open-source solvers available in SciPy. These runtimes make loss evaluation the primary bottleneck during training. We might be able to reduce runtime using other solvers either general- purpose commercial solvers or specialized optimal transport solvers. We leave this as future work.
C Composing Stabilizers
Often, a deployed model will be faced with several simultaneous corruptions. One option in this scenario is to train single stabilizer on the expected combination of corruptions. However, doing so requires full knowledge of the corruptions at training time. In this section, we explore an alternate approach: composing (fusing) single-purpose stabilizers without additional training.
We consider controlled stabilizers without spatial fusion. Assume we have two single-corruption stabilizers, denoted by superscripts 1 and 2. The output ži, of the fused stabilizers at layer i is
ã = B¹ 3 21,, + (1-3%,) @ Ri
(60)
- = Bl, O 21,1 (1-
(61)
B1, (h) (I x1)))))
(62)
Bi, = (1h) (w., 11)).
(63)
Note that we have removed the feature-space inputs 21, 1,,, and z;-1 to the stabilization heads. In experiments, we found this was necessary to prevent unintended interactions between the stabilizers.
The above formulation is roughly equivalent to applying a single stabilizer with decay B1 O B We can modify the method slightly make this strictly true, thereby obtaining commutative composition. We replace
- in Equation 61 with ž obtaining
t, = _ @ (B 2 Zi,t + (1 B(,)) 3 2.,-1) + (1- (1_)),) 2 21,-11
(64)
3 ④ B ④ Zi,t (B,, B3, 0r, +1 1 @ 2,,-1,
(65)
= 3 ④ B. @ 21,1 + (1 B2, о (.,) о E,,-1-
(66)
Intuitively, the fused stabilizer retains the current features ž,, only both decays are near one; this indicates that neither controller backbone detected cormuptioninduueed instability.
In our experiments, we use the initial non-commutative version (it was slightly easier to implement). However, expect the two formulations to give similar results in general.
19

---

D Method Details
This section provides further details of method and implementation, including the particular architecture we use for the controller, our approach for training initialization, and a more formal definition of the spatial fusion mechanism.
D.1 Controller Architecture
The controller backbone uses a simple convolutional architecture with layers. The backbone has 32 channels for HDRNet and NAFNet, and 16 channels for Depth Anything. Controller heads have layers. The number of channels in the last head layer depends on the shape of & and the size of the fusion kernel. For other head layers, use 64 channels for HDRNet and NAFNet, and 32 channels for Depth Anything. All convolutions use a 3 x3 kernel, and all except the head outputs are followed by leaky ReLU with negative slope 0.01.
D.2 Initialization
When training, we initialize such that the predicted 33 values are near 1. This corresponds a rapid decay of the past state and less stabilization; i.c., the stabilizers For controlled stabilizers, this can be achieved by adding a final bias to (before the sigmoid) and initializing this bias to sufficiently positive value v. For the simple learned stabilizer (without controller), the trained parameters are logit values used to generate a decay E [0, 1] via / = (I). Again, we initialize these logits positive value For both the simple learned and controlled variants, we find that e = 4 (resulting in B z 0.98) works well.
initialized close to identity.
D.3 Spatial Fusion
Let N be a spatial neighborhood around the pixel to be stabilized. Let e denote the channel index, index of the stabilized pixel, N the neighbor pixel index, and E 1 m an index into the kernel 17. The output ب, of the spatial fusion stabilizer is given by
,tc +1c
Σ
(67)
(6.))E(W.1.m)))
nh. = Softmax((M(g( ac 1)- 2t, 1lmmmmntmm1 0),
(68)
The summation iterates over locations in the neighborhood, with the kernel index always corre- sponding to the neighbor index k. The indexing on the output of extracts the m channels needed to construct the kernel Tt.c- Note that we have dropped the layer index here for brevity.
E Experiment Details
This section contains experiment details and hyperparameter values.
E.1 Image Enhancement
Base model fine-tuning, We train for 80 epochs (2k iterations per epoch), using the Adam opti- mizer (31], an MSE loss, and batches of 8 randomly sampled frames. The learning rate initially set to 10-4 and is scaled by 0.1 after epochs 40 and 60.
Stabilizer training and evaluation. Each training batch consists of one randomly sampled video snippet containing 7 = 8 consecutive frames. Gradients are computed using BPTT. Stabilizers are trained for epochs (4k iterations per epoch) using the Adam optimizer and our unified loss with 5 || ||2. The learning rate is initialized to 10-3 for the simple learned stabilizer and 10-4 for other variants, and reduced by a factor of 10 after epochs 10 and 15. We evaluate stabilizers on the validation set, processing each video in single pass per video (i.c., 7 > 8 at evaluation).
20

---

Figure 6: VisionSim sequences. Showing scenes bachelors-quarters, classroom, library-homeoffice, and restaurant. For each, we show frames 10, 110, 210, 310, 410, and 510.
F,2 Video Denoising
Base model fine-tuning Each batch consists of 8 randomly sampled, randomly cropped patches of size 256x256. We train for 20 epochs (2k steps per epoch) with Adam and MSE loss. We set the initial learning rate to 10-4, scaling by 0.1 after epochs 10 and 15.
Stabilizer training and evaluation. Each batch consists of one randomly sampled, randomly cropped video snippet of size 256x 256 containing = 8 consecutive frames. We train for 20 epochs (2k steps per epoch) using Adam and the unified loss with 8 ||. ||2. The initial learning rate is set to 10-2 for the simple learned stabilizer and 10-4 for the controlled and spatial variants. In all cases, scale the learning rate by 0.1 after epochs 10 and 15. We evaluate stabilizers the validation set in single pass per video (7 > 8).
E.3 Corruption Robustness
Depth training dataset. We use the VisionSim 25
framework to generate dataset with ground-
truth depth labels, according to the instructions provided at the following URL: https://vissinsiin readthedocs.
http////ttttttoorraa/rarrrrreeee datasgee-dttst.....m .ttll
The resulting dataset contains 50 scenes with 59950 total frames and resolution 800x800. All scenes are indoor and exclusively contain ego-motion. See Figure for several representative sequences.
Depth metries. Depth Anything predicts relative disparity (inverse depth), which requires an affine alignment to the ground truth before computing metrics (see [54] for details). After this alignment, we evaluate the standard AbsRel and D (6 > 1.25) metrics. We exclude outliers by clipping the aligned depth to maximum of 200. It is critical to measure instability after alignment; otherwise, the network can achieve arbitrarily low instability without harming AbsRel and Delta-1 by scaling predictions to small range around zero.
Depth stabilizer training. We train Depth Anything stabilizers on randomly sampled snippets of length 7 = 8, randomly cropped to size 512x512. We train using Adam for 20 epochs (4k iterations
21

---

Figure 7: Elastic transform. A dummy image before and after applying the elastic transform, using the same parameters our experiments (magnitude 50.0 and smoothness 5.0). The image size is 256x256, and the lines have width 16.
per epoch) with a batch size of one. The learning rate is initialized 10-4 and reduced by 0.1 after epochs 10 and 15. We evaluate the unified loss (Ō = || 112) after affine alignment.
Image corruptions. We generate elastic distortions using the Torchvision fastiiTransfoooo orm class with default settings (magnitude a = 50.0 and smoothness a = 5.0). See Figure 7 for a visualization of the elastic transform.
For image denoising, we add all corruptions after Gaussian noise. For example, dropped frames contain zeros, not zero-mean Gaussian noise.
E.4 Adverse Weather Robustness
Base model fine-tuning, We use the same schedule and hyperparameters as in Section E.2 epoch consists of 1k training steps.
Each
Stabilizer training and evaluation. We again use the same schedule and hyperparameters as in Section E.2
The initial learning rate is set 10-1, and each epoch consists of 800 steps.
E.5
Compute Requirements
We train and evaluate compute cluster largely using RTX A4500 GPUs. Fine-tuning and stabilizer training take 1-2 days on a single GPU, and full evaluation takes 1-3 hours.
F
Additional Results
F1
Stabilizer Composition
We evaluate composed stabilizers (Appendix C on the NFS denoising task (a 0.1) using the NAFNet model. We train single-purpose corruption stabilizers using the same hyperparameters as other experiments, then evaluate composition under all possible two-corruption pairings. Results are shown in Table 3 Generally, stabilizer composition is effective if the second corruption does not significantly change the appearance the first. For example, JPEG + impulse works better than impulse JPEG. JPEG compression obscures high-frequency impulse noise, thereby interfering with the impulse noise controller backbone. This limitation could likely be addressed through data augmentation during stabilizer training, which would make the backbones more robust to changes in corruption appearance.
F.2 Semantic Segmentation
Task, dataset, and base model. In this subsection, we analyze a higher-level prediction task: semantic segmentation. We use the DeepLabv3+ model [8] specifically, the MobileNet variant published by [5]. We train and evaluate on the VIPER dataset [56), which contains video sequences captured in game engine (GTA V) and automatically labeled for various vision tasks. The predefined
22

---

Ours (stabilized)
Base model (unstabilized)
First comuption
Second corruption
PSNR
SSIM
Instability
PSNR
SSIM
Instability
Patch drop
Elastic distortion
29.53
22.64
0.660
152.40
Patch drop
Frame drop
33.88
0.919
19.15
17.47
0.658
218.69
Patch drop
JPEG antifacts
29.07
0.845
31.67
18.51
0.637
152.60
Patch drop
Impulse noise
24.85
0.769
60.89
17.49
0.367
153.96
Elastic distortion
Patch drop
29.93
0.850
19.24
18.42
0.657
155.24
Elastic distortion
Frame drop
28.01
0.780
17.79
25.65
0.769
129.61
blastic distertion
JPEG anifacts
29.11
0.842
18.46
25.94
0.759
53.75
distortion
Impulse noise
69.70
Frame drop
Patch drop
33.84
0.916
19.79
17.50
0.659
218.18
Frame drop
Elastic distortion
29.67
0.844
18.69
68
0.768
130.22
Frame drop
JPEG anifacts
25.99
0.737
17.59
26.56
0.720
124.21
Frame drop
Impulse noise
28.95
0.807
$7.28
23.14
0.467
103,00
JPEG artifacts
Pstch drop
30.55
0.638
154.20
JPEG artifacts
Elastic distertion
29.19
0.844
18.16
25.81
0.755
54.48
IPEG artifacts
drop
25.87
124.21
JPEG artifacts
Impulse noise
28,40
0.810
19.74
23.93
0.475
65.24
Impulse noise
Patch drop
30.14
0.831
24.61
16.98
0,351
163,34
Impulse noise
Elastic distortion
26.73
0.790
17.86
23.38
0.600
62.01
Impulsee moise
Frame trop
28.94
0.794
21.21
22.66
0.460
138.58
Impulse noise
artifacts
0.352
Table 3: Stabilizer composition. The effectiveness of composing stabilizers for different combina- tions and orderings of input corruptions. Results are for NAFNet on the NFS dataset with moderate noise (o 0.1). The base model evaluated without corruptions achieves PSNR 36.6, SSIM 0.945, and instability 26.2.
training and validation splits contain 77 sequences (134097 frames) and 47 sequences (49815 frames), respectively. We measure prediction quality using pixel accuracy and mloU on the predefined validation set. Due to the discrete nature of predictions, report categorical instability the fraction of pixels whose category changes between frames (equivalent to Ilo).
Experiment protocol. We fine-tune the unstabilized model, starting with the Cityscapes weights published by [1]. A 1 1x1 convolution is applied the output logits adapt the number of classes for VIPER. We fine-tune for 60 epochs (1925 batches per epoch), using batch size of 16 and cross-entropy loss. Adam was used with an initial learning rate of 10 scaled by 0.1 after epochs 20 and 40.
After fine-tuning, stabilizers attached to the model input, the model output, each InvertedResidual layer, the aspp block, the project block, and the classifies fie block. We then freeze the fine-tuned weights and train stabilizers on snippets of length = 8 with 0.4. Here tried both || l|2 and cross-entropy for 6. Cross-entropy gave slightly better results, despite not satisfying the formal criteria in Section 4 (e.g., it is not symmetric). Therefore, report results with cross-entropy in the remaining experiments.
When training stabilizers, each batch consists of one snippet of length r = 8 frames. We train for 60 epochs (3080 steps per epoch), using Adam with the same learning rate schodule as in fine-tuning
Results. The unstabilized model achieves categorical instability 0.079, mloU 0.406, and pixel accuracy 0.900. After adding stabilizers, we obtain instability 0.059, mloU 0.41 1, and pixel accuracy 0.901. Similar to other tasks, there is significant improvement in stability, along with increase in accuracy (mloU).
F.3 Segmentation Robustness
Task, dataset, and base model. In this subsection, evaluate the DeepLabv3+ segmentation model against the five corruptions from Section 6.3 (patch drop, elastic distortion, frame drop, JPEG artifacts, and impulse noise). We the same dataset (VIPER) and metries in Section F.2
Experiment protocol. We follow the same protocol as Section F22 when training and evaluating corruption stabilizers.
23

---

Curruption
Method
mloU
Accuracy
Instability
Patch drop
Base model
0,060
0.164
0.454
Ours
0.405
0.896
0.064
Elastic distortion
Base model
0.377
0.888
0,090
Ours
0.403
0.898
0,064
Frame drop
Base model
0,369
0.829
0.209
0,406
0.898
JPEG artifacts
Base model
0.109
0,313
0.216
Ours
0.337
0.862
0.066
Impulse noise
model
0.051
0,300
0.312
Ours
0.399
0.894
0.065
Table 4: Segmentation robustness. For all corruptions, our method simultaneously improves mloU, pixel accuracy, and instability. The improvement largest for corruptions that significantly change the appearance of the input, e.g., impulse noise.
Results. See Table F.3 for metric values, and Figure for sample predictions. Our method improves both task metrics and stability across all corruptions. For patch drop, JPEG artifacts, and impulse noise, adding stabilizers allows the model recover from catastrophic prediction failures.
F,4 DAVIS Denoising
Task, dataset, and base model. In addition to NFS, we evaluate NAFNet denoising on the standard
DAVIS benchmark 44 52
DAVIS contains 50 videos (3455 frames) collected at 24 FPS. We use the dataset's predefined train/validation split and scale images to a short edge length of 480 [62 we evaluate with a noise level of 40/255 & 0.16.
62
Following from prior work 44
62
Experiment protocol. We fine-tune the base model and train stabilizers following the same procedure as for NFS (see Sections 6.2
and E.2) Fine-tuning epochs contain 3k steps, and stabilizer training
epochs contain 2.4k steps.
Results. See Figure [ and Table 7 for results. Overall, the behavior is similar to the NFS results in Section 6.2 However, the "win-win" region (where both accuracy and stability are improved) is smaller for DAVIS. This is likely caused by DAVIS's 10x lower frame rate, which corresponds to higher inter-frame motion and lower frame-to-frame correlation.
F5 Adversarial Robustness
In addition to natural corruptions, we evaluate our method in the presence of adversarial corruptions. We consider setting where the attacker has knowledge of the base model and its parameters, but not of the stabilizers. We reason that because the stabilizers have fewer parameters than the base model and can be trained more quickly, a defender could update the stabilizer parameters after weight leak rather than retrain the entire model.
Task and model. We evaluate adversarial robustness on binary classification task derived from the DAVIS dataset. Frames are processed by tightly cropping around an object's 's segmentation mask, treating individual instances separate images, and labeling each crop human nonhuman. As our backbone, we use ResNet-50 [22) pre-trained on ImageNet. We replace the original 1000-class output layer with a two-class linear layer.
Experiment protocol. We freeze all pre- treitra weights except those in the final residual block (1ayer4) and the new classification head. We fine-tune these parameters for 100 epochs the binary classification task. We start with a learning rate of - decreasing it by a factor of 0.1 at epochs 40 and 80, and employ frame-level data augmentation consisting of random rotation, horizontal flip, and color jitter. To preserve temporal context for future stabilizers, each training sample consists of a sequence of eight consecutive frames.
Then, we generate adversarial examples using the iterative Fast Gradient Sign Method (I-FGSM) with overall perturbation bound 0.1 and 20 iterations per image [19. As the attacker does not know the ground-truth class, for each image, we randomly apply either the sign of the gradient step or its negation. We selected these hyperparameters because they produce substantial drop in
24

---

32
Base model
31-
Output fixed
Simple fixed
PSNR
30
Simple leamne
Controlled
29-
Spatial
27-
80
100
Instability
Figure 8: DAVIS denoising. We obtain the best quality/stabilitt tradeoff when using a stabilization controller (" ("controlled" and "spatial"). On DAVIS, spatial fusion does not offer significant advantage compared to basic controlled stabilizer. DAVIS has relatively high inter-frame motion, which often exceeds the size of the spatial kernel (i.e., the maximum translation achievable with spatial fusion).
baseline accuracy, creating a clear opportunity for the stabilizers to improve performance. Then, to defend against the attack, we add controlled stabilizers each bottleneck block the ResNet model. All original ResNet parameters are frozen, and only stabilizers are trained for 20 epochs under the same I-FGSM attack settings as above. The stabilizer training uses an initial learning rate of 10 10-4, reduced by factor of 0.1 after epochs I and 10, and each batch consists of two sequences of eight frames to ensure well-defined gradients. No additional augmentations are applied during this phase.
Results. Under our adversarial setup, the ResNet- 50 tinettund baseline achieves 77.0% accuracy, while our subblizer-augmenteer model reaches 88.8%. an absolute improvement of I 11.8% These results demonstrate that the proposed stabilizer modules can significantly improve resilience to adversarial attacks without requiring complete retraining.
F.6 Spatial Fusion Failures
Discussion. In the denoising experiments (Section 6.2). we observed significant PSNR reduction when using the spatial fusion method under extreme noise. This reduction only appears when evaluating on long sequences.
A closer examination of the outputs reveals blurring/ghosting artifacts that appear after some time has passed. These artifacts look like hard edges "bleeding out" into the surrounding regions. We believe these failures related the spatial fusion stabilizer the output layer. Without this stabilizer, the network output is biased toward the current input frame (due to the network architecture, which predicts noise residual). Adding spatial fusion stabilizer the output provides an independent mechanism for information to flow between pixels, thereby weakening this bias.
Experiment protocol. We ran experiment to determine whether spatial fusion failures can be mitigated by training on longer sequences. We trained the spatial fusion stabilizer under extreme noise sequences of length the training patch size from 256 256 to 180x x 180 to compensate for increased training memory requirements longer sequences. For 8, we doubled the number of iterations per epoch due to the lower number of frames in each iteration. We evaluated the resulting models on the full validation set containing long sequences.
(the default in other experiments) and 16. We reduced
Results. Training with t gave validation PSNR 22.70 and instability 11.04, whereas training with 16 gave PSNR 23.80 and instability 10.16. We expect this trend of improvement to hold as we further increase the training sequence length.
E.7 Uncertainty Estimate
Experiment protocol. To estimate uncertainty in our results, we train and evaluate the spatial fusion stabilizer eight times with different random seeds. We consider image enhancement (HDRNet) for the moderate-strength local Laplacian operator (a = 0.25). The random seed determines the stabilizer weight initialization and the training data shuffle, The training and evaluation protocol identical to that in Sections 6 and E.
25

---

Unstubbiizzd
Input
Stabilized
Unstabilized
Input
Stabilized
Figure 9: Denoising under extreme noise. As increase the level of image noise, frame-wise temporal inconsistency becomes more severe, to the point that becomes apparent when comparing static images. In the top sequence, features. In the jellyfish sequence, the denoiser hallucinates inconsistent spatial structure between frames. In both cases, adding stabilizer noticeably improves temporal consistency. We encourage the reader to view the corresponding video files included with the supplement.
shifting texture in the duck's head in the unstabilized
Results. PSNR ranges from minimum of 32.17 to maximum of 32.32, with of 32.25. SSIM has range 0.926-0.929 (mean 0.927), and instability has range 28.61-28.80 (mean 28.71). For all metrics, variations around the mean are 0.5%.
F88 Figures
Figure 9 shows example sequences denoised under extreme Gaussian noise (o = 0.6). We highlight regions of the image where instability is especially prominent. Differences are more noticeable in videos; we encourage the reader to view the video files included with the supplementary material.
and 13 contain examples of corruption robustness for image enhancement, denoising, depth estimation, and segmentation, respectively. Figure illluuttrates improved weather robustness for denoising on RobustSpring [4
Figures 10 I1
F9
Tables
Table provides complete results for image enhancement (Figure Tables and contain complete results for denoising (Figures and 8).
26

---

Input
Unstahilized
Stabilized
2
y
y/
Parch drop
Elostic disortion
Framee deop
JPEG artiliess
Impalse noise
Figure 10: Image enhancement robustness. The effect of stabilization for image enhancement
(HDRNet) under various image corruptions. See Section 6.3
27

---

Input
Unstahilized
Stabilized
2
y
y/
Parch drop
Elostic disortion
Framee drop
JPEG artilisss
Impalse noise
Figure 11: Denoising robustness, The effect of stabilization for denoising (NAFNett under various
image corruptions. See Section 6.3
28

---

Input
Unstabilized
Stabilized
22-
3 1
3+
y/
Paich drop
Elastic distortion
Frame drop
JPBG artifacts
Impuise noise
Figure 12: Depth estimation robustness. The effect of stabilization for depth estimation (Depth Anything v2) under various image corruptions. Improvements are most prominent for the patch drop, clastic distortion, and frame drop corruptions. The base model already has reasonable robustness to IPEG artifacts and noise. See Section 6.3
29

---

Input
Unstabilized
Stabilized
-1-1
Zt
y
y/
Patch drop
Elostic discortion
Frumee drop
JPEG artilises
Impuloe nokse
Figure 13:
Segmentation robustness. The effect of stabilization for segmentation (DeepLabv3+)) under various image corruptions. See Section map may cause color-class correspondences vary across rows.
F2 Note that the method we use to generate the color
Input
Unstabilized
Stabilized
ye
Rain
Snew
Figure 14: Adverse weather robustness. The effect of stabilization for denoising (NAFNet) under
weather corruptions on the RobustSpring dataset. See Section 6.4
30

---

Moderate intensity
intensity
0.25)
Strength
Instability
PSNR
SSIM
Instability
PSNR
Gaussian
J 0.50
29.46
30.70
0.917
36.26
25.73
0.852
Gaussian
P
27.20
Gaussian
2.00
16.49
27.38
0.848
19.99
23.65
0.773
Goussian
/
26.
22.79
Gaussian
4.00
11.29
25.33
0.782
13.57
22.15
0.700
Goussian
== 6.00
8.76
24.18
0.739
10.49
21.26
0.654
Ouiput lixed
8 = 0.99
32.74
30.92
0.919
40.38
25.84
0.856
Ourput fixed
39.99
Ourput fixed
á 0.95
31.50
30.87
0.919
38.82
25.81
0.855
Ourput fixed
0,90
30.00
30.76
0.917
36.94
25.76
0.852
Output lixed
$ 0.80
27.16
30.40
0.912
33.37
25.57
0.846
Ourput fixed
B 0.60
21.80
29.26
0.893
26.65
24.91
0.824
Simple fixed
á 0.99
32.81
30.92
0.919
40.64
25.84
0.856
Simple fixed
8 0.98
32.55
30.90
0.919
40.47
25.83
Simple fixed
$ 0.95
31.61
30.82
0.918
39.67
25.77
0.853
Simple fixed
8 0.90
29.76
30.59
0.914
37.59
25.62
0.848
Simple fixed
Simple fixed
B 0.60
19.28
28. 10
0.865
23.85
23.99
0.788
Simple learned
À 0.1
32.44
30,92
0.920
39.58
25.84
0.856
Simple learned
0.2
31.34
30.87
0.919
37.64
25.79
0.854
Simple learned
66
Simple learned
À 0.8
23.60
29.75
0.903
26.43
24.90
0.825
Controlled
À = 0.1
31.78
31.25
0.922
38.31
26.26
0.865
Controlled
0.2
30.57
31.13
0.920
36.11
26.11
0.859
Controllod
0.917
31.98
0.855
Controlled
À 0.8
21.79
29.85
0.901
24.64
25.17
0.824
Spatial
À 0.1
32.38
32.81
0.934
40.32
27.97
0.891
Spatial
0.2
31.29
32.70
0.933
38.29
27.79
0.890
Spatial
0
28.66
32.30
0.928
34.22
27.45
0. 0.881
Spatial
0.8
22.31
31.00
0.910
25.61
26.27
0.849
Table 5: Image enhancement results. These results correspond to the experiments in Section 6.1
(Figure 4). We additionally include results for simple Gaussian smoothing of the output, where the standard deviation of the smoothing kernel.
31

---

NFS moderate
NFS strong
Strength
Instability
SSIM
Instability
Gaussian
J 0.50
22.87
36.77
0.945
24.50
33.52
0.903
Gaussian
P
Gaussian
2.00
11.90
32.96
0.920
11.77
31.64
0.889
Goussian
/
Gaussian
4.00
8.18
30.15
0.883
8.01
29.50
0.860
Goussian
== 6.00
6.39
28.60
0.857
6.25
28.17
0.837
Ouiput lixed
8 = 0.99
25.93
36.67
0.943
28.18
33,30
0.900
Ourput fixed
Ourput fixed
á 0.95
24.79
36.76
0.944
26.81
33.40
0.901
Ourput fixed
0,90
23.43
36.79
0.945
25.20
33.50
0.902
Output lixed
$ 0.80
20.91
36.58
0.945
22.22
33,54
0.904
Ourput fixed
B 0.60
16.37
35.34
0.940
16.97
33.07
0.903
Simple fixed
á 0.99
27 27.40
35.95
0.927
32.19
32.22
0. 0.847
Simple fixed
8 0.98
30.21
34.67
0.885
39.59
30.51
Simple fixed
$ 0.95
40.44
31,30
0.734
63.31
26.52
0.491
Simple fixed
8 0.90
53.94
28.14
0.565
91,84
23.11
0.322
Simple fixed
114.95
Simple fixed
B 0.60
$6.70
24.89
0.405
100.92
19.83
0.205
Simple learned
À 0.1
22,98
36.82
0.947
22.34
33.58
0.9U8
Simple learned
0.2
21.80
36.74
0.947
20.58
33.51
0.908
Simple
Simple learned
À 0.8
15.38
35.00
0.939
13.48
32.24
0.898
Controlled
À = 0.1
22.21
37.51
0.952
21.41
34.36
0.916
Controlled
= 0.2
21.28
37.41
0.951
20.21
34.30
0.915
Controllod
17.24
33,93
Controlled
À = 0.8
14.61
35.776
0.942
12.95
33.05
0.903
Spatial
À A 0.1
22.13
37.65
0.953
21.08
34.52
0.917
Spatial
0.2
21.10
37.57
0.952
19.80
34.45
0.917
Spatial
0.4
37.15
0.950
16.99
34.06
Spatial
À A= 0.8
14.25
34.94
0.931
11.84
32.53
0.895
Table 6: Denoising results, part 1/2. These results correspond to the experiments in Section 6.2
(Figure [}.. We additionally include SSIM and results for simple Gaussian smoothing of the output, where y is the standard deviation of the smoothing kernel. See Table for the second half of the data.
32

---

0.6)
DAVIS
10/255)
Strength
Instability
SSIM
Instability
Gaussian
J 0.50
30.23
27.98
0.794
105.62
30.67
0.864
Gaussian
P
70.68
Gaussian
2.00
11.82
28.02
0,803
45.38
23.22
0.717
Goussian
/
0.797
34.27
21.92
Gaussian
4.00
7.59
27.17
0.790
27.95
21.15
0.653
Goussian
== 6.00
26.45
0.778
21.01
20.25
0.626
Ouiput lixed
8 = 0.99
35.64
27.72
0.787
121.46
31.65
0.873
Ourput fixed
Ourput fixed
á 0.95
33,65
27.83
0.790
115.50
31.48
0.871
Ourput fixed
0,90
31.30
27.94
0.793
108.36
30.96
0.867
Output lixed
$ 0.80
26.98
28.12
0.798
94,92
29.54
0.853
Ourput fixed
B 0.60
19.53
28.27
0.804
70.52
26.60
0.810
Simple fixed
á 0.99
45.83
26.35
0.616
120.92
31,14
0. 0.844
Simple fixed
8 0.98
64.92
24.36
0.415
121.40
30.05
Simple fixed
$ 0.95
119.74
20.09
0.191
130.25
26.85
0.581
Simple fixed
8 0.90
180.81
16.56
0.105
148.51
23.76
0.415
Simple fixed
Simple fixed
B 0.60
205.99
12.97
0.059
137.79
20.09
0.270
Simple learned
À 0.1
20.24
28.29
0.812
118.48
31.63
0.873
Simple learned
0.2
17.73
28.27
0.813
114.84
31.49
0.871
Simple learned
Simple learned
À 0.8
10.21
27.69
0.806
80.09
27.89
0.821
Controlled À = 0.1
19.47
29.22
0.824
117.58
31.85
0.876
Controlled
0.2
17.26
29.15
0.824
113.44
31.69
0.873
Controllod
À
0.822
104.21
Controlled
À = 0.8
10.12
28.45
0.813
77.15
27.86
0.815
Spatial
À == 0.1
15.20
22.00
0.665
117.57
31.86
0.876
Spatial
= 0.2
16.55
21.73
0.637
113.49
31.70
0.874
Spatial
À 0.4
22.28
0.692
104.26
31.05
1.865
Spatial
À = 0.8
11.66
22.45
0.669
81.86
28.15
0.824
Table 7: Denoising results, part 2/2. These results correspond to the experiments in Section 6.2
(Figure [}. We additionally include SSIM and results for simple Gaussian smoothing of the output, where j is the standard deviation of the smoothing kernel. See Table for the first half of the data.
33

---

G Licenses and Copyright
Code. We use our own implementation of HDRNet. For NAFNet, we the authors' code (https://gi thub. com www.///giii- reeeecch//aaaarratttetttttttttt /ttps//oao/oo///rrrrreea///////r MIT license). For Depth Anything, we use the anw.hhiinggDpphh-nnnnttteee
-2-Small- HuggingFace module (available under an Apache 2.0 license). VisionSim is released under the MIT license. All scenes are licensed under Creative Commons variant; see this Google Drive folder for attributions and further license details: https://årive gowgle. com/drrie//oll.ers////.//////// http://///........ooooooooooeeaa
Datasets and assets. We were unable to find license information for the Need for Speed dataset. DAVIS uses the BSD license. We use frames from the following YouTube videos in our figures:
34

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We provide proofs for theoretical claims in Appendix A We show empirical results three tasks: image enhancement, image denoising, and monocular depth estimation. Our method designed to be compatible with a wide range of tasks however, we do not make strong claims about the empirical performance untested tasks. We mention some limitations around sim-to-real generalization the discussion section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We include limitations paragraph in the discussion section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
35

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Proofs state the necessary assumptions (ō must expressible in terms of norm on V, and the stabilizer must not modify the first time step's 's prediction).
Justification: Complete proof' are included in Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We disclose complete architectural details and training hyperparameters in Appendix E We will release our code at publication, including configuration files and shell scripts for reproducing our results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
36

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: At publication will release code with detailed README. We use publicly-availabe datasets and well-documented open-source tools. Our codebase includes shell scripts for downloading and preparing the required datasets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/ Sublic/guides TrodeSubmissionooll
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7 //nips /nips.cccpublll. wwwpcppwlcgggp..eeeeeeeeeeeoeeee hwalcc//guis//eeeee http:/oeeeuamiissooooooeeeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We list all hyperparameters and other experiment details in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
We do not have the computational resources to estimate error bars using, e.g., cross-validation.
Guidelines:
37

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We include information on compute requirements Appendix F.5
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://pppp..aaae...c/////////aa
Answer: [Yes]
Justification:
We have reviewed and are in compliance with the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction)
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
38

---

Justification: We believe that the potential societal impact is minimal. The applications of our work are in general-purposs tasks such as image denoising and depth estimation where there is no clear avenue for misuse.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
We do not propose any high-risk models or datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have provided attribution for all code, models, and datasets. We include detailed license and copyright information in Appendix G
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
39

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We have generated a simulated monocular depth dataset part of our experi- ments. The tools for generating this data are publicly available; rather than distributing the data, we will provide instructions for regenerating it.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We have not performed any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We have not performed any research with human subjects.
40

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: We have not used LLMs in any significant capacity.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our L.LM policy for what should or should not be described.
41


Reviews:
[{"review_id": "mjbHIvZY1r", "reviewer": "Reviewer_pzRZ", "summary": "This paper talks about how to fix the flickering and shakiness we see when using normal image neural networks on videos. These models process each frame by itself, so the output is not smooth. The authors made a general method to fix this. They created small \"stabilization adapters\" that you can plug into any pre-trained model without needing to retrain the whole thing. These adapters are controlled by a small neural network that looks at recent frames to decide how much to smooth the image features. They also introduce a new loss function to balance accuracy and stability, and they did some math to prove when it will work properly. They show their method works well for image enhancement, denoising, and depth estimation, making the videos smoother and more robust to noise and other problems", "questions": "1. Your spatial fusion method performed very poorly on the NFS extreme noise setting. You suggest it's an \"unexpected attractor state\" and maybe increasing τ in training could fix it. Have you tried this? Can you give more idea on why this happens? Why only for spatial fusion and not the normal controlled stabilizer?\n2. For the video denoising part, the \"Internal EMA\" baseline was very bad because it tried to smooth the noise residual. This is a very interesting finding. Does it mean your learned controllers are smart enough to learn not to smooth these layers? It would be great if you could show this, for example by plotting the β values that the controller predicts for these layers. I would expect them to be close to 1.\n3. You tested against adversarial attacks using I-FGSM. This is good. Did you consider testing against other types of attacks? For example, attacks that are not based on gradient, to see if the stabilizer is still robust", "limitations": "Yes. The authors included a limitations paragraph in the discussion", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The idea is very practical and useful. You don't need to build and train huge video models from scratch. You can just take an existing, powerful image model and add these lightweight adapters to make it work well on video. This will save lot of time and money for many people.\n2. The paper has good theory. The unified loss and the \"oracle bound\" and \"collapse bound\" are very clever. It gives a clear reason for why we should pick a certain value for the hyperparameter. This is much better than just guessing.\n3. The experiments are very thorough. They test the idea on three different tasks (HDRNet, NAFNet, Depth Anything) which shows that the method is general and not just for one problem. They also test against many types of image corruption, which is great.\n4. The results are impressive. The method makes the video output much more stable, and in many cases, it also improves the PSNR or other metrics, especially when the input is corrupted. The video demos also show the improvement very clearly.\n\nWeaknesses:\n1. he theory has some limitations. The bounds they proved only work for simple loss functions that are a norm. For more complex loss functions used today, the theory might not hold. The authors do mention this, which is good.\n2. The spatial fusion method, which is their best one, failed on the extreme noise experiment. They said it might be some \"attractor state\" but did not fully explain it. This is a point of failure for the method.\n3. They don't report error bars or run experiments multiple times. They say it's because of compute cost, which is understandable, but it makes the results less statistically strong.", "was_revised": true, "final_justification": "After author feedback, i would keep my rating of accept.", "rebuttal": "We thank the reviewer for their feedback and respond to their comments below.\n\n**Lack of error bars.** We agree that adding error bars would strengthen the experimental results. Before camera-ready, we will run one set of experiments (a particular model and task configuration) multiple times to generate confidence bounds.\n\n**Analysis of failure cases with spatial fusion.** This is a great question, also raised by reviewer LzQr. We duplicate our response below to make the rebuttal self-contained.\n\nWe have examined these results more closely. When applying the spatial fusion method under extreme noise, we see some blurring/ghosting artifacts after enough time has passed. These artifacts can look like hard edges \"bleeding out\" into the surrounding regions. We believe this is related to the use of a spatial fusion stabilizer on the output layer. Without this stabilizer, the network output is biased toward the current input frame. However, adding a spatial fusion stabilizer to the output provides an independent mechanism for information to flow between pixels, thereby weakening this bias. \n\nBecause we only see these artifacts after some time has passed, the problem may be resolved by training on longer sequences. We ran a quick experiment to test this idea. We trained the spatial fusion stabilizer under extreme noise on sequences of length 8 (the default in our past experiments) and 16. For length 16, we reduced the total number of training iterations by half, to compensate for the greater number of frames in each iteration. We evaluated these models on the full validation set (containing very long sequences). Training with length 8 gave validation PSNR 22.70 and instability 11.04, whereas training with length 16 gave PSNR 23.80 and instability 10.16. We expect this trend of improvement to hold as we further increase the training sequence length.\n\nWe will incorporate this new experiment, sample results, and the above discussion in the supplement of our camera-ready version.\n\n**Smoothing of the noise residual.** To answer this question, we look at the \"learned EMA\" configuration, which is the same as the \"internal EMA\" variant except that we train the decay parameters (with one parameter per feature channel). We consider the NAFNet NFS model trained under moderate noise with $\\lambda = 0.2$. The table below shows the mean of the decay logits for each layer, along with the feature dimension to give a sense for the level of visual abstraction. Larger logit values correspond to beta near one (minimal smoothing). The last row is for the stabilizer on the model output.\n\nBy far, the most aggressive stabilization is applied to the model output. Within the residual-prediction backbone, we see the most stabilization in the intermediate layers. These intermediate layers correspond to higher-level features that should be more stable than either the noisy input image or the residual prediction. Note that although the mean logit values are quite high, there is significant variation across channels, with many channels having values in the 0-1 range. This suggests that some channels are much more amenable to stabilization, and supports the use of training over hand-tuning the stabilizer decay parameters.\n\nWe will include this new experiment and our discussion in the supplement.\n\n| Module name | Feature dimension | Logit mean |\n| --- | --- | --- |\n| module.encoders.0.0 | 32 | 12.51 |\n| module.encoders.0.1 | 32 | 12.03 |\n| module.encoders.1.0 | 64 | 10.37 |\n| module.encoders.1.1 | 64 | 9.91 |\n| module.encoders.2.0 | 128 | 9.41 |\n| module.encoders.2.1 | 128 | 9.50 |\n| module.encoders.2.2 | 128 | 9.75 |\n| module.encoders.2.3 | 128 | 9.33 |\n| module.encoders.3.0 | 256 | 8.14 |\n| module.encoders.3.1 | 256 | 8.56 |\n| module.encoders.3.2 | 256 | 8.62 |\n| module.encoders.3.3 | 256 | 8.61 |\n| module.encoders.3.4 | 256 | 8.61 |\n| module.encoders.3.5 | 256 | 8.63 |\n| module.encoders.3.6 | 256 | 8.64 |\n| module.encoders.3.7 | 256 | 8.24 |\n| module.middle_blks.0 | 512 | 5.55 |\n| module.middle_blks.1 | 512 | 5.64 |\n| module.middle_blks.2 | 512 | 5.29 |\n| module.middle_blks.3 | 512 | 5.38 |\n| module.middle_blks.4 | 512 | 5.33 |\n| module.middle_blks.5 | 512 | 5.26 |\n| module.middle_blks.6 | 512 | 5.32 |\n| module.middle_blks.7 | 512 | 5.40 |\n| module.middle_blks.8 | 512 | 5.32 |\n| module.middle_blks.9 | 512 | 5.31 |\n| module.middle_blks.10 | 512 | 5.16 |\n| module.middle_blks.11 | 512 | 4.52 |\n| module.decoders.0.0 | 256 | 7.63 |\n| module.decoders.0.1 | 256 | 6.44 |\n| module.decoders.1.0 | 128 | 7.85 |\n| module.decoders.1.1 | 128 | 7.40 |\n| module.decoders.2.0 | 64 | 8.71 |\n| module.decoders.2.1 | 64 | 7.45 |\n| module.decoders.3.0 | 32 | 10.33 |\n| module.decoders.3.1 | 32 | 10.19 |\n| module | 3 | 1.88 |\n\n**Testing against other adversarial attacks.** This is a good suggestion, although so far we have not tried this. In general, we would expect non-gradient-based attacks to be weaker than gradient-based attacks, and therefore easier to defend."}, {"review_id": "nSdGM0NxLI", "reviewer": "Reviewer_TYGq", "summary": "The authors propose a novel method for stabilizing frame-based video inference, enhancing its robustness to temporal inconsistencies and corruptions. The approach is task-agnostic, model-agnostic, and lightweight—capable of being applied on top of pre-trained models without requiring retraining. It is also causal, meaning that predictions for later frames are informed by those of earlier frames.", "questions": "Why do the authors not compare against the consistent video enhancement that they cite in section 2? Namely, references [4,25,29,67,70].\nHow would this perform in transformer architectures? Is it limited only to convolutional architectures? If that was the case, it would be a significant limitation.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The theoretical foundation is solid. The authors have developed their method, and the maths behind are explained soundly and clearly.\n* The method is indeed lightweight, and the training procedure seems straightforward. In principle, it should be perfectly reproducible with the details provided in the manuscript.\n* The method is adaptable for any architecture, although the authors only test it in convolutional layers.\n\nWeaknesses:\n* The experimental evaluation is very limited: the authors provide an ablation study without comparing against any existing SOTA solutions. \n* The experiments are performed only in convolutional architectures. How would this perform in a transformer architecture?", "was_revised": true, "final_justification": "The authors have addressed my concerns.", "rebuttal": "We thank the reviewer for their feedback and respond to their comments below.\n\n**Comparisons with consistent video enhancement.** Per the reviewer's suggestion, we have set up the author-published code for Bonneel et al. and have run their method on the NFS Laplacian task. We are currently seeing severe artifacts in the output of this method; we believe this is due to a misconfiguration and will continue to work on this throughout the discussion period. We will provide these results if we are able to obtain a valid comparison.\n\nThere are several key conceptual distinctions between our method and these prior works. First, unlike consistent video enhancement methods, our approach does not require that the output is a natural video. This flexibility is evidenced by our results for other tasks, such as depth estimation. Since the submission we have also performed experiments for semantic segmentation, which we will include in the camera-ready version.\n\nSecond, we consider not only stability, but also robustness, which allows our method to work even when the input video contains various corruptions (see Figure 6, Table 1, and Section 6.3 of our main paper). For example, the method of Bonneel et al. determines the amount of temporal smoothness to enforce by looking at low-level input cues, meaning it cannot smooth the output correctly in situations where the input video contains unstable corruptions.\n\n**Compatibility with transformer architectures.** Our method is indeed compatible with transformer architectures. When dealing with transformer layers, care must be taken to ensure that the controller output has the correct shape: a 1D list of tokens, rather than a 2D feature map. This can be achieved in several ways: by applying the backbone after an existing input tokenization, by adding a tokenization layer within the backbone, or by tokenizing the output of the controllers. In all of these cases, the controller layers after the tokenization should be switched from convolutions to MLP layers or self-attention blocks."}, {"review_id": "NA1sjOd7YR", "reviewer": "Reviewer_LzQr", "summary": "This paper addresses the issue of temporal inconsistency that arises when single-frame models are applied to video data.\n The authors propose a simple yet effective stabilization module that can be inserted into pre-trained frame-based networks without modifying their parameters, enabling them to perform reliably on video tasks.\n\nThe proposed method introduces a lightweight stabilizer that operates on both intermediate features and final outputs, improving temporal stability and robustness to input corruptions (e.g., noise, compression, missing regions).\nThe method also leverages a unified accuracy–stability–robustness loss, and the authors provide theoretical analysis with the oracle bound and collapse bound to prevent over-smoothing and ensure well-behaved training.\n\nExtensive experiments are conducted on a variety of tasks—image enhancement (HDRNet), video denoising (NAFNet), and depth estimation (Depth Anything v2)—demonstrating that the proposed stabilizers can significantly improve stability and robustness without sacrificing task accuracy, and in some cases even improving it.\n\nImportantly, the stabilizers are causal and memory-efficient, making them well-suited for real-time and latency-sensitive applications.\n By avoiding the need to design task-specific video architectures, this work provides a practical and general-purpose framework for adapting existing frame-based models to video inference with minimal cost and effort.", "questions": "In this paper, the proposed method is evaluated by augmenting image-based baselines for various tasks.\nHowever, for real-world video applications, it would be more appropriate to compare against video-specific baselines.\nThe paper would benefit from an analysis of how the proposed stabilizer performs when applied to existing video-based architectures.\nFor example, it would be interesting to see how much temporal consistency improves when applied to a video denoising model such as TurtleNet (NeurIPS 2024).\nIncluding such comparisons would further strengthen the practical relevance of the work.", "limitations": "N/A - no known limitations", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "[Strengths] \n\nThe paper proposes a simple yet effective module for improving temporal consistency in video inference. \nThe approach is validated across multiple tasks, including image enhancement, denoising, and depth estimation, demonstrating consistent temporal stability improvements. \nThe qualitative video results provide strong evidence for the practical effectiveness of the proposed stabilizer in real-world scenarios.\n\n[weaknesses]\n\nThe proposed stabilizer is essentially a combination of EMA and controller mechanisms, representing a well-engineered integration of existing ideas rather than a fundamentally novel concept.\nAs such, the contribution of the paper lies more in its practicality and generality, while its theoretical or architectural novelty is relatively limited.\n\nAnother weakness is that performance degrades under extreme noise conditions, especially with spatial fusion, which may limit robustness in harsh environments.", "was_revised": true, "final_justification": "The proposed rebuttal addresses all of my concerns. Thank you to the authors for the response. I will keep my original rating for the final recommendation.", "rebuttal": "We thank the reviewer for their feedback and respond to their comments below.\n\n**Performance degradation under extreme noise.** This is a great question, also raised by reviewer pzRZ. We duplicate our response below to make the rebuttal self-contained.\n\nWe have examined these results more closely. When applying the spatial fusion method under extreme noise, we see some blurring/ghosting artifacts after enough time has passed. These artifacts can look like hard edges \"bleeding out\" into the surrounding regions. We believe this is related to the use of a spatial fusion stabilizer on the output layer. Without this stabilizer, the network output is biased toward the current input frame. However, adding a spatial fusion stabilizer to the output provides an independent mechanism for information to flow between pixels, thereby weakening this bias. \n\nBecause we only see these artifacts after some time has passed, the problem may be resolved by training on longer sequences. We ran a quick experiment to test this idea. We trained the spatial fusion stabilizer under extreme noise on sequences of length 8 (the default in our past experiments) and 16. For length 16, we reduced the total number of training iterations by half, to compensate for the greater number of frames in each iteration. We evaluated these models on the full validation set (containing very long sequences). Training with length 8 gave validation PSNR 22.70 and instability 11.04, whereas training with length 16 gave PSNR 23.80 and instability 10.16. We expect this trend of improvement to hold as we further increase the training sequence length.\n\nWe will incorporate this new experiment, sample results, and the above discussion in the supplement of our camera-ready version.\n\n**Comparison against video-specific baselines.** Thanks for the suggestion - we agree that the paper would be strengthened by comparisons against video-specific baselines, especially for tasks like denoising. Although we do not have time to add these comparisons now during the rebuttal, we intend to add at least one such baseline for the camera-ready version.\n\nWe would also like to point out that our method is potentially complementary to many video-based models. Consider, for example, FastDVDNet, a video denoising model that maps five noisy input frames to one centrally-aligned denoised frame. To predict a denoised video, the model is applied in a sliding-temporal-window fashion to the input. Therefore, on each invocation of the model, the time offset of each output or activation value shifts forward by one frame. This mirrors the single-frame models we consider in the paper, where the activations also shift forward by one frame on each invocation. For other models which operate on distinct (non-overlapping) chunks of frames, our objective simply changes from \"temporal consistency between frames\" to \"temporal consistency between chunks.\" The effectiveness of our method in this case would likely depend on the duration of the chunks - as chunks become longer the temporal correlation between them will tend to fall."}, {"review_id": "MSrW86j3I5", "reviewer": "Reviewer_yzUk", "summary": "The submission presents a method to incorporate temporal consistency (stability) on deep neural networks designed for single-frame estimation, so that they can be applied on video data. The goal is being able to reutilize already expensively-trained networks so that they produce temporally-consistent predictions on video, as opposed to training dedicated networks for video. The method consists on designing smoothers (of the output or of intermediate feature layers) like moving averages and to learn the parameters to control the amount of smoothing. The method is tested on three tasks: image enhancement, image denoising and robustness to several types of input corruptions (JPEG artifacts, frame drop, etc.), and in comparison with several variations of the method and other baseline methods, producing, overall good results.", "questions": "- Biased by the duration tau? : Equations (1) to (5) seem to depend on the duration of the video, tau. Later, in the experiments, there seem to be some issues with short videos. Would the method produce different results if the formulas included a normalization factor (1/tau) , that is, divide by tau to avoid long videos to have a much larger influence on the stability and other quantities compared to shorter videos.\n\n- Table 1; image denoising, Frame drop. Why is the proposed approach 5 dB below the base model? Please double check these numbers.\n\n- Fig 2: it is not clear what the meaning of \"tilts towards stabler predictions\". I think I did not find described in the text how one can identify such zones on the diagrams.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The submission is very clearly written; one can grasp the main ideas on the first pass.\n- The motivation is well justified, given that large models (like Depth Anything) can only be trained by large companies with resources. It makes a lot of sense to be able to reutilize them and add some \"controller\" on top of them to leverage such models.\n- The approach is sound, combining controllers, stabilizers and some theoretical analysis.\n- Experiments are comprehensive, on three tasks and producing good results in comparison with several variations of the methods as well as other baselines.\n- The results in the supplementary video are impressive.\n\nWeaknesses:\n- The spatial fusion seems to produce good results in most cases, but it is only shortly introduced at the end of Sec. 5. Please consider explaining it in more detail in a revised version.\n- Figures 4 & 5: consider adding an arrow of which part of the space is the desired region. I assume it is the top left because it would be high PSNR and stable, but it is not clear if this is the goal. Please also consider adding curves to join points of the same type, so that we could see these curves, almost like approaching the Pareto front.", "was_revised": true, "final_justification": "The authors have addressed my comments in the reply. The explanations need to be included in the revised version. \nI remain positive about this submission.", "rebuttal": "We thank the reviewer for their feedback and respond to their comments below.\n\n**Explanation of spatial fusion.** We have included additional details in Section B.3 of the supplementary material, although we agree that the overall clarity of these explanations could be improved. We will revise and expand our description of spatial fusion in the camera-ready version.\n\n**Improvements to plots.** We agree that these changes would improve the readability of the plots, and will incorporate them in camera-ready.\n\n**Potential bias by duration $\\tau$.** In our current experiments, we fix $\\tau$ to a constant value during training, so there is no risk of longer sequences dominating the loss. However, for training setups where the video duration varies widely between sequences, it would be a good idea to introduce a duration-balancing factor $1 / \\tau$ as suggested. We will add a note to Section 3 describing this consideration.\n\n**Reduction in PSNR.** We have double-checked these numbers and this does not appear to be a transcription error or a bug. We have also confirmed that our method gives considerably higher subjective quality on test videos (as can be seen in the supplement video).\n\nWe conjecture that there are two things going on here. First, PSNR (averaged over frames in this case) does not capture the jarring perceptual effect of the dropped frames - so the high PSNR of the base model is somewhat deceptive. This points to the well-known limitations of PSNR as a video quality metric. We would point to the significant improvement in temporal stability as quantitative evidence of improved perceptual quality.\n\nSecond, there are hints of the spatial-fusion denoising failure mode here - PSNR is much higher (exceeding the baseline) on shorter sequences. See the response to reviewers LzQr and pzRZ for an explanation and a potential solution.\n\n**Figure 2 explanation.** By \"tilt toward\" here we mean \"slopes more gently toward.\" That is, deviations from the correct prediction are penalized less heavily if they are in the direction of better stability. We will revise this explanation in the camera-ready version."}, {"review_id": "3zoBvuATPp", "reviewer": "Reviewer_cmJ7", "summary": "This paper addresses the challenge of temporal inconsistency and corruption sensitivity in frame-based neural networks for video inference. The authors introduce a general approach using stabilization adapters that can be inserted into different architectures, combined with a resource-efficient training process that keeps the base network frozen. They propose a unified accuracy-stability-robustness loss to balance output fidelity, temporal consistency, and robustness, supported by theoretical analysis identifying conditions to avoid over-smoothing. The adapters operate causally on both internal features and outputs, generating control signals from spatiotemporal context to modulate predictions. Experiments on different tasks demonstrate the effectiveness of the proposed method.", "questions": "Please see the weaknesses part.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n1. The proposed method can be applied to different single-frame image inference architectures.\n2. The strategy of freezing the base network and only training the adapter parameters during training significantly reduces computational overhead.\n3. The method is supported by detailed theoretical analysis on accuracy-robustness trade-offs.\n\n[Weaknesses]\n\nOverall, the method proposed in this paper is good, but clarifying the following points would further enhance the work:\n\n1. For different tasks discussed in the paper, it appears that only a single network architecture is used for testing. If different networks are switched (especially between different styles, such as convolution vs. Transformer, or complex vs. lightweight architectures), would the proposed method remain effective?\n2. When the noise level is high (e.g., σ=0.6), the proposed method shows poor stability and is highly sensitive to the parameter λ, leading to difficulties in parameter tuning. Additionally, adjusting λ is not straightforward in most cases: when λ exceeds the collapse bound, it may cause prediction collapse, and the \"stability-accuracy win-win region\" for some tasks is narrow, requiring meticulous manual tuning.\n3. The experimental tasks, including enhancement, denoising, and depth estimation, are all relatively mature fields. How does the proposed method perform on more challenging tasks, such as low-light enhancement and deblurring?", "was_revised": true, "final_justification": "Thanks for the authors' reply. I have no further questions.", "rebuttal": "We thank the reviewer for their feedback and respond to their comments below.\n\n**Generalization across architectures.** While we have not tried different architectures within a single task, we considered several architectures in our experiments: NAFNet for denoising, HDRNet for image enhancement, and Depth Anything v2 for depth estimation. Since the submission, we have also added results for segmentation using DeeplabV3; we will include these results in camera-ready. These architectures cover a range of designs and include both lightweight and heavyweight models. While we believe our results are sufficient to support the main claim of the paper regarding the generality of the proposed method, if the reviewer feels this is critical, we are open to evaluating another architecture for one of the existing tasks in the camera-ready version.\n\n**Tuning $\\lambda$.** The failure mode under extreme noise ($\\sigma = 0.6$) is not related to difficulty in tuning $\\lambda$ but rather a combination of architecture- and task-specific factors. We have provided detailed analysis in response to the suggestions from reviewers LzQr and pzRZ.\n\nOverall we found tuning $\\lambda$ to be straightforward, due in part to the guarantees provided by the oracle bound. In general, we found good results with $\\lambda$ in the range 0.2-0.4. We did not find that the results were sensitive to fine adjustments of $\\lambda$ - variations within the oracle bound give a smooth, well-behaved tradeoff between accuracy and stability.\n\n**Evaluation on other tasks.** We have added results on semantic segmentation since the submission and will include these in camera-ready. We would also point out that denoising and low-light enhancement are two sides of the same coin (although the noise model may need to be adjusted to, e.g., Poisson in low light)."}]

Author rebuttal:

Meta-review: The paper proposes a method for stabilization of frame-based video inference. The contributions of this paper have a practical character and likely to make an impact on related approaches, given the modularity and transferability of the trained adapters to different architectures. The experimental analysis is solid, being it performed on three different tasks and showing good results compared to other methods as well as other baselines. The authors have also discussed limitations (e.g. the results in highly-noisy cases) and provided clarifying arguments for the related doubts. All-in-all the paper provides strong contributions, with thorough experiments and is supported by a convincing rebuttal.
Decision: Accept (poster)