ROSE: Remove Objects with Side Effects in Videos
Authors: Chenxuan Miao
Yutong Feng
Jianshu Zeng
Zixiang Gao
Liu Hantang
Yunfeng Yan
Donglian Qi
Xi Chen
Bin Wang
Hengshuang Zhao
Venue: NeurIPS 2025 poster ()
Keywords: video inpainting; diffusion transformer; video generation
Primary area: applications

Abstract: Video object removal has achieved advanced performance due to the recent success of video generative models. However, when addressing the side effects of objects, \textit{e.g.,} their shadows and reflections, existing works struggle to eliminate these effects for the scarcity of paired video data as supervision. This paper presents \method, termed \textbf{R}emove \textbf{O}bjects with \textbf{S}ide \textbf{E}ffects, a framework that systematically studies the object's effects on environment, which can be categorized into five common cases: shadows, reflections, light, translucency and mirror. Given the challenges of curating paired videos exhibiting the aforementioned effects, we leverage a 3D rendering engine for synthetic data generation. We carefully construct a fully-automatic pipeline for data preparation, which simulates a large-scale paired dataset with diverse scenes, objects, shooting angles, and camera trajectories. ROSE is implemented as an video inpainting model built on diffusion transformer. To localize all object-correlated areas, the entire video is fed into the model for reference-based erasing. Moreover, additional supervision is introduced to explicitly predict the areas affected by side effects, which can be revealed through the differential mask between the paired videos. To fully investigate the model performance on various side effect removal, we presents a new benchmark, dubbed ROSE-Bench, incorporating both common scenarios and the five special side effects for comprehensive evaluation. Experimental results demonstrate that \method achieves superior performance compared to existing video object erasing models and generalizes well to real-world video scenarios.

ROSE: Remove Objects with Side Effects in Videos
Chenxuan Miao 14, Yutong Feng"; Jianshu Zeng', Zixiang Gao², Hantang Liu², Yunfeng Yan', Donglian Qi', Xi Chen', Bin Wang', Hengshuang Zhao"t 'Zhejiang University, "KuBByte AI, Peeknng University, "The University of Hong Kong
(weiyuchhoumou26, fengyutong. fyt, {1iuhantang77, chauncey0620, Sanwang393)@gmall. com edu. cn hszhao@cs hku. hk
AI, gzx2401210062 }@gmail.com
[yyff, qidl) edu.
Abstract
Video object removal has achieved advanced performance due the recent success of video generative models. However, when addressing the side effects of objects, egg., their shadows and reflections, existing works struggle to eliminate these effects for the scarcity of paired video data as supervision. This paper presents ROSE, termed Remove Objects with Side Effects, a framework that systematically studies the object's effects environment, which can be categorized into five common cases: shadows, reflections, light, translucency and mirror. Given the challenges of curating paired videos exhibiting the aforementioned effects, we leverage 3D rendering engine for synthetic data generation. We carefully construct autoaaiu pipeline for data preparation, which simulates large- scale paired dataset with diverse scenes, objects, shooting angles, and camera video inpainting model built diffusion
trajectories. ROSE is implemented transformer. To localize all object-correlated areas, the entire video is fed into the model for reference-based erasing. Moreover, additional supervision is introduced to explicitly predict the areas affected by side effects, which can be revealed through the differential mask between the paired videos. To fully investigate the model performance on various side effect removal, we presents new benchmark, dubbed ROSE-Bench, incorporating both common scenarios and the five special side effects for comprehensive evaluation. Experimental results demonstrate that ROSE achieves superior performance compared existing video object erasing models and generalizes well to real-world video scenarios. The project page is https://rose202:: inpaint.gi git.gg.thuu...... io
1 Introduction
Removing objects in visual contents represents valuable technique with widespread applications in both daily and industrial scenarios. This task targets to re-fill the masked region of objects via reasonable and consistent content, regarding the context in surrounding environment. Prior towards either image or video object removal have explored to leverage flow- based pixel propagation to restore the masked region with neighboring information 44 or adopt the inpainting paradigm to directly generate the masked content 33 23 Powered by the significant capability targe-scale models (27] 8 23 on generalized visual creation, the inpainting- methods exhibit satisfying crasing performance in diverse scenarios of image and vidco.
works [15] 38
14 3
Despite the advanced performance, however, existing works are still restricted due to the lack of paired training samples that follows real-world physical rules. The paired samples represents data
Project Leader
"Corresoondigg Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Reflection
Light
Trunstocnnt
Figure 1: Video object removal results generated by ROSE (zoom in for better view). Every two lines are an example where the above input video with mask and the bottom is inference result. We sequentially show cases of various side effects studied in this paper.
with and without the object, where the object's 's influence on the environment is correspondingly changed, such as shadow on ground. Most works leverage the segmentation dataset, e,g., DAVIS (26) and YouTube- VOS [40]. to construct artificial pairs, either directly pasting object from another sample, or masking the object with zero value. While simple and scalable, these strategies fail to reflect the side effects of object, e.g., shadows, reflections and lighting changes. Therefore, models supervised by the artificial pairs typically generate unnatural outputs with side effects left in environment. To tackle this, OmniEraser [38] manages to filter out such image pairs from the sequential frames videos with static camera motion. However, when confronting with video object removal, it is impractical leverage higher dimensional data to construct the paired dataset.
To address these problems, we propose prepare the paired video samples via 3D rendering. Recent advancements on the rendering engines 77] make it practical to generate high-qualiff and strictly-alignes synthetic video pairs. We design fully-automatic data preparation pipeline to create large-scale video set for object removal. More concretely, we collect batch of base environments and split them into multiple scenes containing various objects. Following that, the pipeline automatically generates cameras focusing the objects to be removed. and apply random camera trajectories. The rendering engine enables us to activate or disable the object, and also precisely render the object masks. Thus, could obtain list of triples consisting of the original video, edited video with object removed, and the corresponding mask video, which contain perfectly simultaneous temporal contents. Furthermore, systematically study the various types of side effects in videos, including light source, mirror, reflection, shadow, and translucency. Equipped with the data preparation pipeline, we efficiently construct comprehensive dataset including all the above side effects diverse scenes.
To fully utilize the synthetic data, we present ROSE, an efficient framework based on video inpainting to remove object in videos with their side effects. To help distinguish the object-interacted region

---

in environment, we directly feed the whole video into the model, in contrast to previous works that fills the object area with zero mask. The complete video serves as a powerful reference guidance on model, to localize the side effects concerning the intrinsic attributes of the object. We also apply random augmentation strategies on the mask to cope with various input in inference. Furthermore, we introduce an additional supervision to explicitly predict the difference mask between edited and original videos. We implement this by injecting a mask predictor based on the hidden representations of the inpainting model. The aforementioned architectures of ROSE are observed to enhance the model's s capability to attend and erase the side effects in videos.
To facilitate a comprehensive evaluation on the object removal results with side effects, we construct a new benchmark, named ROSE-Bence consisting of both realistic and synthetic video data. Through extensive experiments, we demonstrate that ROSE achieves state-of of-th-of-the-ar performance on video object removal, and effectively adapts to real-world scenarios.
2 Related Work
have shown strong performance in text-to-video generation. By integrating transformers 34 diffusion transformers (25) improve video quality and temporal consistency. State-of-the leverage large-scale video-text datasets (2] 41 and hybrid architectures for efficiency and fidelity. Recent DiT-based latent diffusion models, such as Wan2.1 [35] and MAGI-I [. excel in long video generation: Wan2.1 I uses causal 3D VAEs with 1:256 compression and flow matching for real-time synthesis, while MAGI-I employs an autoregressive DiT for chunk-wise generation with strict causality. These advances underscore DiTs' strength in balancing quality, efficiency, and control.
Diffusion Transformers for Video Generation. Recent diffusion models 10 28
31
32
methods
Video Inpainting. Early video inpainting methods primarily used 3D CNNs 5 36 12 to
model spatial-temporal features, but their limited receptive fields hindered long-range propagation. Subsequently, optical flow [16] 22 [66] and homography [21 4 were introduced to guide pixel propagation. To improve efficiency and accuracy, Zhou et al. 44 proposed ProPainter, combining
optical flow and attention mechanisms. Recently, with the rise of diffusion models [0, 28), diffusion- based video inpainting has emerged 33 20 39
45
29 Li et al. (23 proposed DiffuEraser, extending
the image inpainting model BrushNet 15 to videos via a two-stage training scheme.
Dataset Construction
3.1 Paired Erasing Videos Preparation using 3D data
Acquiring paired data samples that depict scenes with and without objects and their side effects represents significant challenge in object removal task. Though recent work explores generating such image pairs from videos with static camera motion (38), it is impossible to obtain video pairs in higher dimension using this technique. To tackle this problem, we propose to utilize the adequate 3D data together with advanced game engine, i.e., the Unreal Engine 77] to synthesize the paired video data. As illustrated in Fig. we present automatic data preparation pipeline as follows:
Scene and Object Sampling. We begin by collecting large-scale virtual environments from public 3D asset platforms such as Fab (6. Each environment is sufficiently complex and diverse, covering wide range of indoor and outdoor scenes, including urban settings, natural landscapes, and artificial constructions. We manually subdivide these base environments into smaller scenes, each containing one or more candidate objects for removal. In total, we collect 28 high-quality environments and split them into 450 unique scenes. The selected scenes include wide variety of object bots-both static and vynamic-including vehicles, animals, plants, and more. This ensures a diverse training corpus that enhances the generalization ability of the inpainting model.
Multi-view Generation with Object Masks. Given a sampled object in a scene, we randomly assign multiple camera views with varying angles and distances within predefined ranges. A key advantage of using a 3D engine the ability to generate accurate object masks via programmable post-process shaders, avoiding reliance on segmentation models [17. For each object, we apply custom shader that renders the object in white and masks the rest in black, producing precise binary masks. Per-frame mask videos are automatically generated through scripting.
3

---

Scenes Set
Environments Set
split into scenes
sample
Q Select an object
somple
Generate multiple cameros focusing the object
Automatic obtain masks
Comera 1
Comera
Camera
Camerc 4
Final Data Triples
Mask-based Filtering
1. Calculate: Σ picels
render
al pocets
Judge:
schresholdd then remove
traject oryies
Unedited video
Masked video
Edited video
Figure 2: Paired video preparation pipeline using 3D data, which be divided into: and object sampling, multi-view generation with masks, valid view filtering and video data rendering.
Valid View Filtering. To ensure the quality of videos and avoid object-occlusion cases, we further filter out views by calculating the ratio of foreground pixels in the mask. Ratios lower than threshold suggest videos with insufficient mask coverage, e.g., due to occlusion or mislabeling, Such videos are discarded avoid introducing noisy supervision into the training set.
Video Pair Rendering. After filtering, we render both the unedited (original) and edited (object- removed) video sequences by toggling the visibility of the selected objects in the engine. The camera moving is sampled from pre-defined set with random disturbing, e.g., zooming in and out. All video pairs rendered resolution of 1920 1080 and frame length of 90 frames seconds). Since the camera trajectories and object placement are determined via scripted generation, the original video, the corresponding mask video, and the edited video remain spatially and temporally aligned on per-fraane weamframe basis. Such an alignment is critical for enabling pixel-wise supervised learning.
3.2 Categorize Side Effect in Videos
To improve the generalization ability the model and its robustness under various complex real- world conditions, we deliberately construct the dataset composed of six distinct categories. These categories are carefully designed to simulate typical yet challenging side effects that commonly occur in practical scenarios, such object-light interactions, mirror reflections, and translucent materials. By explicitly injecting such variations into the training process, we aim to equip the model with the capacity understand and handle diverse object-environment relationships beyond trivial inpainting cases. We summarize the definition of side effects on the environment as follows:
Common: Objects with minimal interaction with surrounding context, representing typical inpainting cases. Their removal causes little disruption to spatial layout or visual semantics.
Light Source: This category includes objects that function as light emitters. Their removal changes the global illumination, affecting shadows, reflections, and overall scene appearance.
Mask
Inpet
Ground Truth
Common
Light Source
Mirror
Shadow
Figure 3: Illustration of the various side-effect categories studied in the dataset of ROSE.

---

MSELoss
object side effect
Difference Predictor
MLP GELU
Ground bifference
Resize
Diffusion Loss
Input Mask
Block -
Original
Video Inpainting Model
Frozen
Trainable
Concatenate
Subtract
Edited Video
feuatia Noiia
Figure 4: The framework of ROSE. We concatenate the noisy latents with the original input video and masks, consumed by a video inpainting model. An additional difference mask predictor is introduced to predict the correlated area in video, automatically computed from the input video pairs.
Mirror:
Objects reflected in mirrors require spatial reasoning and semantic understanding to inpaint both the object and its mirrored counterpart, ensuring visual consistency.
Reflection: Compared to the Mirror category, it emphasizes reflective surfaces like water, requiring the model to infer and complete indirect visual cues from reflections.
Shadow: Shadows linked objects require joint removal, making inpainting sensitive to lighting and spatial structure to ensure coherence across both object and shadow regions.
Translucent: Semi-transparent objects expose the background with blending refraction. Inpainting must recover both visible cues and hidden structures for realistic restoration.
4 Method
4.1 Overview
In this section, we elaborate the model architecture of ROSE for conducting video object removal, as illustrated in Fig. 4 In brief, ROSE is implemented as an inpainting model continued from the
foundation video generative models [35 [8] (the Wan2.1 model 355 this paper). Following the
general architecture in eeffusiio-basse inpainting models 19 we extend the model input with the original input video together with object masks. Distinguished from the typical setting that multiply the mask onto input video, we directly feed the whole video to assist the understanding on environment. The input masks, with precise boundary generated by 3D engine, are further augmented to enhance model robustness. To better supervise the model to localize the subtle object-environment interactions, we introduce an additional difference mask predictor to explicitly predict the side effect areas. We present the detail of ROSE in the following sub-sections.
4.2 Reference-based Object Erasing
We start by formulate the video inpainting task in ROSE. Given an input video and a binary mask sequence M, where the area of object be removed is filled with value 1, the target is to generate an object-erased video V. For the video condition consumed by the model, most prior paradigm, feeding the network with only the
methods 223 5 44 36) follow "mask-and-inpailn non-object area V (1 M)) where indicates point-wise product. Suppose the noisy latents of diffusion model as X, then the model input can be regarded as [X; V o (1 M); Such manner explicitly eliminate the object from input, and is friendly for model convergence. However, when confronting the side effect removal, isolating the object from the model makes it challenging to localize the object-related region. In contrast, recent work on image modality has explored to guide the model with the masked region for reference
[14]. In this paper, we adapt such reference-basee erasing, modifying the model input as (X; V; M). Experimental results suggest introducing the whole video guidance significantly increase the performance. We attribute the advancement that the inner
5

---

mark video
Imcatemat
Pres loss paraligm
video
Origieal
Itbo
Cirdle
Dilated
Ereded
Our paradigm
Comparison between the previous paradigm and our reference- paradigm.
Figure 6:
Figure 5: Visualization of various mask augmen- tation strategies adopted in training.
attention mechanism is effective for seeking the inter-region correlations in videos. Given the object region input, the model thereby leverage prior knowledge to localize the side effect regions, thus outperforms the model with masked video input. Furthermore, the complete video as input serves to enhance the temporal consistency of output video, for introducing the original object-environment interactions. The visualization comparisons between the two paradigms are shown in Fig.
4.3 Mask Augmentation
In real-world applications, user-provided masks often vary in precision, size, and shape-ranging from accurate segmentation maps coarse bounding boxes sparse point annotations. Since the masks generated by 3D engine is perfectly accurate, training solely on such ideal masks can lead to performance gap at deployment. To mitigate this, introduce set of mask augmentation strategies that simulate diverse mask types likely to appear in practice. As shown in Fig. 5 we adopt five variants: (i) Original mask, precise binary map from ground-truth annotations; (ii) Point-wise mask, an extremely sparse point simulating minimal user input; (iii) Bounding box mask, coarse rectangular region enclosing the target; (iv) Dilated mask, obtained via morphological dilation to simulate loose annotations; and (v) Eroded mask, generated by erosion to mimic under-segmentation. These variants randomly sampled during training, which exposes the model diverse, imperfect masks and improves its generalization to real-world inputs.
4.4 Explicit Supervision via Difference Mask Prediction
Beyond the diffusion loss targeting reconstruct the regions of object and its side effect, we introduce an additional supervision into ROSE. Specifically, inject difference mask predictor into the framework, predicting binary masks indicating all the areas to modified in video.
The core idea is to leverage the complementary information in the video pairs for training. When an object is removed from scene, often leaves behind subtle but semantically significant side effects, such shadows, reflections, and occlusions. To explicitly guide the model in attending to these regions, we compute a binary difference mask by comparing the original video xo E x Xxxx and its edited counterpart No.
The difference mask do {0, 1) fx is defined as:
{1. if xx0
(t.h,w) (t,h,,)) >8
df (t.b.,))
X0
(I)
00 otherwise
where 8 fixed threshold (5 0.09 this paper). The resulting binary mask highlights pixel-level differences induced by object removal and is downsampled to match the latent resolution, yielding the ground-truth difference mask d, E {0, ff.
Difference Mask Predictor. To guide the model in identifying regions influenced by object removal, we design difference mask predictor Do. which takes as concatenated token features as input, extracted from multiple transformer blocks. x E B= denote the fused feature sequence, where I, F, H, x W, represents the total number of tokens and Diotal is the aggregated channel dimension after selecting and concatenating multiple transformer layers. The difference mask predictor consists of two-layer MLP that reduces D Droaal to a scalar prediction per token. Its

---

Masked Frames
ROrEs
Draser
ProPainter
FuseFormer
NZ.ch/oob
Figure 7: Qualitative comparison between our method and existing approaches on real-world samples. Our model demonstrates superior ability and effectively handles complex object-envirommen interactions. including shadows, reflections, and illumination changes.
output then reshaped into 3D
grid with the same shape of video latents:
d Interpolate (Reshape(Po (x),, size (F, H)))
where the predicted mask d, E j0, 1| is upsampled via trilinear interpolation from coarse patch-level grid (Fp, Hp, W,) to the full resolution (F, H, W). The module is trained under MSE loss supervision against the ground-truth difference mask d, described in Eq. (3). It functions as an auxiliary self-localization signal to encourage the model to be sensitive to subtle visual effects introduced by object edits. Then the training objective ROSE consists of two terms: the standard diffusion denoising loss and the auxiliary mask prediction loss:
L ==
||e a|12 + > pl|d d
where balances the two objectives. This formulation enables the difference mask predictor to guide the model in localizing and identifying regions where object-environment interactions occur.
5
Experiments
5.1 Experiment Settings
Training Data. Our dataset contains 16,678 synthetic video pairs rendered in Unreal Engine, each seconds (90 frames) at 1920x resolution. features diverse urban, rural, and natural scenes with dynamic weather, lighting. and interactive objects.

---

Table 1: Quantitative comparison the synthetic paired benchmark (PSNR1 SSIMT LPIPS.)
Category
Metric
ROSE(Ours)
DiffuEraser 23
ProPainter (44
FuseFormer 24
FloED
FGT [2)
PSNR
36.5998
30,9326
31.9972
31.2325
29,8932
28.4331
Common
SSIM
0.9517
0.9204
0,9466
0,9154
0,9066
0.8819
LPIPS
0.0413
0.0825
0.0515
0.0658
0.0738
0.0832
PSNR
33.7876
28.9976
30.2427
28.5520
27 27,8932
27,5809
Shadow
0.9220
0.8972
0.8834
0.8547
LPIPS
0.0626
0.1119
0.0619
0.1035
0.1208
0.1172
PSNR
30,0739
22,6541
23.4291
22.8571
22.3125
21 4579
Light Source
SSIM
0.9209
0.8832
0.8924
0.8630
0.8596
0.8433
0.1403
0.1589
0.1347
PSNR
27.7344
26.2914
26.9373
25.7707
25,1018
24.3986
Reflection
SSIM
0.8715
0.8619
0.8763
0.8345
0.8413
0.8421
LPIPS
0.1129
0.1405
0,1072
0.1437
0.1651
0,1520
PSNR
28.3498
22.1228
22,1206
22.3175
21 21.3789
22.6013
Mirror
SSIM
0.9381
0,8855
0.8994
0.8671
0.8678
0.8594
LPIPS
0.0878
0.1751
0.1447
0.1596
0.1845
0.1653
PSNR
31.4264
28.4520
29.8910
28.1712
27.3924
27,4802
Translucent
SSIM
0.9470
0.9259
0,9397
0,9168
0.8956
0,9034
LPIPS
0.0598
0.1036
0.0722
0.0914
0.1134
0.1210
31.1221
26.5024
27.1991
26.2566
25.4847
25.2353
Mean
SSIM
0.9170
0.8981
0,9148
0.8795
0.8697
0,8641
LPISS
0.0772
0.1284
0.0946
0.1208
0.1324
0.1289
Evaluation Benchmark and Metrics. Existing benchmarks in the video inpainting domain mainly suffer two limitations. First, most of them lack access to paired edited videos following real- world physical rules, which restricts quantitative evaluation due to the absence of ground-truth. Second, they overlook the side effects induced by enverteeniionmmett interactions hat are critical for assessing the semantic correctness and realism of inpainting. Consequently, these benchmarks fail to capture fine-grained challenges that frequently arise in real-world applications.
To address these gaps, we construct ROSE- Bench, a comprehensive evaluation bench- mark on video object removal, consisting of following subsets:
Table 2: Ablation study on ROSE-Bench (PSNR SSIM LPIPSL)
Category
w/ MRG
MA
w/
(i) Synthetic paired benchmark tailored for evaluation under diverse physical interaction effects. Using the same simulation approach described in Sec. 3 the benchmark con- sists of 6 representative categories: common, light source, mirror, reflection, shadow, and translucent, each modeling a specific class of object-environment interaction. Every category contains 10 high-quality triplets of video sequences, i.e., original, edited, and mask videos, offering precise and controllable evaluation of model behavior under different side-effect conditions.
32.58
Common
SSIM
0.937
0.950
0.949
0.943
LPIPS
0.053
0.040
0.046
0.045
Shudow
SSIM
0.14
0.20
0.922
0.200
LPIPS
0,081
0.061
0.072
0.064
Light
SSIM
0.894
0.933
0.911
0.922
LPIPS
0,112
0,074
0,082
0,077
Reflect
SSIM
0.836
0,843
0.815
0.841
LPIPS
0,131
0.109
0.111
0.1
22.63
Mirror
SSIM
0.905
0.941
0.932
0.932
LPIPS
0,142
0.076
0.086
0,092
Translucent
SSIM
0.925
0.949
0,948
0.946
LPIPS
0.087
0,052
0.056
0,059
PSNR
27.28
30.84
29.77
30.82
(ii) Realistic paired benchmark constructed using a copy-and-paste strategy based on the video segmentation dataset dataset DAVIS (26). We copy masked object from one video into another. The resulting video with inserted object is treated as input, while the original unaltered video serves as the ground-truth -ruth. This process allows us to construct realistic and diverse test cases that mirror practical editing scenarios while preserving access to ground-truth supervision. For quantitative evaluation on paired bench- mark, compute PSNR [[]. SSIM (37).
LPPPS
0.101
0.071
0.076
0.074
Table 3: Quantitative comparison on realistic paired benchmark.
Method
PSNR
SSIM
LPIPS
ROSE(Ours)
31.34
0.923
0.092
DiffuEraser 23
29.97
0.128
ProPuinter 44
32.81
0.917
0.122
FuseFormer
26.52
0.885
0.151
FloED M
28.48
0.881
0.147
FGT 42
27.53
0.874
0.135
8

---

and LPIPS 43 across both synthetic and real-world test sets. These metrics capture both low-level structural fidelity and perceptual similarity, assessing the model performance under various side-effect challenges.
(iii) Realistic unpaired benchmark containing real videos with masks. Different from the second subset, we directly feed real-world videos into model, which are also sampled from DAVIS [26]. To conduct evaluation without ground-truth, we select related metries from the VBench [33, widely-adopted benchmark on texi-to-video generation, for evaluating the quality of output videos on motion smoothness, background consistency and temporal flickering.
Implementation Details. In the training process, resize all the video pairs into the resolution of 720 x 480 and use 81 frames for training. The backbone model a controllable generation variant of Wan2.1 1.3B version 35 We fully train the model together with the difference mask predictor in 80000 optimization steps with 0.00002 learning rate on 4 NVIDIA H800 GPUs.
5.2 Comparisons with Previous Methods
Quantitative Evaluation. For guantitative evaluation, we compare our method with flow- based transformers (ProPainter [44], FuseFormer [24]. FGT (42)) and diffusion- based methods (DiffuEraser [23]. FLoED
)). We evaluate all methods on the three components of ROSE-Bench: synthetic paired benchmark (Tab. []. realistic paired benchmark (Tab. B). and real-world videos (Tab. D.. Our model achieves superior performance in object removal, as measured by PSNR [.). SSIM 337, and LPIPS [43]. and excels in maintaining motion smoothness, background consistency, and subject consistency in Tab.
Table 4: VBench-based evaluation the realistic unpaired benchmark. (Best scores are bolded).
Motion Smoothness
Background Consistency
Temporal Flickering
Subject Consistency f
Imaging Quality
Method
DiffuEraser
0.972
0.902
0.931
0.891
0.658
ProPainter
0.975
0.917
0.932
0.903
0,626
FuseFormer
0.971
0.905
0,938
0.892
0,625
FloED
0.973
0.904
0,932
0.889
0.618
FGT
0.971
0.897
0.933
0.895
0.614
ROSE(Ours)
0.975
0.923
0.936
0.908
0.630
Table 5: Inference Efficiency.
Method
All Parameters(M)
Average Runtime(s/trame)
Average GPU Memory(G)
ROSE(Ours)
1564.4
1.39
21.05
DiffuEraser
ProPainter
45
39.4
0,10
11.75
PuseFonner
64.1
0.13
16.30
FloED 9
1346.8
1.63
31.41
PGT 42
42,3
1.43
14.38
Table 6: Ablation studies on the backbone model testing synthetic paired benchmark.
Metries
DiffuEraser before training
DiffuEraser after training
using baseline settings
ROSE(OurS)
PSNRT
26,5024
27.0162
27. 27.2863
31.1221
SSIMT LPIPSL
0,8981
0.9010
0.9025
0.9170
0.1284
0.1106
0.1013
0.0772
Qualitative Evaluation. For qualitatitve evaluation, we compare our method with ProPainter 44
FuseFormer 24 and DiffuEraser (23] Qualitative visualization results can be seen in Fig. In
the Fig. / we demonstrate cases with various different side effects like shadows, reflection and lumination changes and we can obviously find that our model shows superior performance over other methods. The side effects areas that previous works fail to fill in have been framed in red boxes.
Inference Efficiency. Tab. compares the parameter scale, inference latency, and memory footprint across different methods. All evaluations are conducted on 65-frame input videos with a resolution of 720 x 480, using f6oatl6 precision and NVIDIA H800 GPUs.

---

5.3 Ablation Study
We perform ablation studies to demonstrate the effectiveness of our designs. We keep training settings same as in bec. ensure the faimesss of comparisons and we evaluate our methods on synthetic paired benchmark. We set the baseline with the following settings: use the "mask-and- inpaint" paradigm, without mask augmentation and difference mask predictor. And in Tab. 2 MRG stands for mask region guidance. MA stands for mask augmentation and DMP stands for difference mask predictor. In Tab. we have shown that our primary designs are effective and useful.
Also, Tab. 6 reports quantitative results comparing (i) DiffuEraser 23 without training on ROSE- Dataset, (ii) DiffuEraser trained on COSE-Datasse (iii) 1an2.1 35 trained on ROSE-Dataset under baseline settings, and (iv) our proposed method ROSE. The results demonstrate that training on our HOSE-Datase consistently improves overall performance, while the stronger base model further enhances object removal capability.
Discussion
This paper introduces ROSE, unified framework for video object removal that addresses both target objects and their side effects, such as shadows, reflections, and lighting distortions. By leveraging synthetic data from 3D rendering pipeline, we alleviate real-world data scarcity while ensuring diverse scenes and camera motions. Our diffusion transformer architecture excels in object localization and side effect removal via differential mask supervision. The proposed ROSE- Bench offers systematic evaluation for object-environment interactions, addressing key gap in video inpainting. Extensive experiments show that ROSE significantly outperforms prior methods and generalizes well to real-world videos. These contributions advance video editing and set new benchmarks for handling complex visual artifacts. Future work will explore real-time optimization and broader environmental effects to further bridge synthetic and real-worke domains. Despite its strengths, ROSE has limitations: (1) It may produce flickering artifacts under large motion, as shown in Tab. (2) Inference time grows with video length, redueing efficiency on long sequences.
References
[1] Sand AI. Magi-1: Autoregressive video generation at scale. http:///gittuub..o/casoeetaa MAGI-1 1 2025.
[2] Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: rertesstttoonnt for vision and text using cross-modal contrastive learning. In NeurlPS, 2021.
[3] Yuxuan Bian, Zhaoyang Zhang, Xuan Ju, Mingdeng Cao, Liangbin Xie, Ying Shan, and Qiang Xu. Videopainter: Any-length video inpainting and editing with plug-and-play context control. arXiv preprint axXiv:2503005639 2025.
[4] Jiayin Cai. Changlin Li. Xin Tao, Chun Yuan, and Yu- Yu-Wing Tai. Devit: Deformed vision transformers in video inpainting. In ACM MM, 2022.
[5] Ya- Liani Chang, Zhe Yu Liu, Kuan- Ying Lee, and Winston Hsu. Free-form video inpainting with 3d gated convolution and temporal patchgan. In /CCV, 2019.
[6] Epic Games. Fab, httpp::///// hwtp:://wwww........ com/ 2024.
[7] Epic Games. Unreal engine 5.3. httpp://wwws unreal www.alnngennn.e....m com/ 2024.
[8] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. ICML, 2024.
[9] Bohai Gu, Hao Luo, Song Guo, and Peiran Dong. Advanced video inpainting using optical flow-guided efficient diffusion. arXiv preprint 472007241 2024.
[10] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurlPS, 2020.
[11] Alain Hore and Djemel Ziou. Image quality metries: Psnr ssim. In ICPR, 2010.
10

---

[12] Yuan-Ting Hu, Heng Wang. Nicolas Ballas, Kristen Grauman, and Alexander G. Schwing, Proposal-based video completion. In ECCV, 2020.
[13] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024.
[14] Longtao Jiang, Zhendong Wang, Jianmin Bao, Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, and Houqiang Li. Smarteraser: Remove anything from images using masked-region guidance. In CVPR, 2025.
[15] Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu, Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion. In ECCV, 2024.
[16] Dahun Kim, Sanghyun Woo, Joon- Young Lee, and In So Kweon. Deep video inpainting, In CVPR, 2019.
[17] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan-Yes Yen Lo, al. Segment anything. In CVPR, 2023.
[18] Weijie Kong. Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, Kathrina Wu, Qin Lin, Junkun Yuan, Yanxin Long, Aladdin Wang, Andong Wang. Changlin Li, Duojun Huang, Fang Yang. Hao Tan, Hongmei Wang. Jacob Song, Jiawang Bai, Jianbing Wu, Jinbao Xue, Joey Wang, Kai Wang. Mengyang Liu, Pengyu Li, Shuai Li, Weiyan Wang. Wenging Yu, Xinchi Deng, Yang Li, Yi Chen, Yutao Cui, Yuanbo Peng, Zhentao Yu, Zhiyu He, Zhiyong Xu, Zixiang Zhou, Zunnan Xu, Yangyu Tao, Qinglin Lu, Songtao Liu, Dax Zhou, Hongfa Wang, Yong Yang. Di Wang. Yuhong Liu, Jiang, and Caesar Zhong. Hunyuanvideo: A systematic framework for large video generative models, 2025.
[19] Black Forest Labs. Flux. http:////tttuubbccmm//aaaaerrrrs
2024.
[20] Minhyeok Lee, Suhwan Cho, Chajin Shin, Jungho Lee, Sunghun Yang, and Sangyoun Lee. Video diffusson models are strong video inpainter. arXiv preprint arXiv:2408. /1402, 2024.
[21] Sungho Lee, Scoung Wug Oh, DaeYeun Won, and Scon Joo Kim. Copy-and-paske networks for deep video inpainting. In ICCV, 2019.
[22] Ang Li, Shanshan Zhao, Xingjun Ma, Mingming Gong. Jianzhong Qi, Rui Zhang, Dacheng Tao, and Ramamohanarao Kotagiri. Short-term and long-tenn tor--term context aggregation network for video inpainting. In ECCV, 2020.
[23] Xiaowen Li, Haolan Xue, Peiran Ren, and Liefeng Bo. Diffueraser: A diffusion model for video inpainting. 2025.
[24] Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu Sun, Xiaogang Wang, Jifeng Dai, and Hongsheng Li. Fuseformer: Fusing fine-graines information in transformers for vidco inpainting. In ICCV, 2021.
[25] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
[26] Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Korkine-Homung. benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016.
[27] Dustin Podell, Zion English, Kyle Laccy, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. 2023.
[28] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffuusin models. CVPR, 2022.
[29] Dvir Samuel, Matan Levy, Nir Darshan, Gal Chechik, and Rami Ben-Ari. Omnimattezero: Fast training- free omnimatte with pre-trained video diffusion models. arXiv preprint orXiv:2503. 18033, 2025.
[30] Fengyuan Shi, Jiaxi Gu, Hang Xu, Songcen Xu, Wei Zhang, and Limin Wang. Bivdiff: A training-free framework for general-purpose video synthesis via bridging image and video diffusion models. In CVPR, 2024.
[31] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. arXiv preprini arXiv:1503.03585, 2015.
[32] Yang Song, Jascha Sohl-Dit Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differentiall equations. In ICLR, 2021.
11

---

[33] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov. Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky. Resolution-robus! large mask inpainting with fourier convolutions. arXiv preprini urXi::2109.07661 2021.
[34] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention you need. NeurIPS, 2017.
[35] Wan Video. Wan: Open and advanced large-scale video generative models. htttpa//ggithbb. Wan- Video/Wan2.11 2025.
[36] Chuan Wang. Haibin Huang, Xiaoguang Han, and Jue Wang. Video inpainting by jointly learning temporal structure and spatial details. AAAI, 2019.
[37] Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From visibility to structural similarity. TIP, 2004.
[38] Runpu Wei, Zijin Yin, Shuo Zhang, Lanxiang Zhou, Xueyi Wang. Chao Ban, Tianwei Cao. Hao Sun, Zhongjiang He, Kongming Liang, and Zhanyu Ma. Omnieraser: Remove objects and their effects in images with paired video-frame data. arXiv preprint Stti::2501.0399)) 2025.
[39] Jianzong Wu, Xiangtai Li, Chenyang Si, Shangchen Zhou, Jingkang Yang, Jiangning Zhang, Yining Li. Kai Chen, Yunhai Tong, Ziwei Liu, et al. Towards language-driven video inpainting via multimodal large language models. arXiv preprini arXiv:2401. 212224411.022,, 2024.
[40] Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang, Dingcheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: 2018.
video object segmentation. In ECCV,
[41] Yue Xu, Jiabo Ye, Yifan Xu, Hang Zhou, Wayne Wu, and Ziwei Liu. Video-Ilava: Learning multimodal vidco instruction-oollownngg In EMNLP, 2023.
[42] Kaidong Zhang, Jingjing Fu, and Dong Liu. Flow-guided transformer for video inpainting. In ECCV, 2022.
[43] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metrie. In CVPR, 2018
[44] Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, and Chen Change Loy. Propainter: Improving propagation and transformer for video inpainting. In ICCV, 2023.
[45] Bojia Zi, Shihao Zhao, Xianbiao Qi, Jianan Wang. Yukai Shi, Qianyu Chen, Bin Liang, Kam-Fai Wong. and Lei Zhang. Cococo: Improving text-guided video inpainting for better consistency, controllability and compatibility. In AAAI, 2025.
[46] Xueyan Zou, Linjie Yang. Ding Liu, and Yong Jae Lee. Progressive temporal feature alignment network for video inpainting. In CVPR, 2021.
12

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made in the abstract and introduction accurately summarize the contributions of the paper. Specifically, the paper proposes novel framework that improves vidco inpainting quality by introducing Difference Mask Predictor Mask Region Guidance. These components are explicitly mentioned in the abstract and are substantiated by both qualitative and quantitative experiments. The scope and generalizability are clearly discussed with respect to diverse scenarios in the benchmark.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The paper includes dedicated Limitations in Sec. It acknowledges that current model still has some temporal flickering problems and the inference time increases with the video length.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed the paper.
The authors encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results these assumptions (e.g., independence assumptions, noiseless settings, model well well-speeine specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
violations of
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should he articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection,
outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
Question:
Answer: [Yes]
13

---

The paper includes a theoretical formulation of the loss functions and optimization objectives, all of which are fully stated with the necessary assumptions. While formal theorems proved, the mathematical foundations are complete and clearly documented either the main text or supplemental material.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should clearly stated or referenced in the statement of any theorems.
The proofs either appear the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of paper should be complemented by
formal proof' provided in appendix or supplemental material.
Theorems and Lemmas the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper fully discloses the architectural details, dataset preprocessing steps, training schedules, loss functions, and evaluation protocols used produce the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe steps taken make their results reproducible or verifiable.
Depending on contribution, reproducibility can accomplished various ways. example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data often one good way accomplish this, but reproducibility can also be provided via detailed instructions access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
how to replicate the results,
While NeurIPS does not require relcasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should deseribe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (c.g.. with open- open-source datasct instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does paper provide open access to the data code. with sufficient instructions to faithfully reproduce the main experimental results, as described supplemental material?
Answer: [Yes]
Justification:
The code and pre- trained models will be released upon acceptance, and the supplemental material contains detailed instructions to run experiments. including dataset preparation. model training, and evaluation. An anonymized GitHub link is included for reviewing purposes.
14

---

Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines ftttps://nips.cc/pbbbe
for more details.
While we encourage the release of code and data, understand that this might not be possible,
so "No" an acceptable answer. Papers cannot rejected simply for not including code, unless this is central to the contribution (e.g.. for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:/ htmps://nis.... cc/public/ StlpeesCCoSSuuumissseeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental sefting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: All relevant details regarding training (e.g., data splits, learning rate, optimizer, hatch size, and number epochs) and evaluation (e.g., metries, checkpoints) are clearly described Sec. This allows readers to understand and replicate the experimental protocol.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance the experiments?
Answer: [No]
Justification: We follow the convention prior works and report the performance on the standard settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error hars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should he explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error hars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should he careful not to show in tables or figures
symmetric error hars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
15

---

8. Experiments compute resourees
Question: each experiment, does paper provide sufficient information on computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: The paper specifies the compute infrastructure used, the number of training epochs, and the total training time. An estimated compute cost for each model variant is also provided in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether full research project required more compute than experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https:://eurr:: http:/ppp...ppplll/////heeeo...e
Answer: [Yes]
Justification: The research adheres the NeurlPS Code of Ethics. All datasets used publicly available do not involve personally identifiable information or sensitive content. human or animal subjects were involved.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss both of them in supplementary materials.
Guidelines:
answer means that is no societal impact of work performed. the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), faimess considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point out. For example, the quality generative models could used to generate deepfakes for disinformation. On other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
legitimate to point out that an improvement in
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional unintentional) misuse the technology.
there are negative societal impacts, the authors could discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
16

---

11. Safeguards
Does the paper describe safeguards that been put in place responsible release data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We describe the safeguards in the Appendix.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
original owners of assets (e.g., code, data, models), used in the paper, properly credited and are license and terms of use explicitly mentioned and properly respected?
Question: Are the
Answer: [Yes]
Justification: datasets and codehases used are properly cited and used under their respective licenses. License information is included in the supplemental material.
Guidelines:
The answer NA means that the paper does not use existing assets.
authors should cite the original paper that produced the code package or dataset.
authors should state which version the asset is used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets released, the license, copyright information, and terms of use in the package should he provided. For popular datasets, papprswwi thcode. danaetattttt has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
existing datasets that are re-packaged, both original license and license of derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged reach out to asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The newly introduced benchmark subset and pre-trained models are fully documented and will be made available with metadata, license information, and usage examples. Documentation is provided in the supplemental material.
Guidelines:
The answer NA means that the paper does not release Rescarchers should communicate the details of the dataset/code/model as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
17

---

Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer means that paper does not involve crowdsourcing nor research with human subjects.
Including this information in supplemental material is fine, but main contribution of paper involves human subjects, then as much detail as possible should he included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals an equivalent approval/review based on requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer means that paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe usage of LLMs it an important, original. or non standard component of the core methods in this research? Note that if the LLM used only for writing. editing. or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is required.
Answer: [NA]
Justification: This paper does not involve LLMs any important components.
Guidelines:
The answer NA means that the method development in this rescarch does not involve LLMs any important, original, or non-standard components.
Please refer LLM policy (https:/ http://ppp...e//ommeeeeeee////// for what should or should described.
18

---

A Dataset Details.
In the data preparation process, we synthesize video sequences within the Unreal Engine (UE) environment. The virtual cameras are programmatically controlled automatically focus the target object designated for removal. To simulate realistic motion, the cameras move randomly along one of the three principal axes -X, Y, or Z-in the UE world coordinate system. The movement distance along the chosen axis is adaptively adjusted according to the spatial scale and available space within each specific seene, ensuring plausible camera trajectories without clipping or unnatural transitions. By rendering the scene under these controlled camera motions, obtain synchronized video triplets consisting of the original unedited video, the corresponding masked video with the target object occluded, and the edited video where the object is removed.
B Training Details.
During training. we use a batch size of and randomly select a continuous sequence of 81 frames from triplets the original, masked, and edited vidcos as input. We extract and concatenate features from Transformer layers block.5, block. and Block.25(30 blocks in total). resulting in a feature tensor of shape [1, 28350, 4608), where 28350 corresponds to the flattened patch grid (c.g.. 10 x 15 x 189) and 4608 is the combined feature dimension from the three layers. The Difference Mask Predictor consists of linear layer projecting the 4608- dimensional features to 256-dimensional hidden space, followed by a GELU activation and final linear layer mapping to single output channel. The output, initially [1, 28350, 1]. is reshaped to [1, 1, 10, 15, 189] and then upsampled via trilinear interpolation to the full resolution [1, 1, 81, 480, 720). matching the ground ground-truth difference mask size. This design enables efficient patch-wir wise prediction of spatiotemporal difference masks, providing fine-g grained supervision for the video editing task.
C Potential impacts.
This work expected significantly contribute the field of video object removal by providing a more robust and effective framework for erasing undesired regions in video sequences. By greatly enhancing the quality, temporal consistency, and semantic coherence of the inpainting results, it will not only push the boundaries of current vidco inpainting techniques but also offer a solid foundation for future rescarch and practical applications in video editing, surveillance anonymization, and content restoration.
Positive Impact.
Negative Impact. Video inpainting technology, while powerful for restoring or editing visual content, also bring about negative impacts such as facilitating misinformation through realistic content manipulation, undermining the credibility of vidco evidence in legal and forensic contexts, and raising ethical concerns regarding privacy and consent. If misused, it may distort historical records, violate intellectual property rights, propagate biased or misleading visual narratives, posing serious risks to information integrity and social trust.
D More Visualization Results.
In Figs. 8 to 10 we provide additional qualitative results to further demonstrate the generalization ability of our model. Specifically, Fig. presents more representative examples under common scenarios. Fig. shoows cases where the imput masks are imperfect, illustrating that our model remains robust and can still generate plausible results. Furthermore, Fig. I0 includes results on unseon domains such as underwater, mobile, and drone footage, revealing the model's ' strong adaptability to diverse real-world conditions.
E
More Comparison Visualization Results.
In Fig. II
we present more comparison results of our model with DiffuEraser 23 ProPainter
44
and
FuseFormer (24]
F Some Design Thinkings.
Applicability of Mask Region Guidance (MRG) to Other Models. The motivation of MRG is well justified by its significant improvements shown in Tab. 2 To further examine its generality, we initially considered applying MRG to other representative video inpainting models, such as ProPainter (44 and DiffuEraser (33 However. we found that both models are structurally incompatible with our MRG paradigm, which makes a fair comparison infeasible. ProPuinter relies on an image propagation mechanism that requires a masked video as input. This pipeline design fundamentally conflicts with the pixel-level regional supervision of MRG, features. Consequently, adapting ProPainter to MRG would require
which operates explicit
19

---

Figure 8: More visualizations.
re-engincering its propagation module, rendering it impractical for our framework. DiffuEraser, on the other hand, employs dual-branch architecture built upon pretrained image inpainting network (BrushNet). To integrate MRG into DiffuEraser, must redesign BrushNet's architecture and introduce additional training objectives mask-region alignment. Such modifications would alter the model's intrinsic inductive biases and deviate from its original optimization setup, preventing meaningful comparison.
Given these architectural and methodological constraints, we decided not to pursue further experiments with these two models. Nonetheless. our investigation highlights that MRG is conceptually orthogonal and could, in principle, be generalized to future architectures designed with explicit mask aware conditioning.
20

---

Figure 9: Cases about dealing with imperfect mask (The imperfect parts have been marked by red box. Zoom in for better view.)
G Safeguards
All data used in this work are synthetically generated within Unreal Engine, containing no real human, biometric, or copyrighted content. To prevent potential misuse, we plan to release the ROSE model and dataset under a research-only license. Model checkpoints will be accessible only to verified academic users who agree to responsible-use terms. We explicitly prohibit applications related to deepfake creation, disinformation, any malicious video manipulation.
21

---

Figure 10: More unseen domain cases like underwater, mobile and drone footage.
22

---

ROSE(OurS) Misked Frnmes
Poofainer
Fese
Maskel Frants
ROSE(()un)
DifaEraaz
ProPainter
Fune Fermer
Figure 11: More comparisons.
23


Reviews:
[{"review_id": "wjT2gm8Dwx", "reviewer": "Reviewer_kwEn", "summary": "This paper introduces **ROSE (Remove Objects with Side Effects)**, a framework for video object removal that explicitly models and removes object-induced visual side effects such as shadows, reflections, lighting changes, and translucency. ROSE combines a novel synthetic dataset generation pipeline using Unreal Engine with a reference-based inpainting model that incorporates a difference mask predictor to guide side-effect localization. The authors also present **ROSE-Bench**, a benchmark tailored for evaluating side-effect-aware object removal, and demonstrate that ROSE significantly outperforms prior methods across synthetic and real-world datasets.", "questions": "1. **Domain Gap**: How does ROSE's performance degrade when applied to uncurated, real-world videos with side effects outside the defined categories? Have you considered domain adaptation techniques to bridge this gap?\n2. **Failure Cases**: Can you provide quantitative metrics and visual examples of failure modes, such as flickering or artifacts in scenes with large motion or complex lighting?\n3. **Difference Mask Evaluation**: What is the standalone accuracy or effectiveness of the difference mask predictor across different types of side effects?\n4. **Inference Efficiency**: What are the model’s runtime characteristics (e.g., FPS, memory usage) on long videos? Are there strategies for reducing compute requirements?\n5. **Generalizability**: How transferable is ROSE to new video domains, such as mobile or drone footage, or to settings with unseen environmental side effects?", "limitations": "Yes, the authors shortlist main limitations and societal impacts.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n* **Timely Problem Formulation**: The paper targets a practical and underexplored problem (removing not only objects but their visible environmental effects) which is both novel and relevant.\n* **Robust Synthetic Dataset**: The large-scale, automatically generated dataset provides high-quality, temporally aligned video pairs, enabling strong supervision for training and evaluation.\n* **Architectural Design**: The use of full-reference video input and a difference mask predictor enhances temporal consistency and helps isolate object-related changes effectively.\n* **Comprehensive Evaluation**: Experiments span synthetic, realistic, and unpaired real-world videos, with consistent improvements across standard metrics and strong ablation studies that isolate component contributions.\n\n**Weaknesses:**\n\n* **Limited Real-World Validation**: The model’s performance on truly in-the-wild videos is insufficiently analyzed. Quantitative comparisons for real-world unpaired settings are lacking, and failure modes, particularly flickering under motion, are noted but not deeply explored.\n* **Computational Demands**: Both training and inference are resource-intensive due to the diffusion-based backbone and large-scale rendering pipeline. Practical deployment concerns are not addressed.\n* **Ablation Gaps**: While key components are evaluated, there is limited analysis of the difference mask predictor’s standalone effectiveness or its per-category contributions. The backbone model’s role is also not isolated.\n* **Unclear Sim-to-Real Transfer**: The domain gap between synthetic training data and complex real-world scenarios is acknowledged but not quantitatively assessed. The generalization capacity of ROSE remains uncertain.", "was_revised": true, "final_justification": "The authors' rebuttal and the subsequent discussion have strengthened the paper. They successfully resolved my concerns regarding computational costs by providing concrete inference metrics. They also addressed questions about generalizability and failure modes by supplying new quantitative VBench data for new domains and known failure cases.\n\nThe primary unresolved issue is the scope of the \"in-the-wild\" evaluation. The authors' claim that the limitations of their \"copy-and-paste\" benchmark are not significant remains an unsupported assertion.\n\nDespite this, the paper's strengths and the new data provided are sufficient to warrant a recommendation for acceptance. The unresolved point is a notable limitation but does not invalidate the core contributions.", "rebuttal": "Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.\n\n**W1 & Q2.** **Limited Real-World Validation: The model’s performance on truly in-the-wild videos is insufficiently analyzed. Quantitative comparisons for real-world unpaired settings are lacking, and failure modes,  particularly flickering under motion, are noted but not deeply explored.**\n\nIn Tab.4, we have evaluated real-world unpaired data with quantitative metrics from VBench. Since the real-world unpaired dataset lacks the ground-truth edited video, we can’t evaluate with metrics like PSNR, SSIM and LPIPS. To comprehensively evaluate ROSE’s performance on real-world scenarios, we constructed real-world paired dataset as mentioned in Sec.5.1 which has the ground-truth edited video we needed for evaluation. And we apologize for that currently we can't provide some visual examples of failure where the videos exist flickering under very large motion for the requirements of conference committee. Later we'll supplement more examples including failure cases in the paper and our open-source project page.\n\n**W2 & Q4. Computational Demands: Both training and inference are resource-intensive due to the diffusion-based backbone and large-scale rendering pipeline. Practical deployment concerns are not addressed. What are the model’s runtime characteristics (e.g., FPS, memory usage) on long videos? Are there strategies for reducing compute requirements?**\n\nFirstly, in practical deployment scenarios, we evaluated our model on a single NVIDIA H800 GPU. For a 6-second video (15 fps, 720×480 resolution) using float16 dtype, the inference process requires approximately 20-22GB of VRAM, with a per-frame inference time of 1.39 seconds.\n\nSecondly, the rendering pipeline is specifically designed for dataset preparation and does not contribute to runtime computational costs. Rendering a 6-second video (15 fps, 1920×1080 resolution) takes approximately 9 seconds.\n\nThirdly, detailed runtime characteristics and comparative benchmarks are presented in Table below. While diffusion-based mechanisms are widely adopted for their superior generation capabilities, our model demonstrates competitive performance in real-world applications. As shown in the table, our model's per-frame inference time and memory usage are comparable to current state-of-the-art inpainting models, making it practical for production environments. And in real-world applications, we can adopt temporal tiling window to address longer videos and the processing time is proportional to the frames.\n\nSettings: input video is 65 frames with resolution of 720*480; GPU is NVIDIA H800; data type is torch.float16\n\n| **Inference Efficiency** |  |  |  |\n| :---: | :---: | :---: | :---: |\n|  | **All Parameters(M)** | **Average Runtime(s/frame)** | **Average GPU Memory(G)** |\n| **DiffuEraser** | 1952.3 | 1.02 | 26.12 |\n| **ProPainter** | 39.4 | 0.10 | 11.75 |\n| **FuseFormer** | 64.1 | 0.13 | 16.30 |\n| **FloED** | 1346.8 | 1.63 | 31.41 |\n| **FGT** | 42.3 | 1.43 | 14.38 |\n| **ROSE(Ours)** | 1564.4 | 1.39 | 21.05 |\n\n**W3 & Q3. Ablation Gaps: While key components are evaluated, there is limited analysis of the difference mask predictor’s standalone effectiveness or its per-category contributions. What is the standalone accuracy or effectiveness of the difference mask predictor across different types of side effects? The backbone model’s role is also not isolated.**\n\nThank you for your valuable feedback. We sincerely apologize for the lack of clarity in our explanation of **Tab.1**. To clarify, the results presented in the table are **cumulative**, meaning that each module, i.e. **MRG,** **MA, and DMP**, was added sequentially. Therefore, the specific contribution of the **DMP module** can be observed by comparing the last two columns of the table.\n\nFirstly, to further demonstrate the **standalone accuracy and effectiveness** of the DMP module, we conducted additional experiments, as shown in the first table below. The baseline setting employs the *\"mask-and-inpaint\"* paradigm without incorporating any of the proposed modules or strategies.\n\nFrom the experimental results, we can clearly observe the quantitative improvements achieved by introducing the difference mask predictor (DMP) module to supervise the training process. This further validates the effectiveness of our proposed approach.\n\nSecondly, to validate the impact of our synthetic dataset and assess the role of the backbone model, we conducted an ablation study on DiffuEraser. Specifically, we compared its performance before and after training with our dataset. The results (shown in the second table below) indicate that while our dataset contributes to measurable improvements in DiffuEraser’s performance, the overall enhancement remains limited. This observation motivated us to explore a more robust backbone model to achieve higher performance.\n\n**Table 1: Ablation studies on each module**\n| **Category** | **Metrics** | **Base** | **w/ MRG** | **w/ MA** | **w/ DMP** |\n| :---: | :---: | :---: | :---: | :---: | :---: |\n| Common | PSNR | 32.58 | 35.24 | 33.54 | 35.68 |\n|  | SSIM | 0.937 | 0.950 | 0.949 | 0.943 |\n|  | LPIPS | 0.053 | 0.040 | 0.046 | 0.045 |\n| Shadow | PSNR | 30.65 | 33.29 | 31.63 | 32.85 |\n|  | SSIM | 0.914 | 0.920 | 0.922 | 0.920 |\n|  | LPIPS | 0.081 | 0.061 | 0.072 | 0.064 |\n| Light | PSNR | 24.99 | 30.37 | 28.89 | 30.13 |\n|  | SSIM | 0.894 | 0.923 | 0.911 | 0.922 |\n|  | LPIPS | 0.112 | 0.074 | 0.082 | 0.077 |\n| Reflection | PSNR | 25.39 | 27.71 | 26.97 | 27.41 |\n|  | SSIM | 0.836 | 0.843 | 0.845 | 0.841 |\n|  | LPIPS | 0.131 | 0.109 | 0.111 | 0.110 |\n| Mirror | PSNR | 22.63 | 28.45 | 26.50 | 27.65 |\n|  | SSIM | 0.905 | 0.941 | 0.932 | 0.932 |\n|  | LPIPS | 0.142 | 0.076 | 0.086 | 0.092 |\n| Translucent | PSNR | 27.43 | 30.98 | 31.14 | 31.24 |\n|  | SSIM | 0.925 | 0.949 | 0.948 | 0.946 |\n|  | LPIPS | 0.087 | 0.052 | 0.056 | 0.059 |\n| **Mean** | PSNR | 27.28 | 30.84 | 29.77 | 30.82 |\n|  | SSIM | 0.902 | 0.918 | 0.916 | 0.917 |\n|  | LPIPS | 0.101 | 0.071 | 0.076 | 0.074 |\n\n**Table 2: Ablation studies on the backbone model**\n|  | **Metrics** | **DiffuEraser** **Before training** | **DiffuEraser** **After training** | **Wan using baseline settings** |\n| :---: | :---: | :---: | :---: | :---: |\n| **Mean** | PSNR | 26.5024 | 27.0162 | **27.2863** |\n|  | SSIM | 0.8981 | 0.9010 | **0.9025** |\n|  | LPIPS | 0.1284 | 0.1106 | **0.1013** |\n\n**W4. Unclear Sim-to-Real Transfer: The domain gap between synthetic training data and complex real-world scenarios is acknowledged but not quantitatively assessed. The generalization capacity of ROSE remains uncertain.**\n\nWe acknowledge the potential domain gap between synthetic training data and complex real-world scenarios. However, our model demonstrates strong generalization capabilities in the diverse real-world scenarios we tested.\n\nTo rigorously assess our model’s real-world performance, we conducted quantitative evaluations using both paired and unpaired real-world data:\n\n- Paired Data (Tab. 2): Using a \"paste-and-copy\" approach to construct paired real-world data, we measured the model’s ability to remove objects and inpaint the affected regions. The metrics confirm high performance in these tasks.\n- Unpaired Data (Tab. 4): For scenarios lacking ground-truth videos, we employed VBench metrics (e.g., video quality, temporal consistency) to evaluate the model’s output. These results further validate our model’s robustness and generalization capacity in real-world domains.\n\nWhile the domain gap remains a challenge in sim-to-real transfer, our extensive visual and quantitative results demonstrate that ROSE generalizes effectively to the tested real-world scenarios. We will continue to explore broader conditions to further bridge this gap.\n\n**Q1. Domain Gap: How does ROSE’s performance degrade when applied to uncurated, real-world videos with side effects outside the defined categories? Have you considered domain adaptation techniques to bridge this gap?**\n\nThank you for your insightful question regarding the domain gap and ROSE's performance on uncurated real-world videos.\n\nFirst, as demonstrated in Fig. 1 and Fig. 7, **ROSE exhibits strong performance across both synthetic and real-world domains**, outperforming baseline models in most scenarios. During testing, we observed that the model adapts well to real-world videos, though it encounters challenges with highly complex side effects **such as ripples** (Fig. 1, rows 2–4) where the results, while improved, may not be flawless. \n\nWhile we have not yet implemented advanced domain adaptation techniques, we are open to suggestions. Your recommendations on specific methods to bridge this gap would be highly valuable.\n\nWe appreciate your feedback and would welcome any further insights you might have.\n\n**Q5. Generalizability: How transferable is ROSE to new video domains, such as mobile or drone footage, or to settings with unseen environmental side effects?**\n\nThanks for your suggestion. First, we evaluate ROSE across more real-world video domains, including mobile, drone footage, and underwater scenes. The results indicate robust performance of ROSE in these scenarios. Due to conference committee requirements, we are currently unable to share visualization results, and will present them in the revised version.\n\nSecond, while the model handles known side effects effectively, its performance on unseen types remains an area for further exploration. For example, when dealing with the scenarios where objects have more complex physical interactions with the environment like ripples generated by a duck and water vapor generated by hot food, our model may fail to perceive those intricate interactions and thus can't remove those side effects together with objects. But with current experience, we will include such limitations in revised version, and will leverage a wider range of synthetic technologies to address these issues in future works."}, {"review_id": "qJPkiNCy1J", "reviewer": "Reviewer_12Y3", "summary": "The paper proposes a video object removal framework that also removes associated side effects like shadows, reflections, and lighting. To address the lack of real paired data, the authors use a 3D rendering engine to generate synthetic video pairs. ROSE is a diffusion-based inpainting model that uses full video context and introduces a difference mask predictor for explicit supervision. A new benchmark, ROSE-Bench, is created to evaluate various side-effect scenarios. Experiments show that ROSE outperforms prior methods in both accuracy and realism.", "questions": "See weakness", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1.The paper addresses a practical  problem in video inpainting—removing not just objects but also their environmental side effects such as shadows and reflections. Its approach is well-motivated, with a clear design that combines a diffusion-based inpainting model, full video context input, and a difference mask predictor for explicit supervision\n\n2.The use of a 3D rendering pipeline to generate paired synthetic videos is effective and well-documented. Additionally, the introduction of ROSE-Bench provides a useful benchmark for evaluating side-effect removal. \n\n3. This paper is well-written and easy-to-follow.\n\n**Weakness**\n\n1. While the paper presents strong quantitative and qualitative comparisons, it lacks video-based results, which are crucial for evaluating temporal consistency and perceptual quality in video inpainting. The current comparisons rely mostly on metrics and static frame visualizations, which may not fully reflect performance on dynamic content. Including sample videos would strengthen the evaluation.\n\n2. Additionally, the paper does not report inference time or model size, making it difficult to assess the method’s efficiency and scalability compared to prior work.\n\n3. To better validate the effectiveness of the constructed dataset, the authors should consider training baseline video inpainting models (e.g., DiffuEraser or ProPainter) on the same synthetic data and comparing their improvements. Since ROSE builds on top of Wan, a powerful generative model, it's unclear how much of the performance gain comes from the dataset versus the backbone.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.\n\n**W1. While the paper presents strong quantitative and qualitative comparisons, it lacks video-based results, which are crucial for evaluating temporal consistency and perceptual quality in video inpainting. The current comparisons rely mostly on metrics and static frame visualizations, which may not fully reflect performance on dynamic content. Including sample videos would strengthen the evaluation.**\n\nThanks for your suggestions. We mainly show temporal sequential results in Fig.1 of main paper and Fig.1&2 in supplementary material, which could briefly reflect the erasing performance on dynamic content. These frame sequences demonstrate temporal consistency of ROSE's outputs and high-qualified video reconstruction. Due to conference committee requirements, we are unable to share full video demonstrations now, and will present more results in both revised version and open-sourced project page. We appreciate your understanding and welcome further discussion on these results.\n\n**W2. Additionally, the paper does not report inference time or model size, making it difficult to assess the method’s efficiency and scalability compared to prior work.**\n\nFirstly, in practical deployment scenarios, we evaluated our model on a single NVIDIA H800 GPU. For a 6-second video (15 fps, 720×480 resolution) using float16 dtype, the inference process requires approximately 20-22GB of VRAM, with a per-frame inference time of 1.39 seconds.\n\nSecondly, detailed runtime characteristics and comparative benchmarks are presented in Table below. While diffusion-based mechanisms are widely adopted for their superior generation capabilities, our model demonstrates competitive performance in real-world applications. As shown in the table, our model's per-frame inference time and memory usage are comparable to current state-of-the-art inpainting models, making it practical for production environments. And in real-world applications, we can adopt temporal tiling window to address longer videos and the processing time is proportional to the frames.\n\nSettings: input video is 65 frames with resolution of 720*480; GPU is NVIDIA H800; data type is torch.float16\n\n| **Inference Efficiency** |   |   |   |\n| :---: | :---: | :---: | :---: |\n|  | **All Parameters(M)** | **Average Runtime(s/frame)** | **Average GPU Memory(G)** |\n| **DiffuEraser** | 1952.3 | 1.02 | 26.12 |\n| **ProPainter** | 39.4 | 0.10 | 11.75 |\n| **FuseFormer** | 64.1 | 0.13 | 16.30 |\n| **FloED** | 1346.8 | 1.63 | 31.41 |\n| **FGT** | 42.3 | 1.43 | 14.38 |\n| **ROSE(Ours)** | 1564.4 | 1.39 | 21.05 |\n\n**W3. To better validate the effectiveness of the constructed dataset, the authors should consider training baseline video inpainting models (e.g., DiffuEraser or ProPainter) on the same synthetic data and comparing their improvements. Since ROSE builds on top of Wan, a powerful generative model, it’s unclear how much of the performance gain comes from the dataset versus the backbone.**\n\nThank you for your valuable suggestion. To validate the effectiveness of our synthetic dataset, we conducted **preliminary experiments by training DiffuEraser** on the same data and comparing its performance with the original model. The results demonstrated that the trained DiffuEraser exhibited some capability in mitigating object removal side effects.\n\nHowever, due to the conference committee's requirements, we are currently unable to include the visual comparison results in this submission. We plan to supplement these details in the final version of the paper.\n\nDuring our experiments, we observed that the performance of the trained DiffuEraser, while improved, **did not fully meet our expectations**. Since our focus is more on the removal effect of side effects, DiffuEraser after training didn't give satisfactory visual results. Also, we provide **a table below which quantitatively shows the enhancement of DiffuEraser** after training on the synthetic dataset. This led us to explore alternative base models, and we ultimately adopted Wan due to its stronger generative capabilities. The switch to Wan enhanced the model's overall performance, confirming that the choice of backbone architecture plays a critical role in the task. Also we shows the improvement made by changing the backbone model to Wan. In this table, we trained Wan with baseline settings, which uses \"mask-and-inpaint\" paradigm and without using any strategies and modules we design.\n\nTo further boost the model's robustness and inpainting quality, we introduced additional techniques, including mask region guidance, mask augmentation, and a difference mask predictor. These innovations collectively address the limitations observed in earlier experiments and contribute to the superior results presented in our work.\n\n**Table 1: Ablation studies on the backbone model testing on synthetic paired data**\n|  | **Metrics** | **DiffuEraser**  **Before training** | **DiffuEraser** **After training** | **Wan using baseline settings** | **ROSE(Ours)** |\n| :---: | :---: | :---: | :---: | :---: | :---: |\n| **Mean** | PSNR | 26.5024 | 27.0162 | 27.2863 | **31.1221** | \n|  | SSIM | 0.8981 | 0.9010 | 0.9025 | **0.9170** | \n|  | LPIPS | 0.1284 | 0.1106 | 0.1013 | **0.0772** |\n\n#"}, {"review_id": "ezuJozxEx2", "reviewer": "Reviewer_9ojy", "summary": "This work proposes to create a video removal dataset that focuses on removing the objects and the side effects of the objects. Firstly, the authors investigate five types of side effects. Then, a 3D rendering engine is used to generate the samples and ensure the quality of side effects in the videos. In the training stage, an auxiliary branch is used to achieve additional supervision, which could help the model learn to recognize the region of interest. Also, the ROSE-Bench is used to evaluate the performance of video object erasing models.", "questions": "1. Does ROSE-Bench include similar imperfect masks as well to better support the function of mask augmentation?\n2. Does the low PSNR in Tab. 2 mean that ROSE cannot generalize well to the real data due to the synthetic training dataset?", "limitations": "The suggestions are provided above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "S:\n1. The construction of a high-quality video removal dataset is necessary for the community. And the authors conclude six side-effect categories, which could guide following works.\n2. The 3D data successfully solves the problems of side effects in the previous datasets through the proposed construction pipeline.\nW:\n1. Fig. 1 is not clear when I zoom in, please use PDF images.\n2. The synthetic dataset only includes videos with camera motion and without object motion, which limits the capability of processing videos with moving objects. This point is only mentioned in Sec. 6, no visual results are shown in the paper.\n3. The side effects are divided into six types, but only three types are shown in Fig. 7.\n4. The lack of experiments on investigating the effectiveness of the mask-based filtering.\n5. Sec. 4.3 claims that the user-provided masks are often imperfect and thus proposes mask augmentation to mitigate this issue. But there is no such case in the experiment to show how ROSE could deal with the imperfect masks.\n6. The visual comparisons are superior in Fig. 7, but the PSNR value in Tab. 2 is lower than ProPainter.", "was_revised": true, "final_justification": "The authors provide clear explanations and directly clarify my concerns. All of the concerns are carefully solved by listing the experimental results and the analysis. What I care about the most is the performance on real-world video datasets and the capability of handling imperfect masks. The authors state the rationale for why the model trained on videos with only camera motion can generalize well to real videos, and the comparison between the performance of dealing with GT and SAM2 masks. It’s a pity that I cannot see the visualization results now. But I would like to say that the intrinsic limitation of the synthetic dataset and the data construction pipeline (only camera motion) could affect the performance. Overall, I would raise my score from Borderline reject to Borderline accept.", "rebuttal": "Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.\n\n**W1. Fig. 1 is not clear when I zoom in, please use PDF images.**\n\nWe apologize for the quality of figure and will replace it with clear PDF file in the revised version.\n\n**W2. The synthetic dataset only includes videos with camera motion and without object motion, which limits the capability of processing videos with moving objects. This point is only mentioned in Sec. 6, no visual results are shown in the paper.**\n\nFirstly, though there lack videos with moving object in the synthetic dataset, ROSE still show great performance on erasing moving object, as shown in Fig.1 in the supplementary material. The performance could be attributed as follows: The main purpose of ROSE is to guide the video model to distinguish the side effect area of selected object and conduct joint erasing. Erasing the object itself is not the main challenge of ROSE, which requires to fill the object area with proper video dynamics. Therefore, ROSE shows generalized performance on either moving or static objects since the object masks are provided, focusing on erasing the side effect areas. Furthermore, videos with camera motion help to construct relative object motion, which also enhance the model's performance on moving objects.\n\nSecondly, we show the limitations of ROSE in Fig.1 (line1-8) of supplementary material, showcasing objects with large motion or non-rigid motions. Though sometimes showing filckering effects, the overall performance of ROSE is still reasonable in many scenarios with substantial motions.\n\nThirdly, for future improvement, since constructing synthetic video with object motion is challenging, it could be explored to combining pseudo real-world data or generated data into the dataset for further improve the generalization of ROSE.\n\n**W3. The side effects are divided into six types, but only three types are shown in Fig. 7.**\n\nThanks for the suggestion. In Fig.7 of the main paper, we show four types of side effects (shadow, common artifacts, reflection, and light source). In Fig.2 of the supplementary material, we present further visual comparisons covering reflection, shadow, light source, and mirror effects. We acknowledge that the translucent category was not included. We have prepared the visualization results on this category. Since no images can be attached in the rebuttal, we will add them in the revised version.\n\n**W4. The lack of experiments on investigating the effectiveness of the mask-based filtering.**\n\nThank you for your observation regarding the mask-based filtering method presented in Fig.2. Below, we provide a detailed explanation of its purpose and implementation:\n\n**Objective of Mask-Based Filtering**\n\nIn our synthetic dataset construction pipeline using Unreal Engine, efficiency is critical due to the large-scale generation of video pairs. A key challenge is ensuring the quality of these pairs by identifying and filtering out problematic cases, such as those with severe occlusion.\n\n**Implementation Details**\n\n（1）We first render videos alongside their corresponding object masks.\n\n（2）By calculating the ratio of masked pixels to total pixels in each frame, we quantitatively assess occlusion levels.\n\n（3）Camera trajectories resulting in occlusion ratios beyond a predefined threshold are automatically discarded.\n\n**Deterministic Nature of the Process**\n\nSince the filtering mechanism is entirely rule-based and deterministic, the outcome is fully predictable given the input masks and thresholds to ensure purpose that the targeting object exists in the video. \n\nThis approach significantly accelerates dataset construction by eliminating manual inspection, while ensuring consistent quality across all generated video pairs.  We appreciate your attention to this methodological detail and hope this clarification addresses your question.\n\n**W5. Sec. 4.3 claims that the user-provided masks are often imperfect and thus proposes mask augmentation to mitigate this issue. But there is no such case in the experiment to show how ROSE could deal with the imperfect masks.**\n\nThank you for your valuable feedback. We appreciate your observation regarding the handling of imperfect masks in our experiments. Due to the conference committee's requirements, we were unable to include specific examples demonstrating ROSE's performance with imperfect masks in the current submission. However, we plan to supplement these cases in future work to provide a more comprehensive evaluation.\n\n**Real-World Application**\n\nIn practice, our framework leverages SAM2, a state-of-the-art segmentation model, to generate high-quality masks based on user input. SAM2's robustness ensures that most masks are accurate, but even for imperfect cases, ROSE is designed to:\n\n（1）Automatically infer the complete object mask from partial or noisy inputs.\n\n（2）Produce plausible and stable results despite mask inaccuracies.\n\n**Experimental Validation**\n\nTo quantitatively assess ROSE's resilience to imperfect masks, we conducted an additional experiment comparing results using ground-truth masks and results using intentionally imperfect masks generated by SAM2. The quantitative results (shown in the table below) confirm that ROSE maintains strong performance even with imperfect mask inputs, underscoring its practical applicability.\n\nWe hope this clarification addresses your concern, and we welcome further discussion on this point.\n\n**Table 1: Experiments on the input masks during inference**\n| **Category** | **Metrics** | **Using ground-truth masks** | **Using imperfect masks generated by SAM2** |\n| :---: | :---: | :---: | :---: |\n| Common | PSNR | 36.60 | 36.23 |\n|  | SSIM | 0.952 | 0.951 |\n|  | LPIPS | 0.041 | 0.040 |\n| Shadow | PSNR | 33.79 | 33.67 |\n|  | SSIM | 0.923 | 0.919 |\n|  | LPIPS | 0.063 | 0.071 |\n| Light | PSNR | 30.07 | 29.95 |\n|  | SSIM | 0.921 | 0.921 |\n|  | LPIPS | 0.086 | 0.087 |\n| Reflection | PSNR | 27.73 | 27.69 |\n|  | SSIM | 0.872 | 0.867 |\n|  | LPIPS | 0.113 | 0.118 |\n| Mirror | PSNR | 28.35 | 27.35 |\n|  | SSIM | 0.933 | 0.926 |\n|  | LPIPS | 0.088 | 0.097 |\n| Translucent | PSNR | 31.43 | 31.42 |\n|  | SSIM | 0.947 | 0.947 |\n|  | LPIPS | 0.060 | 0.061 |\n| **Mean** | PSNR | 31.12 | 31.05 |\n|  | SSIM | 0.923 | 0.920 |\n|  | LPIPS | 0.077 | 0.079 |\n\n**W6 & Q2. The visual comparisons are superior in Fig. 7, but the PSNR value in Tab. 2 is lower than ProPainter. Does the low PSNR in Tab. 2 mean that ROSE cannot generalize well to the real data due to the synthetic training dataset?**\n\nThe evaluation data of Tab.2 is constructed using *\"copy-and-paste\"* approach to create real-world video pairs. The results **only reflect the performance of erasing object itself**. ROSE shows comparable results on PSNR and better results on SSIM and LPIPS, which demonstrates that ROSE can effectively remove the object in real videos.\n\nHowever, the results of Tab.2 **does not reflect the side effect erasing**, which is the core objective of ROSE. Through extensive **visual comparisons** and **quantitative experiments** in Fig.2 and Tab.3, we have demonstrated that our model excels in this key capability.\n\nAlso, while ProPainter achieves a marginally higher PSNR score, our model demonstrates **superior overall performance** by outperforming it in both SSIM and LPIPS metrics. This balanced excellence across all three standard evaluation criteria confirms that our method delivers more structurally accurate results, produces visually superior outputs and maintains competitive fidelity.\n\nTo sum up, with comprehensive comparison on the three subsets evaluation of ROSE-Bench, we demonstrate that ROSE achieves satisfying performance on both side effect and object removal on real-world and synthetic videos.\n\n**Q1. Does ROSE-Bench include similar imperfect masks as well to better support the function of mask augmentation?**\n\nThank you for your question regarding our mask augmentation approach.\n\nWe sincerely apologize that we are unable to provide visual demonstrations of the mask augmentation effects at this time due to the requirements of the conference committee.\n\nTo substantiate the effectiveness of our mask augmentation method, we have included comprehensive quantitative results in the table under W5's answer. The data clearly demonstrates our model's robust performance when processing imperfect masks. The empirical evidence confirms that our augmentation strategy successfully enhances the model's capability to handle various mask imperfections while maintaining stable output quality."}, {"review_id": "ufQqVl5D8D", "reviewer": "Reviewer_8DvF", "summary": "Removing objects in videos is challenging because not only the objects themselves but also their associated effects must be removed.\nThe authors address this issue by proposing the ROSE framework.\nSince real-world data containing both objects and their side effects is scarce, they construct a novel synthetic dataset using a rendering tool.\nThis dataset enables effective training of the ROSE framework.\nTo handle side effects, the method incorporates mask region guidance into a diffusion-based backbone.\nAdditionally, to account for potential human errors in object masking, they introduce mask augmentation techniques during training.\nWith these contributions, the proposed framework achieves effective and robust video object removal.", "questions": "1.The paper requires a comparative analysis of efficiency with baseline methods such as ProPainter and DiffuEraser. In particular, metrics such as FLOPS and per-frame runtime during inference should be reported. Additionally, as mentioned in the limitations, it would be valuable to show the difference in per-frame inference time between long and short clips.\n\n2. The motivation in lines 156–159 clearly justifies the necessity of MRG, and its effectiveness is well demonstrated through the \"+MRG\" results in Tab. 1. Given this, it would be interesting to see whether similarly dramatic improvements can be observed when mask region guidance is applied to other methods as well. Do baseline models like ProPainter or DiffuEraser exhibit the same performance gains when MRG is incorporated?\n\n3. Mask augmentation, despite being presented as a key component of the proposed training strategy, shows only marginal performance improvements in Tab. 1, even leads to decreased performance. What are the results when only MRG and DMP are applied without MA? As discussed in Section 4.3, the authors should provide concrete examples and analysis of specific scenarios where this augmentation brings actual benefits, along with visual or quantitative evidence to support its effectiveness.\n\n4. I’m curious whether masking only part of an object results in side effects being removed only for that region, or if the effect of mask augmentation is strong enough to eliminate all related side effects. For example, in a scene where a person and a horse are jumping over a fence, what would the result look like if only the person is masked out?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Novel dataset, which contains various inpainting scenarios with object mask and multi-view scenes.\n2. Effectively removing not only the target object but also its side effects reflects the natural human expectation of comprehensive object removal in video content.\n\nWeaknesses\n1. While the proposed method shows promising performance, an analysis of computational efficiency (e.g., number of parameters, FLOPs) is missing. A comparison of resource usage with baseline methods is necessary to demonstrate that the framework achieves its results without excessive computational overhead.\n2. Ablation analysis is insufficient. It is important to provide a detailed breakdown of how each module contributes to the overall performance improvement, based on the ablation results.", "was_revised": true, "final_justification": "By reading the rebuttal, I found that the paper has become more complete in terms of necessary content. In particular, the detailed explanations regarding the ablation studies and the inclusion of computational resource analysis provide more objective metrics for evaluating the paper’s performance. Several other concerns have also been resolved. If the points raised in the rebuttal are properly incorporated into the final version, I believe the paper will offer thorough and well-supported explanations. Accordingly, I am raising my score to Accept.", "rebuttal": "Thank you for your constructive suggestions. We will follow your recommendations to polish our paper.\n\n**W1 & Q1. While the proposed method shows promising performance, an analysis of computational efficiency (e.g., number of parameters, FLOPs) is missing. A comparison of resource usage with baseline methods is necessary to demonstrate that the framework achieves its results without excessive computational overhead**\n\nFirstly, in practical deployment scenarios, we evaluated our model on a single NVIDIA H800 GPU. For a 6-second video (15 fps, 720×480 resolution) using float16 dtype, the inference process requires approximately 20-22GB of VRAM, with a per-frame inference time of 1.39 seconds.\n\nAlso, detailed runtime characteristics and comparative benchmarks are presented in Table below. While diffusion-based mechanisms are widely adopted for their superior generation capabilities, our model demonstrates competitive performance in real-world applications. As shown in the table, our model's per-frame inference time and memory usage are comparable to current state-of-the-art inpainting models, making it practical for production environments. And in real-world applications, we can adopt temporal tiling window to address longer videos and the processing time is proportional to the length of frames.\n\nSettings: input video is 65 frames with resolution of 720*480; GPU is NVIDIA H800; data type is torch.float16\n\n| **Inference Efficiency** |  |  |  |\n| :---: | :---: | :---: | :---: |\n|  | **All Parameters(M)** | **Average Runtime(s/frame)** | **Average GPU Memory(G)** |\n| **DiffuEraser** | 1952.3 | 1.02 | 26.12 |\n| **ProPainter** | 39.4 | 0.10 | 11.75 |\n| **FuseFormer** | 64.1 | 0.13 | 16.30 |\n| **FloED** | 1346.8 | 1.63 | 31.41 |\n| **FGT** | 42.3 | 1.43 | 14.38 |\n| **ROSE(Ours)** | 1564.4 | 1.39 | 21.05 |\n\n**W2 & Q3. Ablation analysis is insufficient. It is important to provide a detailed breakdown of how each module contributes to the overall performance improvement, based on the ablation results. As discussed in Section 4.3, the authors should provide concrete examples and analysis of specific scenarios where this augmentation brings actual benefits, along with visual or quantitative evidence to support MA’s effectiveness.**\n\nThank you for your valuable feedback. We sincerely apologize for the lack of clarity in our explanation of **Tab.1**. To clarify, the results presented in the table are **cumulative**, meaning that each module, i.e. **MRG,** **MA, and DMP**, was added sequentially. Therefore, the specific contribution of the **DMP module** can be observed by comparing the last two columns of the table.\n\nFirstly, to further demonstrate the **standalone accuracy and effectiveness** of the DMP module, we conducted additional experiments, as shown in the first table below. The baseline setting employs the *\"mask-and-inpaint\"* paradigm without incorporating any of the proposed modules or strategies.\n\nFrom the experimental results, we can clearly observe the quantitative improvements achieved by introducing the difference mask predictor (DMP) module to supervise the training process. This further validates the effectiveness of our proposed approach.\n\nSecondly, to validate the impact of our synthetic dataset and assess the role of the backbone model, we conducted an ablation study on DiffuEraser. Specifically, we compared its performance before and after training with our dataset. The results (shown in the second table below) indicate that while our dataset contributes to measurable improvements in DiffuEraser’s performance, the overall enhancement remains limited. This observation motivated us to explore a more robust backbone model to achieve higher performance.\n\nThirdly, to support MA’s effectiveness, we conduct an experiment in inference, comparing the effect of output video while dealing with ground-truth masks and imperfect masks generated by SAM2. The quantitative results can be seen in the subsequent third table. And from the table we can find that by adopting MA in training process, our model performs robustness while the input masks are not perfect. For visual results proving the robustness, we’re sorry that we can’t provide them currently for the requirements of conference committee and we promise we’ll supplement them later in our open-source project page.\n\n**Table 1: Ablation studies on each module**\n| **Category** | **Metrics** | **Base** | **w/ MRG** | **w/ MA** | **w/ DMP** |\n| :---: | :---: | :---: | :---: | :---: | :---: |\n| Common | PSNR | 32.58 | 35.24 | 33.54 | 35.68 |\n|  | SSIM | 0.937 | 0.950 | 0.949 | 0.943 |\n|  | LPIPS | 0.053 | 0.040 | 0.046 | 0.045 |\n| Shadow | PSNR | 30.65 | 33.29 | 31.63 | 32.85 |\n|  | SSIM | 0.914 | 0.920 | 0.922 | 0.920 |\n|  | LPIPS | 0.081 | 0.061 | 0.072 | 0.064 |\n| Light | PSNR | 24.99 | 30.37 | 28.89 | 30.13 |\n|  | SSIM | 0.894 | 0.923 | 0.911 | 0.922 |\n|  | LPIPS | 0.112 | 0.074 | 0.082 | 0.077 |\n| Reflection | PSNR | 25.39 | 27.71 | 26.97 | 27.41 |\n|  | SSIM | 0.836 | 0.843 | 0.845 | 0.841 |\n|  | LPIPS | 0.131 | 0.109 | 0.111 | 0.110 |\n| Mirror | PSNR | 22.63 | 28.45 | 26.50 | 27.65 |\n|  | SSIM | 0.905 | 0.941 | 0.932 | 0.932 |\n|  | LPIPS | 0.142 | 0.076 | 0.086 | 0.092 |\n| Translucent | PSNR | 27.43 | 30.98 | 31.14 | 31.24 |\n|  | SSIM | 0.925 | 0.949 | 0.948 | 0.946 |\n|  | LPIPS | 0.087 | 0.052 | 0.056 | 0.059 |\n| **Mean** | PSNR | 27.28 | 30.84 | 29.77 | 30.82 |\n|  | SSIM | 0.902 | 0.918 | 0.916 | 0.917 |\n|  | LPIPS | 0.101 | 0.071 | 0.076 | 0.074 |\n\n**Table 2: Ablation studies on the backbone model**\n|  | **Metrics** | **DiffuEraser**  **Before training** | **DiffuEraser** **After training** | **Wan using baseline settings** | **ROSE(Ours)** | \n| :---: | :---: | :---: | :---: | :---: | :---: | \n| **Mean** | PSNR | 26.5024 | 27.0162 | 27.2863 | **31.1221** | \n|  | SSIM | 0.8981 | 0.9010 | 0.9025 | **0.9170** |\n|  | LPIPS | 0.1284 | 0.1106 | 0.1013 | **0.0772** |\n\n**Table 3: Experiments on the input masks during inference**\n| **Category** | **Metrics** | **Using ground-truth masks** | **Using imperfect masks generated by SAM2** |\n| :---: | :---: | :---: | :---: |\n| Common | PSNR | 36.60 | 36.23 |\n|  | SSIM | 0.952 | 0.951 |\n|  | LPIPS | 0.041 | 0.040 |\n| Shadow | PSNR | 33.79 | 33.67 |\n|  | SSIM | 0.923 | 0.919 |\n|  | LPIPS | 0.063 | 0.071 |\n| Light | PSNR | 30.07 | 29.95 |\n|  | SSIM | 0.921 | 0.921 |\n|  | LPIPS | 0.086 | 0.087 |\n| Reflection | PSNR | 27.73 | 27.69 |\n|  | SSIM | 0.872 | 0.867 |\n|  | LPIPS | 0.113 | 0.118 |\n| Mirror | PSNR | 28.35 | 27.35 |\n|  | SSIM | 0.933 | 0.926 |\n|  | LPIPS | 0.088 | 0.097 |\n| Translucent | PSNR | 31.43 | 31.42 |\n|  | SSIM | 0.947 | 0.947 |\n|  | LPIPS | 0.060 | 0.061 |\n| **Mean** | PSNR | 31.12 | 31.05 |\n|  | SSIM | 0.923 | 0.920 |\n|  | LPIPS | 0.077 | 0.079 |\n\n**Q2. The motivation in lines 156–159 clearly justifies the necessity of MRG, and its effectiveness is well demonstrated through the \"+MRG\" results in Tab. 1. Given this, it would be interesting to see whether similarly dramatic improvements can be observed when mask region guidance is applied to other methods as well. Do baseline models like ProPainter or DiffuEraser exhibit the same performance gains when MRG is incorporated?**\n\nThank you for your insightful observation. Indeed, we initially explored the possibility of using the models you mentioned. However, we encountered several challenges that led us to reconsider their suitability for our framework.\n\nFirst, **ProPainter relies on an image propagation approach**, which requires a masked video as input. This design is fundamentally incompatible with our mask region guidance paradigm, making further experimentation with ProPainter impractical for our specific use case.\n\nAs for DiffuEraser, its dual-branch architecture depends on a pretrained image inpainting model (BrushNet). Specifically, its inpaintng ability relies on the pretrained BrushNet's image inpainting ability and to adapt MRG for training, it needs to change BrushNet's architecture completely and involve more additional design, which makes it difficult to conduct a reasonable comparison experiment. Given these constraints, we decided against pursuing further experiments with these two models.\n\n**Q4. I’m curious whether masking only part of an object results in side effects being removed only for that region, or if the effect of mask augmentation is strong enough to eliminate all related side effects. For example, in a scene where a person and a horse are jumping over a fence, what would the result look like if only the person is masked out?**\n\nThank you for raising this interesting question. The effect of partial masking depends on the specific context of the image and the relationships between objects.\n\nFor instance, in the scenario you described (Fig.7, line 2) where a person and horse are jointly jumping over a fence, masking only the person would successfully remove just the human figure while preserving the horse. This demonstrates that when objects are distinctly separate entities, partial masking can produce localized effects.\n\nHowever, when objects are intrinsically connected, the outcome differs. As shown in Fig.1 of the supplementary material featuring a cow with a neck knot, masking just the cow's body while leaving the knot unmasked produces the same result as masking both elements. This occurs because the knot is perceived as an integral part of the cow's anatomy by the model."}]

Author rebuttal:

Meta-review: All reviewers are in agreement to accept the paper. The rebuttal added substantial content that **must be included in the final version**. One remaining concern, shared also by the AC and other fellow reviewers, is the real-world performance, given the synthetic training setup. The qualitative results seem to demonstrate that this may be fine, but it is somewhat hard to validate this, given that qualitative results can always be cherry-picked. Still, as the authors provide code release, this probably can be validated as the work becomes public. The AC thus follows this unanimous recommendation to accept the paper.

One reviewer went missing after the initial review, and thus have been flagged for this behavior.
Decision: Accept (poster)