Deforming Videos to Masks: Flow Matching for Referring Video Segmentation
Authors: Zanyi Wang
Dengyang Jiang
Liuzhuozheng Li
Sizhe Dang
Chengzu Li
Harry Yang
Guang Dai
Mengmeng Wang
Jingdong Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Referring Video Object Segmentation
Flow Matching
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Flow Matching for Referring Video Object Segmentation

Abstract: Referring Video Object Segmentation (RVOS) requires segmenting specific objects in a video guided by a natural language description. The core challenge of RVOS is to anchor abstract linguistic concepts onto a specific set of pixels and continuously segment them through the complex dynamics of a video. Faced with this difficulty, prior work has often decomposed the task into a pragmatic `locate-then-segment' pipeline. However, this cascaded design creates an information bottleneck by simplifying semantics into coarse geometric prompts (e.g, point), and struggles to maintain temporal consistency as the segmenting process is often decoupled from the initial language grounding. To overcome these fundamental limitations, we propose FlowRVS, a novel framework that reconceptualizes RVOS as a conditional continuous flow problem. This allows us to harness the inherent strengths of pretrained T2V models, fine-grained pixel control, text-video semantic alignment, and temporal coherence. Instead of conventional generating from noise to mask or directly predicting mask, we reformulate the task by learning a direct, language-guided deformation from a video's holistic representation to its target mask. Our one-stage, generative approach achieves new state-of-the-art results across all major RVOS benchmarks. Specifically, achieving a J&F of 51.1 in MeViS (+1.6 over prior SOTA) and 73.3 in the zero shot Ref-DAVIS17 (+2.7), demonstrating the significant potential of modeling video understanding tasks as continuous deformation processes.

Published as a conference paper at ICLR 2026
DEFORMING VIDEOS TO MASKS: FLOW MATCHING FOR REFERRING VIDEO SEGMENTATION
Zanyi Wang Dengyang Jiang Mengmeng Wang 6.1f Jingdong Wang?
Liuzhuozheng Li
Sizhe Dang
Chengzu Li
Harry Yang²
Guang Dai
SGIT Lab, State Grid Corporation of China Uniierriiyy of California, San Diego 'hee Hong Kong University of Science and Technology "The University of Tokyo
University of Cambridge
*Zeejang University of Technology
'Baidu
Locate
Segment
Grounding
Segmentation Model
Flow Matching Model
Model
(A) Segment' Paradigm
(8) Seaied Forkwer
Verformane Incraane
runniin
The pania the other pania's back.
(G) Qualitative Performance of FlowRVS
(D) Quantitative Performance Comparison 6grours)
Figure 1: FlowRVS replaces the cascaded srcate-then-ee segmentt paradigm (A) with unified, end- to-end flow model (B). This new paradigm avoids information bottlenecks, enabling superior han- dling of complex language and dynamic video (C) and achieving state-of-the-art performance (D).
ABSTRACT
Referring Video Object Segmentation (RVOS) requires segmenting specific ob- jects in a video guided by natural language description. The core challenge RVOS anchor abstract linguistic concepts continuously segment them through the complex dynamics of a video. Faced with this difficulty, prior work has often decomposed the task into a pragmatic pipeline. However, this cascaded design creates an infor- mation bottleneck by simplifying semanties into coarse geometric prompts (e.g., point), and struggles to maintain temporal consistency the segmenting process is often decoupled from the initial language grounding. To overcome these fun- propose FlowRVS, novel framework that reconceptu-
specific set of pixels and
damental limitations, alizes RVOS as conditional continuous flow problem. This allows us to harness the inherent strengths of pretrained T2V models, fine-gres fine-grained pixel control, text- video semantic alignment, and temporal coherence. Instead conventional gen- erating from noise mask directly predicting mask, we reformulate the task by learning a direct, language-guided deformation from a video's holistic repre- sentation to its target mask. Our one-stage, generative approach achieves new state-of-thee the-hhhrrttt art results across all major RVOS benchmarks. Specifically, achiev- ing a J&F of 51.1 in MeViS (+1.6 over prior SOTA) and 73.3 in the zero shot Ref-DAVIS17 (+2.7), demonstrating the significant potential of modeling video understanding tasks as continuous deformation processes. Code available at: htttps://gtthiii hubbb...gttubbt..///.. com/ xmz cimmiill 111/F www.iii/Fooooot. 1//FlowRVS
Equual contribution. This work was completed during the internship at SGIT AI Lab.
Corresponding author. *Project lead.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Referring Video Object Segmentation (RVOS) (Khoreva al. 2018 Gavrilyuk al. 2018 Hu
et al. 2016) requires the machine to segment objects described by natural language queries, which critical to intelligent systems to precept and interact with the real world (Jiang et al. 2025 Li et al. 2023). The core challenge of RVOS lies in resolving a fundamental spatio-temporal correspondence dilemma: anchoring abstract linguistic concepts onto dynamic and fine-grained pixel space. Cur- rent paradigms often rely on instance-centric approaches, which first identify and then track object instances. While effective, even modern query-based (e.g., ReferFormer (Wu et al. 2022 ) or VLM- based (e.g., LISA (Lai et al. 2024)) methods can introduce an information bottleneck by collapsing rich semantics into intermediate object-centric representations. This can limit holistic scene under- standing and temporal consistency, especially as the segmentation of each frame, while conditioned, doesn't 't stem from a single, unified spatio-temporal deformation process (Liang et al. 2025b: Ren
et al. 2024
Bai et al. 2024 Lin et al. 2025).
To address these limitations, we argue that pretrained Text-to-Vides (T2V) models offer a funda- mental solution since their native capabilities for fine-grained, text-to-pixel synthesis and spatio- temporal reasoning directly counter the bottlenecks of the "locatetthen-segmen" paradigm. While some attempts T2V models as powerful frozen feature extractors for separate decoder (e.g., VD-IT (Zhu et al. 2024). HCD (Zhang et al., 2025)), this two-stage design decouples the model's generative dynamics from the final task. Our work fundamentally differs: we propose repurpose the entire generative process itself, learning a direct,
eeguage-guided deformation flow from video to mask. DepthFM (Gui et al. (025) have adapted the entire generative process itself for visual- to-visual tasks like depth estimation based on a image generation model. While these pioneering efforts validate the generative approach, they also expose a shared, critical blind spot: they fail to fully utilize the dynamic, text-driven reasoning that T2V models are capable of and RVOS demands. The -xttureextttation approach remains decoupled, forcing a separate decoder to reconstruct tem- poral relationships from temporally isolated features, squandering the T2V model's inherent video coherence, Meanwhile, the image-to-depth flows proposed by previous styles completely neglect text condition, rendering them fundamentally incapable of addressing the core RVOS challenge: producing different masks for the same vidco based on varying textual queries. Thus, we argue that deeper, more principled alignment with the T2V paradigm is required: one that treats the entire process as a single, unified, eeguagg--uided flow from video pixels to required masks.
This unified flow with existing powerful T2V pretrain model (e.g, Wan) brings several benefits: (1) their pixel-level synthesis training provides fine-graine -reigra control, which enables them to distinguish and handle more delicate objects when locating specific targets; (2) their text-condition generation ensures powerful multi-modal alignment, which allows them to ground rich linguistic semantics directly in the pixel space without as much information loss as first mapping in coarse geomet- ric intermediaries; (3) their video-native architecture provides inherent spatio-temporal reasoning, naturally unifying language guidance with temporal consistency.
However, simply leveraging the T2V framework is not enough. As shown in Figure standard T2V generation divergent process: it maps simple noise prior to a set of possible videos, exploring broad trajectory space. RVOS, conversely, is a convergent task: it must map a complex, high-entropy video to a single, low-entropy mask. This transforms the problem into a deterministic, guided information contraction, where the text query acts as the crucial selector that isolates the precise target from the rich visual input (c.g., distinguishing "the smaller monkey" from "the bigger monkey"). This core insight that RVOS is convergent flow directly informs contributions. To successfully manage this asymmetric transformation, we introduce a suite of principled adaptations: (1) boundary-biased sampling strategy to force the model to master the erucial, high-certainty start of the trajectory where the video's influence is strongest; (2) a direct video injection mechanism preserve the rich source context throughout the contraction process; and (3) task-specific VAE adaptation and start point augmentation to create a stable latent space for this unique mapping.
Summarizing, our contributions are as follows:
We reformulate RVOS learning continuous, text-conditioned flow that deforms video's spatio-temport representation into its target mask, directly resolving the correspondence be- tween language and dynamic visual data.
2

---

Published as a conference paper at ICLR 2026
We propose suite of principled techniques that successfully enable the transfer of powerful text-to-video generative models to this challenging video understanding task.
Our proposed framework, FlowRVS, establishes a new state of the art on key benchmarks. No- tably, it achieves a significant improvement of 1.6 % J&F on the challenging MeViS dataset and 2.7 % I JFF on the zero-shot DAVIS 2017 benchmark.
2
RELATED WORK
Referring Video Object Segmentation aims to segment target object within video based natural language expression (Xu et al. [0188] Ding et al. (023). This task demands both visual- linguistic understanding and robust temporal segmenting. Early approaches often adapted frame- level referring image segmentation models and appended temporal linking mechanisms as a post- processing step (Khoreva et al., 2018). More recent and competitive methods have evolved into more integrated yet predominantly multi-stage pipelines. A dominant paradigm involves a "locate- then-segment" strategy, where a powerful multi-modal model first grounds the textual reference to a spatial region, which then guides separate segmentation process for each frame.
LLocate-then-Sggmeen Paradigm manifests in several forms. A significant breakthrough came with the introduction of query- based architectures, inspired by the success of DETR- -tyle (Zhu et al., 2020) transformers in vision tasks (Wu et al. [0022] Yan et al. 2024b), this new paradigm reframed RVOS by treating language as query to the visual features. Furthermore, multimodal model based approaches like LISA (Lai et al., 2024). VISA Yan et al. 2024a) and ReferDINO (Liang et al, 2025b a) leverage pretrained model's ' reasoning grounding ability, such LLa et al. 2023) or DETR-basee GroundingDINO (Ren et al. 2024). to perform initial object localization, and then introduce custom-designed mask decoder to generate the final segmentation. A similar phi- losophy is seen the "VLM+SAMM" family of methods (Luo et all 2023 Wu et al.) (023). which use Vision-Language Model for bounding box prediction, followed by generic segmentation model like SAM (Kirillov et al.] 20033] Cuttano et al., 2025) to produce pixel-level masks (He & Ding 2024 Lin at. 20255 Pan et al. (025). Another line of work explores repurposing generative models: VD-IT (Zhu et al. 2024) first extracts features from a pretrained text-to-video diffusion model and then feeds these features into a separate DETR-like architecture for mask prediction. While these methods have pushed the performance boundaries, their reliance on intermediate rep- resentations whetthe object queries or extracted features can introduce information bottlenecks that prevent a truly holistic, end-to-end optimization of the video-to-mask correspondence problem.
Generative Modeling is largely catalyzed by the advent of latent di diffusion models (Rombach al., 2022). Building this success, the frontier rapidly expanded into the temporal domain, leading to a surge of powerful text-to-video (T2V) models (Liu et al. 2024 Wan et al. 2025 Gao et al, 2025). Recent works have begun to leverage these models for RVOS. A notable approach (c.g.,
VD-IT (Zhu et al. 2024), HCD (Zhang et al. 2025)) utilizes T2V models as powerful frozen feature extractors for separate segmentation decoder. Concurrent work ReferEEerything (REM) (Bagchi) ct al. 2025) also explores video-to-mask generation but relies on direct one-step prediction. Our work fundamentally differs: instead of extracting features learning a static mapping, we re- purpose the entire generative process, fine-tuning the core model to learn a direct video-to-mask deformation flow. This avoids the bottleneck inherent in two-stage pipeline. Furthermore, unlike conditional generation frameworks like ControlNet(Zhang et al. 2023) that add external guidance to divergent, noise-to-image to-image process, our method learns a convergent, discriminative transforma- tion from the video source itself. Beyond diffusion, Flow Matching (Lipman et al.) 2022) Liu et al.) 2022) offers a significant theoretical advancement by learning a velocity field to transport samples along deterministic ODE path. This has been leveraged for visual tasks like depth estimation (e.g.. DepthFM (Gui et al. 2025)). but these methods are typically text-agnostic. Our work makes a critical distinction: introduce the natural language query as the core conditional force that mod- ulates the entire ODE path. This elevates the framework from a simple translation to a dynamic, multi-modal reasoning engine, reframing RVOS learned, conditional deformation.
Liu
3

---

Published as a conference paper at ICLR 2026
Training
Inference
The dog rotattin while keeping pows green flynng
The dig retuting white keeping pows green fling saucer
1-t
NSE
lods
141
Flow Matching Model
Var
Flow Matching Model
14
tinestop
ODE Sampling
boundary soard fire sampling
Interpolation
20
dxs fd de
Paradigms
Performance from bad to good
multssepp
demaiinng muies mask
predicting mock/ollcitty
Dowing frum video mack
Figure 2: FlowRVS reformulates RVOS as a text-conditioned continuous flow, learning velocity field via Flow Matching stabilized by boundary-biased time sampling in latent space. During infer- ence, ODE solver uses this field to deterministically deform the video latent to the target mask, this video to mask paradigm superior to noise-based or one-step prediction approaches.
METHOD
3.1 PROBLEM REFORMULATION RVOS AS A CONTINUOUS FLOW
Traditionally, RVOS approached as discriminative, one-step prediction task. model fa trained to learn a direct mapping M = fo(V. c) from video-text pair to mask sequence. How- ever, this direct mapping is fundamentally challenged by the need to collapse dynamic, high- dimensional video into a precise pixel-mask under the complex constraints of linguistic instruction, all within a single transformation.
To overcome this limitation, depart from direct prediction and reconceptualize RVOS text- conditioned continuous flow problem. We propose to model segmentation as a gradual, determin- istic deformation process that transforms the video's representation into the target mask's. This is governed by an Ordinary Differential Equation (ODE), where our goal is to learn the velocity field (2z C, t) that dictates the evolution of latent state Zz:
dz. dt v(z,, C, with boundary conditions
Peideo and Pmask
(1)
The trajectory starts from the video latent zo and is guided by the text query e to terminate at the specific mask latent 21. This transforms the learning objective from mastering a single, complex global function to learning simpler, local velocity field.
However, adapting this generative-ne nativee paradigm to a discriminative task like RVOS is not straight- forward, but a fundamental inversion of the generative process, as illustrated in Figure 3 Standard T2V generation divergent, one-to-many process: it starts from a simple, fixed noise distribution and has broader exploration space in the initial steps to generate diverse set of plausible videos. contrast, our approach
convergent, video-text-to-one task. It begins with a complex, high- entropy video latent An and must follow more tightly mapped direction to a single, correct mask. Here, the text query is no longer a creative prompt but a critical, disambiguating force. The initial velocity computed from Zo must be precise enough to distinguish "the smaller monkey" from "the bigger monkey." An error in this first step is irrecoverable, dooming the entire trajectory to fail. This places paramount importance correctly learning the starting point of the flow.
3.2 TRANSFERRING TEXT-TO-VIDEO MODEL TO RVOS
naive, uniform treatment of the trajectory, inherited from generative modeling, fails to account for the unique asymmetric nature of the video-to-mask flow. This asymmetry-42 high-certainty,

---

Published as a conference paper at ICLR 2026
Noise to Video
Video to Mask
more tightly mapped direction in the inittal step determined by video and text query
The smaller monkay
Thee bigger munday
"A cut is
running
geat
blac black
broader exploration space and direction the initial stop
Figure 3: Repurposing generative process for a discriminative task. Unlike standard T2V genera- tion which maps noise to diverse videos (left), our method maps complex video to single mask (right). This transforms the process into deterministic, convergent task where the text query the crucial element that selects the precise target from the visual input (e.g., distinguishing the 'smaller' from the 'bigger' monkey).
structured start and a low-certainty, sparse end-demands a non-uniform approach to learning the velocity field. Therefore, we introduce suite of three synergistic strategies grounded in a single principle: fortifying the flow's origin. These are designed not as independent tweaks, but as cohesive framework successfully adapt the powerful T2V model for RVOS.
Boundary-Biased Sampling (BBS). We hypothesize that the most critical learning signal resides at the beginning of the trajectory, where the model computes the initial "push" away from the video manifold based the text query. To exploit this, introduce BBS, curriculum learning strategy that oversamples timestep / 0. By concentrating the gradient updates on this initial, high-influence decision, we force the model to first master the crucial text-guided velocity computation. As em- pirically demonstrated in Table 2 this focused learning strategy is the key to stabilizing the training process, transforming the failing baseline into highly effective model by ensuring well-posed initial value problem for the ODE.
Start-Point Augmentation (SPA). To prevent the model from overfitting to discrete points on the data manifold and to encourage the learning of smoother, more generalizable flow, we introduce Start-Point Augmentation (SPA). During training. we transform the initial video latent through stochastic encoding and normalization process. This technique effectively presents the model with richer, locally continuous distribution of starting points centered around the original video latent. This acts as a powerful regularizer, forcing the model to learn a velocity field that is robust not just on the manifold, but also in its immediate vicinity.
Direct Video Injection (DVI). In our video-to- mndeo-t--mask formulation, the initial video latent Zo is not merely starting point, but the foundational context for the entire transformation. To ensure this context remains accessible throughout the flow, introduce Direct Video Injection (DVI). We implement this by concatenating the original video latent Zo with the current state 2, along the channel dimension transforms the velocity prediction at every subsequent point from v(z,, t) to v([z,, zo], t), explicitly conditioning each local update on the global origin. This simple yet effective strategy provides persistent, high-fidelity reference to the source video, preventing trajectory drift and improving fine-grained accuracy with negligible computational overhead.
each ODE step without introducing heavy computational burden. This
3.3 ANALYSIS OF ALTERNATIVE PARADIGMS
To motivate our final choices, we first analyze the fundamental limitations of three plausible al- ternative paradigms for adapting T2V model to RVOS. As empirically validated in ablation studies, each of these alternatives fails due to a core mismatch with the nature of RVOS. To ensure a fair comparison, all alternative paradigms were built upon the same Wan2.1 pre-trained model and
5

---

Published as a conference paper at ICLR 2026
trained under the exact same supervised setting (same optimizer, learning rate, and duration) as final model. And we implemented the same fine-tuned VAE to reveal the superiority of our proposed flow-based paradigm.
Direct Mask Prediction (Worst Performance). A direct, single-step mapping from the video and text latents the mask latent represents the classic discriminative paradigm. We argue this approach is fundamentally ill-posed due to what we term "information collapse." " The mapping from a high- entropy. complex video manifold to a low-entropy, sparse mask manifold is a drastic information contraction. Forcing neural network to learn this in single, abrupt step leads to collapsion of the rich visual context into coarse approximation rather than performing a precise, guided refinement. The model is not learning a transformation, but rather a brittle pattern recognition function.
Noise-to-Mask Flow (Suboptimal), This paradigm mirrors standard text-to-video generation, starting from Gaussian noise z (0, 1) and conditioning the video context. This approach demotes the video from the primary source of information to a secondary condition. Weaken the guidance of the video in the process would possibly force the entire, high-dimensional video context to be injected via a simple concatenation at each step, creating a severe information bottleneck. The model is tasked with generating the mask's complex nempo-l structure from scratch based on this limited conditional signal, rather than progressively refining the rich, structured information already present in the video itself.
One-Step Velocity Prediction (Better, but Limited). This paradigm makes the model predict the full velocity vector = n Zo in a single inference step. significantly outperforms the previous two baselines, confirming our hypothesis that learning residual velocity is more stable and ef- fective objective than predicting an absolute state. However, its performance is still fundamentally capped. We assume that it is limited by the need to compute the entire, often large-magnitde de- formation in a single forward pass, lacking the capacity for the gradual, iterative refinements that a multi-step process allows.
This analysis solidifies our central thesis: a multi-step, video-to-mask flow the most effective paradigm for RVOS, but only when augmented with our proposed start-point focused adaptations to bridge the critical gap between its generative origins and the discriminative demands the task.
EXPERIMENTS
4.1 BENCHMARK AND METRICS
We evaluate framework three standard RVOS benchmarks. McViS (Ding et al. 2023 is challenging, motion-centric benchmark featuring 2,006 long videos and over 28,000 fine-gras grained annotations that emphasize complex dynamics. Ref-YouTuule YouTube-VOS (Wu et al. 2022 is the large- scale benchmark, comprising 3,978 videos that test for generalizability across a wide diversity of objects and scenes. Ref- Ref-DAVIS17 (Khoreva et al. 2018) a high-quality, densely annotated dataset of 90 videos, serving as key benchmark for segmentation precision and temporal consistency.
Following standard protocols, report region similarity (J, Jaccard Index), contour accuracy (F, F-measure), and their average (J&F) as our primary evaluation metrics.
4.2 IMPLEMENTATION DETAILS
Our framework built upon the publicly available Wan 2.1 text-to-video model, which features a 1.3B parameter Diffusion Transformer (DiT) (Wan et al. (025). Throughout all training stages, we keep the pretrained text encoder and the VAE encoder entirely frozen. Our training focuses exclu- sively on fine-tuning the DiT block to learn the conditional flow. Crucially, the VAE decoder is specifically adapted for the segmentation task by being fine-tuned separately on the MeViS training set, allowing it to specialize in reconstructing high-quality masks from the latent space. Our train- ing protocol varies by dataset to align with the same evaluation protocols of compared methods. For experiments on Ref- Tube VOS, we follow two-stage training strategy. The model is first pre-trained on a combination of static image datasets (RefCOCO/+/g) (Yu et 2016, Kazemzadehi et al. 2014) to learn foundational visual- linguisic grounding. Subsequently, this pre-trained model

---

Published as a conference paper at ICLR 2026
Table Comparison of our one-stage FlowRVS with other previous Docate-then-seemeee methods on ViS, Ref- YouTube- VuS and Ref-DAVIS datasets. We further include methods based on large VLMs for comparison. Bold and underline indicate the two top results.
MeViS
Ref- YouTube- VOS
Ref-DAVIS17
Method
J&F
J
F
J&F
J
F
J&F
J
F
Locate-then-segment
MTTR [CVPR'2)
30.0
28.8
31.2
55.3
54.0
56.6
ReferFormer [CVPR'22]
31.0
29.8
32.2
62.9
61.3
64.6
61.1
58.1
64.1
SOC INIPS' 23]
66.0
64.1
67.9
64.2
61.0
67.4
OnlineRefer [ICCV'23]
32.3
31.5
33.1
63.5
61.6
65.5
64.8
61.6
67.7
LISA [CVPR'24]
37.2
35.1
39.4
53.9
53.4
54.3
64.8
62.2
67.3
DsHmp [CVPR'24]
46.4
43.0
49.8
67.1
65.0
69.1
64.9
61.7
68.1
VideoLISA [NIPS'24]
42.3
39.4
45.2
61.7
60.2
63.3
67.7
63.8
71.5
VD-IT [ECCV'24]
64.8
63.1
66.6
63.0
59.9
66.1
VISA [ECCV'24]
43.5
40.7
46.3
61.5
59.8
63.2
69.4
66.3
72.5
SSA [CVPR'25]
48.9
44.3
53.4
64.3
62.2
66.4
67.3
64.0
70.7
SAMWISE [CVPR'2] 25]
49.5
46.6
52.4
69.2
67.8
70.6
70.6
67.4
74.5
ReferDINO [ICCV25]
49.3
44.7
53.9
69.3
67.0
71.5
68.9
65.1
72.9
One-stage generation based
FlowRVS (ours)
51.1
47.6
54.6
69.6
67.1
72.1
73.3
68.4
78.2
fine-tuned on the Ref-YouTTube-Ve training set. The final weights from this stage then used for zero-shot evaluation on the Ref-DAVIS17 benchmark without any further fine-tuning to prove FlowRVS's generalization ability. For the more challenging MeVIS dataset, which empha- sizes complex motion understanding, we train our model directly on its training set from scratch, without leveraging any static image pre-training. More hyperparameters settings be found in Appendix A
4.3
MAIN RESULTS
We compare our proposed FlowRVS against wide range of baselines in Table 1 Our results show that FlowRVS significantly and consistently outperforms previous approaches. We highlight some key features as follows:
Dominance on Complex Motion-Centric Benchmarks. The most significant advantage of our framework is demonstrated on MeViS, the most challenging benchmark designed to test nuanced understanding of motion-centric language. FlowRYS achieves a J J&F score 50.7, establishing a new SOTA and surpassing the previous best method, SAMWISE, by substantial 1.2 point margin. This result is particularly noteworthy as MeViS features long videos with complex object interac- tions and appearance changes- scenarios where the limitations of multi-stage, cascaded pipelines are most exposed. The superior performance of FlowRVS directly validates our core thesis: holis- tic, end-to-end flow that models the entire video-to-mask transformation is fundamentally better suited to capture and reason about complex spatio-temporal dynamics.
TLocate-then-Segment Paradigms. Our performance gains particularly meaningful when compared directly against methods that epitomize the 'locatetthen-segment paradigm, such VISA (VLM-based) and ReferDINO prouuddng-model-bassee FlowRVS outperforms VISA-13B by a remarkable 7.0 points and ReferDINO (best results an- nounced in the paper) by 1.4 points. This underscores the advantage of our one-stage approach. By avoiding the irreversible information loss inherent in collapsing semantics into an intermedi- ate geometric or feature prompt, our continuous flow process maintains a high-fidelity, text-guided transformation from start to finish, leading to more accurate and robust segmentation. The funda- mental advantages our one-stage flow paradigm are further illustrated in our qualitative compar- isons (Figure For the query "The white rabbit which is jumping," ReferDINO provides coarse, static grounding of the rabbit but misses the jumping action's details, whereas FlowRVS delivers a precise, dynamic segmentation. More critically, for the temporal query "The first tiger...", VD-IT's
Superiority
On MeVis,

---

Published as a conference paper at ICLR 2026
NeVIS
white robbir which jumpling
white monkeyy layling back the
ReferDiNo
VD-TT
saart
ReferDiNo
VD-IT
mockey moving from right left.
The first tiger area with through
Figure 4: Qualitative comparison on challenging temporal and linguistic reasoning. Prior paradigms struggle: VD-IT produces temporally unstable masks due its frame-wist decoder, while Refer- DINO fails to interpret long-range descriptions. Our method, FlowRVS, demonstrates superior tem- poral coherence and language grounding by leveraging an end-to-end generative process.
Table Ablations FlowRVS on the MeVIS valid, set. BBS: Boundary- Noundary-Biased Biased Sampling (prob- ability p). DVI: Direct Video Injection. SPA: Start Point Augmentation. WI: Weight Init from Wan.
ID Method Configuration
BBS (p) SPA
DVI
WI
J&F
F
Alternative Paradigms
MultiStep Noise-to-Mask Flow (a)
32.3
29.6
35.0
Onestep Mask Prediction
38.9
36.2
41.5
Onestep Velocity Prediction
50.8
47.1
54.4
Our MulriStep
Flow
(c) Base Flow
0.0
47.9
42.9
52.9
(d) BBS
0.25
55.2
50.7
59.6
(c) + BBS
0,50
57.9
53.8
62.1
(I) BBS
0.75
56.5
52.5
60.4
(g) + SPA
0.50
58.6
54.2
63.0
(h) DVI (ours default)
0.50
60.6
55.9
65.2
() -WI
0.50
21.1
20.3
21.9
decoupled decoder fails to resolve the ambiguity and tracks the wrong target. In contrast, FlowRVS correctly identifies and tracks the first tiger throughout, demonstrating superior global reasoning.
Exceptional Zero-Shot Generalization
Ref-DAVISI7. The generalization capability of our model best illustrated by its zero-shot performance on Ref-DAVISI7. Without any fine-tuning on the DAVIS dataset, the model trained on Yef-YYuTTube-oL VOS achieves J&FF score of 73.3. This result is not only state-of-the of-the-art but is also significantly higher than many previous methods that were explicitly trained fine-tuned similar high-quality datasets. This strong zero-shot trans- ferability suggests that our flow-based paradigm, by learning a more fundamental and continuous mapping between video and its corresponding mask guided by language, develops more general- izable understanding of spatio-temporal correspondence that is less prone to datasct-specific biases.
4.4
ABLATION STUDIES
We conduct our ablation studies on the challenging MeViS dataset, its complex, motion-centric scenarios provide a rigorous testbed for our design choices. To ensure a consistent and fair com- parison, all results are reported on the valid, set, following the protocol in prior work (Ding al. 2023). The results are summarized in Table
Analysis of Alternative Paradigms. Our investigation begins by establishing the limitations of alternative paradigms (rows a-c). The Noise-to-M Mask Flow (a), which mirrors standard generative practices, performs poorly (32.3 J&F). This confirms our hypothesis that demoting the video to
8

---

Published as a conference paper at ICLR 2026
Table 3: Analysis of VAE adaptation strategies. We measure both the mask reconstruction quality (Recon.) and the resulting performance (Perf.) on the MeViS valid, set with a fixed flow model. VAE is tuned on Me training set.
VAE Adaptation Strategy
Recon. J&F
Perf. J&F
Frozen VAE
98.8
60.0
Finetuning Decoder (ours default)
99.1
60.9
secondary condition (concatenate with noise) creates a severe information bottleneck, forcing the model to generate the mask from scratch. The Onestep Mask Prediction model (b) also struggles (36.2 J&F), validating that a single, abrupt mapping insufficient to bridge the vast represen- tational chasm between video and mask. Notably, shifting the objective from state prediction to Onestep Velocity Prediction (c) yields a substantial +14.6 J&F gain. This key result proves that learning residual (velocity) is fundamentally more stable and effective task, providing strong initial validation for our bos-b reformulation.
Effectiveness of Start-Point Focused Adaptations. Having confirmed the video-to-mask flow as the most promising direction, we dissect our proposed adaptations (rows c-h). The Base Flow model (c), trained with naive uniform sampling, performs poorly at 47.9 J JFF, even worse than the one-step velocity predictor. This confirms that a multi-step process is not inherently superior; it must be correctly stabilized. The introduction of Boundary-Bíased Sampling (BBS) provides the definitive solution. As shown in rows (f)-(f) forcing the model to focus on the start of the trajectory by oversampling
unlocks the potential of the multi-step step flow. Even moderate bias of p 0.25 (d) brings a massive +7.3 point improvement. The performance peaks at p = 0.5 (e), yielding a total gain of +10.0 J&F over the baseline. While a more extreme bias of p 0.75 (f) leads to slight performance drop, the score of 56.5 remains substantially higher than the baseline, demonstrating that the strategy is robust and effective across a reasonable range of hyperparameters. This confirms that mastering the initial, text- text-guided velocity is the most critical factor for success. Finally, Direct Video Injection (DVI) (h) provides a persistent context anchor throughout the trajectory, preventing drift and adding significant +2.0 J&F.
almost single-handediy
Effectiveness of the T2V Pretrain Model. Finally, we validate the central premise of our work: leveraging the power of large-scale T2V models. As shown in row (i), training our model from scratch without the pretrained weights (-WI) results in a complete performance collapse to 21.1 J&F. This underscores that our contributions are not generic training method, but are specifi- cally designed to effectively harness and adapt the powerful priors learned by generative foundation models for this challenging discriminative task.
VAE Adaptation and frozen VAE. A crucial step in our method is adapting the pretrained VAE to accurately transform between the latent space and the pixel space of binary masks. As shown in Table 3 we evaluate the effectiveness of the decoder adaptation. Consistent with observations in concurrent work REM (Bagchi et al. 2025). we find that the frozen VAE decoder can already support competitive segmentation performance (60.0 J F.F. However, fully finetuning the decoder further bridges the domain gap between continuous video latents and binary masks. This adaptation yields superior reconstruction quality and provides solid performance gain (+0.9 FFF over the frozen baseline.
5 CONCLUSION AND FUTURE WORK
In this work, we introduce FlowRVS, a framework that moves beyond using T2V models as mere feature extractors and instead reformulates RVOS as a continuous, text-conditioned flow from video to mask. Our core contribution is demonstrating that this paradigm shift, when combined with our proposed start-point focused adaptations (BBS, SPA. DVT, successfully aligns the generative strengths of T2V models with the discriminative demands of the task, leading to state-of of-the--n art performance. Our findings validate that the key unlocking these models lies in principled adap- tation, proving that by fortifying the flow's 's structured starting point, the philosophical gap between generative processes and discriminative objectives can be effectively bridged.

---

Published as a conference paper at ICLR 2026
Looking forward, believe the paradigm of modeling understanding tasks conditional defor- mation processes holds significant potential beyond RVOS. And our insight in stabilizing discrimi- native, pointto--criiiaall flows provide crucial blueprint for the future. As even bigger and more powerful foundation models emerge, these techniques will be essential for harnessing their full po- tential and applying their remarkable capabilities the vast amount of video understanding tasks.
ACKNOWLEDGMENT
This work was supported by Zhejiang Province Natural Science Foundation of China under Grant No. LQN25F030008 and the National Natural Science Foundation of China under Grant No. 62403429.
ETHICS STATEMENT
This manuscript does not cover any topics experiments related to ethics.
REPRODUCIBILITY STATEMENT
This manuscript provides detailed implement details, hyperparameter settings for reproduction. In addition, the code will be open-sourced later at depended acceptance
USE OF LLMS
We leverage large language models (LLLMs) to draft and polish our manuscripts, only for ensuring each sentence is clearer and the narrative flows coherently.
REFERENCES
Anurag Bagchi, Zhipeng Bao, Yu-Xiong Wang, Pavel Tokmakov, and Martial Hebert. Refer- everything: Towards segmenting everything we can speak of in videos, 2025. URL https:
//axxiv.rr//ba//2110 23287
Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Zheng Zhang, and Mike Zheng Shou. One token to seg them all: Language instructed reasoning segmentation in videos. Advances Neural Information Processing Systems, 37:6833-668 68599, 2024.
Claudia Cuttano, Gabriele Trivigno, Gabriele Rosi, Carlo Masone, and Giuseppe Averta. Samwise: Infusing wisdom in sam2 for text-driven video segmentation. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3395-3405, 2025.
Henghui Ding, Chang Liu. Shuting Hc, Xudong Jiang, and Chen Change Loy. Mevis: large-scale benchmark for video segmentation with motion expressions. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2694 2703, 2023.
Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang. Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al. Seedance 1.0: Exploring the boundaries of video generation models. arXiv preprint arXiv:2506.09) 201000016611,... 2025.
Kirill Gavrilyuk, Amir Ghodrati, Zhenyang Li, and Cees GM Snoek. Actor and action video seg- mentation from a sentence, In Proceedings the IEEE conference on computer vision and pattern recognition, pp. 5958-5966, 2018
Ming Gui, Johannes Schusterbauer, Ulrich Prestel, Pingchuan Ma, Dmytro Kotovenko, Olga Grebenkova, Stefan Andreas Baumann, Vincent Tao Hu, and Bjöm Ommer. Depthfmm: Fast gen- erative monocular depth estimation with flow matching. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 3203-3211, 2025.
10

---

Published as a conference paper at ICLR 2026
Shuting He and Henghui Ding. Decoupling static and hierarchical motion perception for referring video segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13322-1331,, 2024.
Ronghang Hu, Marcus Rohrbach, and Trevor Darrell. Segmentation from natural language expres- sions. European conference on computer vision, pp. 108-124. Springer, 2016.
Dengyang Jiang, Zanyi Wang, Hengzhuang Li, Sizhe Dang, Teli Ma, Wei Wei, Guang Dai, Lei Zhang, and Mengmeng Wang. Affordancesam: Segment anything once more in affordance grounding. arXiv preprint arXiv:2504. 1iii25044..600, 2025.
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. In Proceedings of the 2014 conference empirical methods natural language processing (EMNLP), pp. 787-798, 2014.
Anna Khoreva, Anna Rohrbach, and Bernt Schiele. Video object segmentation with language refer- ring expressions. In Asian conference computer vision, pp. 123-141 Springer, 2018.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF international conference on computer vision, pp- 4026, 2023.
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Rea- soning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9579-9589, 2024.
Gen Li, Varun Jampani. Deqing Sun, and Laura Sevilla-Lara Locate: Localize and transfer object parts for weakly supervised affordance grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10922-10931, 2023.
Tianming Liang, Haichao Jiang, Wei-Shi Zheng, and Jian-Fang Hu. Referdino-plus 2nd solution for 4th pvuw mevis challenge at evpr 2025, arXiv preprint 25X3::2503.33509 2025a.
Tianming Liang, Kun- Yu Lin, Chaolei Tan, Jianguo Zhang, Wei-Shi Zheng, and San-Fang Hu. Referdino: Referring video object segmentation with visual grounding foundations. arXiv preprint KVVv:2501.14007 2025b.
Lang Lin, Xueyang Yu, Ziqi Pang, and Yu-Xiong Wang. Glus: Global-local reasoning unified into single large language model for video segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8658 8667, 2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02277, 2022.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.088 000000000,.....00000000000000003 2022.
Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jianfeng Gao, et al. Sora: A review on background, technology, limitations, and opportunities of large vision models. arXiv preprint a7Xiv:240..17177, 2024.
Zhuoyan Luo, Yicheng Xiao, Yong Liu, Shuyan Li, Yitong Wang. Yansong Tang, Xiu Li, and Yujiu Yang. Soc: Semantic-assisted object cluster for referring video object segmentation. Advances in Neural Information Processing Systems, 36:26425-26477, 2023.
Feiyu Pan, Hao Fang, Fangkai Li, Yanyu Xu, Yawei Li, Luca Benini, and Xiankai Lu. Semantic and sequential alignment for referring video object segmentation. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19067-19076. 2025.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open-wort models for diverse visual tasks. arXiv preprint arXiv:2401. 2..2..414444 14159, 2024.

---

Published as a conference paper at ICLR 2026
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684-10695, 2022.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. XrXiv preprint arXiv:2503.2031 20314, 2025.
Dongming Wu, Tiancai Wang, Yuang Zhang, Xiangyu Zhang, and Jianbing Shen. Onlinerefer: A simple online baseline for referring video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2761-2770, 2023.
Jiannan Wu, Yi Jiang, Peize Sun, Zehuan Yuan, and Ping Luo. Language as queries for referring video object segmentation. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4974-4984, 2022.
Ning Xu, Linjie Yang, Yuchen Fan, Jianchao Yang. Dingeheng Yue, Yuchen Liang, Brian Price, Scott Cohen, and Thomas Huang. Youtube-vos: Sequence-to-sequenee video object segmenta- tion. In Proceedings of the European conference on computer vision (ECCV), pp. 385-601, 2018.
Cilin Yan, Haochen Wang, Shilin Yan, Xiaolong Jiang. Yao Hu, Guoliang Kang, Weidi Xie, and Efstratios Gavves. Visa: Reasoning video object segmentation via large language models. In European Conference Computer Vision, pp. 98 18--15 115. Springer, 2024a.
Shilin Yan, Renrui Zhang, Ziyu Guo, Wenchao Chen, Wei Zhang, Hongyang Li, Yu Qiao, Hao Dong, Zhongjiang He, and Peng Gao. Referred by multi mulli-modality: A unified temporal transformer for video object segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 6449 6449-6457, 2024b.
Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara I. Berg. Modeling context in referring expressions. In European conference on computer vision, pp. 69-85. Springer, 2016.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEF/CVF international conference computer vision, pp. 3836-3847, 2023.
Ruixin Zhang, Jiaqing Fan, Yifan Liao, Qian Qiao, and Fanzhang Li. Temporallcooniiiional re- ferring video object segmentation with noise-free text-to-video diffusion model. arXiv preprint arXiv:2508. 13584, 2025.
Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020.
Zixin Zhu, Xuelu Feng, Dongdong Chen, Junsong Yuan, Chumming Qiao, and Gang Hua. Exploring pre-trained text-to-video diffusion models for referring video object segmentation. In European Conference on Computer Vision, pp. 452-469. Springer, 2024.
12

---

Published as a conference paper at ICLR 2026
APPENDIX
A HYPERPARAMETERS SETTINGS
All our models are trained using the Adam W optimizer. We detail the key hyperparameters for our main experiments on Yeu-YouTe -ob and MeViS in Table
Table 4: Key hyperparameters for the training of FlowRVS. We detail the settings for the 2D pre- training, the main DiT fine-tuning on video datasets, and the separate VAE decoder adaptation.
Hyperparameter
2D Pre-training (RefCOCO+/g)
Video DiT Fine-tuning (Ref- YT-VOS & MeViS)
VAE Decoder Fine-tuning (on MeViS)
Optimizer Configuration
Optimizer
AdamW
AdamW
AdamW 8c-5
Peak Learning Rate
7c-5
6c-5
LR Schedule
None
None 0.
None
Warmup Steps
Weight Decay
5e-4 (0.9, 0.999)
5e-4 (0.9, 0.999)
5e-4 (0.9, 0.999)
Adam S2
Training Schedule Loss
Total Training Epochs
7/6
Global Batch Size
8
Per-GPU Batch Size
1
Gradient Accumulation Steps
1
Mixed Precision
b6loat16
bfloat16 L2 (MSE Loss)
b6foat16
Combined Focal Dice Loss
Loss Function
L2 (MSE Loss)
(a 0.25, 2.0)
B MORE QUALITATIVE RESULTS
In this section, provide additional qualitative results of FlowRVS challenging video-text pairs. These examples further demonstrate that our holistic, flow-based approach successfully handles complex language and dynamic scenes, particularly in scenarios involving significant occlusion and nuanced textual descriptions.
C COMPREHENSIVE RESULTS
While method demonstrates strong performance, it is not without limitations. We present two typical failure modes on Figure 8 For complex relational phrases requiring fine-grained interaction understanding, such as "swings tail and strikes the head" the top row, the model correctly iden- tifies the primary subject (the horse) but fails to isolate the specific horse performing the action. This suggests a limitation in comprehending intricate multi-part actions. ilar objects and temporal ordering cues ("the second bird") on the bottom row, the model struggles accurately resolve the ambiguity. incorrectly segments the first bird that moves, indicating that while our temporal modeling is strong, it can be confounded by challenging counting and ordering logic within dense scenes.
scenarios with multiple sim-
To address the question of generalization to unseen actions and challenging description, we tested our model on phrases not present in the training data, such as "the dog somersaulting." As shown on Figure 9 FlowRVS successfully identifies and segments the dog throughout its complex, non- rigid motion. This demonstrates that our method does not merely memorize action-object pairings from the training set. Instead, by leveraging the rich spatio-temport and semantic priors from the pretrained T2V model. develops a more fundamental understanding based on open-set vocabulary queries that allows it to generalize novel and dynamic actions.
13

---

Published as a conference paper at ICLR 2026
cat walking from the back to the front.
Lighter one of two fighting yaks.
White dog play with the other dog.
The bike does not move at all.
Figure 5: visualization example of FlowRVS results on Mi Vallduu
The man riding motorcycle in circles,
The ship falling sideways.
The bull that lost its balance.
The two aircrafts navigating above the sea surface and preparing to land.
Figure 6: visualization example of FlowRVS results on MeViS-
D DETAILS ABOUT PROPOSED IMPROVEMENTS
D.1 BOUNDARY-BIASED SAMPLING (BBS)
BBS is training strategy to emphasize the crucial initial step of the flow (t 0). Formally, sample the timestep t from a mixed probability distribution, whose probability density function f()
is defined as: f(1) p (() p) (1)0), 1) where 5(t) the Dirae delta function representing
14

---

Published as a conference paper at ICLR 2026
A giraffe is taking a food from a person.
A kangaroo standing in the grass and jumping away.
A person skiing on rail.
A skateboard is being rode by a person wearing red pants up the hill. Figure 7: visualization example of FlowRVS results on Ref- YouTube- VOS- VOS-Valid.
The horse that swings its tail and strikes the head of the other horse.
The second bird to reach the bottom of the cage.
Figure 8: failure cases FlowRVS results.
point mass at f = O, u(t(), 1) is the uniform distribution on the interval [0, 1}. and is the bias probability. practice, this means we sample 1 with probability (p=0.5).
D.2 START-POINT AUGMENTATION (SPA)
SPA is crucial regularizer to improve the robustness of our convergent (video-to-mask) flow. For a given video V. the VAE encoder Eg, predicts posterior distribution g(=\V) = [= )f)), where μy and o are directly inherited from the VAE. Instead of using the deterministic mean μAv, SPA samples the starting point z/ from this posterior: z6 ~ (Z2lu . This sampled latent 2/ then normalized before being used the ODE solver. By augmenting the training data with samples from the local neighborhood of each video's 's true latent representation, SPA forces the model to learn a smoother and more generalizable velocity field.
D.3 DIRECT VIDEO INJECTION (DVI)
DVI provides the model with persistent anchor to the original video content throughout the ODE trajectory. It is implemented by concatenating the current state 2, with the initial video latent žn- This changes the input tensor's shape from [B, C, T, H, W] to BB 2 C, T. H, W]. This is handled
15

---

Published as a conference paper at ICLR 2026
The dog somersaulting.
Figure 9: novel action phrases out of Me training.
by modifying the first convolutional layer of the DiT to accept 2 C input channels, while all subsequent layers remain unchanged.
16


Reviews:
[{"review_id": "73uMKaWyUx", "reviewer": "Reviewer_vQpF", "summary": "This paper proposes FlowRVS, a novel framework for Referring Video Object Segmentation (RVOS) that reimagines the task as a text-conditioned continuous flow from video to mask, leveraging the structure of pretrained Text-to-Video (T2V) generative models. Instead of the conventional “locate-then-segment” pipeline—which first grounds language into coarse geometric prompts (e.g., bounding boxes) and then segments—FlowRVS learns a deterministic, end-to-end deformation in latent space that directly maps a video’s spatio-temporal representation to its target segmentation mask under linguistic guidance.\nThe key insight is that RVOS is a convergent (video → mask) rather than divergent (noise → video) process. To adapt T2V models for this discriminative task, the authors introduce three technical innovations: \n\n1. Boundary-Biased Sampling (BBS): oversamples early timesteps to stabilize learning of the initial velocity. \n2. Start-Point Augmentation (SPA): regularizes the flow by perturbing the initial latent. \n3. Direct Video Injection (DVI): concatenates the original video latent at every ODE step to preserve global context.\n\nExperiments show state-of-the-art results across major RVOS benchmarks: \n1. MeViS: 51.1 J&F (+1.6 over prior SOTA) \n2. Ref-DAVIS17 (zero-shot): 73.3 J&F (+2.7) \n3. Ref-YouTube-VOS: 69.6 J&F\nThe approach demonstrates superior handling of complex motion, temporal reasoning, and fine-grained language grounding.", "questions": "1. Inference efficiency: How many ODE steps are used during inference? Could the method be made faster via learned solvers or step reduction without significant performance drop? \n2. Generalization to unseen verbs/actions: The MeViS benchmark emphasizes motion-centric language. Does FlowRVS generalize to novel action phrases not seen during training (e.g., “the dog somersaulting”)? \n3. Role of T2V pretraining: Would the same gains be achievable with a non-generative video backbone (e.g., ViViT) adapted with your flow framework? Or is the T2V prior essential? \n4. Decoder adaptation: What is the performance drop if the VAE decoder is not fine-tuned on segmentation masks? This would clarify whether gains are due to the flow or just better mask reconstruction.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Originality: The paper introduces a new problem formulation—RVOS as a text-conditioned flow from video to mask—departing from both discriminative pipelines and generative synthesis. This is more than a technical tweak; it’s a paradigm shift. \n\n2. Quality: Experimental design is rigorous, with ablations, cross-dataset zero-shot evaluation, and comparison to strong baselines (including VLM+SAM and grounding-model-based methods). \n\n3. Clarity: Concepts like “convergent vs. divergent flow” are explained intuitively. The three proposed techniques (BBS, SPA, DVI) are clearly motivated and linked to the core challenge. \n\n4. Significance: Demonstrates that pretrained T2V models can be repurposed for discriminative understanding tasks with proper adaptation—opening a new direction for leveraging generative priors in video understanding.", "weaknesses": "1. Computational cost: The paper does not report inference time or FLOPs. Given the use of an ODE solver with multiple steps and a 1.3B-parameter DiT, it’s unclear whether FlowRVS is practical for real-time applications. A comparison with efficient baselines (e.g., MTTR, ReferFormer) in terms of speed would strengthen the evaluation. \n\n2. Limited analysis of failure cases: While qualitative results show success on complex queries, the paper lacks discussion of scenarios where FlowRVS fails (e.g., ambiguous language, fast motion, or occlusion). \n\n3. VAE decoder fine-tuning: The VAE decoder is fine-tuned on MeViS, but it’s unclear how much performance gain comes from this vs. the flow formulation itself. An ablation on decoder adaptation would help isolate contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "euy8WcD969", "reviewer": "Reviewer_7uih", "summary": "This paper proposes FlowRVS, a method that formulates the Referring Video Object Segmentation (RVOS) task as a conditional continuous flow problem, enabling the use of pretrained text-to-video (T2V) models for RVOS. The method achieves state-of-the-art performance on major RVOS benchmarks.", "questions": "See weakness part.\n\nThere is a concurrent work titled “Temporal-Conditional Referring Video Object Segmentation with Noise-Free Text-to-Video Diffusion Model.” It would be helpful if the authors could compare their approach with this work, highlighting similarities and key differences in modeling assumptions, architecture, and performance.\nThis is not a weakness point, but rather a point of curiosity and clarification for understanding how FlowRVS relates to other diffusion-based RVOS methods developed concurrently.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper is well written in the introduction, which provides a clear motivation and overview of the problem.\n- The experimental results are strong, showing state-of-the-art performance across standard RVOS benchmarks.\n- The proposed approach leverages multiple strategies to effectively adapt a pretrained T2V model to the RVOS problem.", "weaknesses": "- Several claims are inaccurate or overstated. For example, in Lines 59–64, the paper suggests that \"locate-then-segment\" is the dominant paradigm. In fact, it is only one of several common approaches. Moreover, language queries are not often decoupled; many recent methods (e.g., ReferFormer) utilize textual features throughout the entire temporal segmentation process, applying cross-attention between text and video features rather than limiting language conditioning to the localization stage.\n- The idea of applying T2I/T2V generation to RVOS is not novel. Prior work such as VD-IT [ECCV 2024] has already introduced the use of pretrained text-to-video diffusion models for RVOS.\n- The training setup is inconsistent across datasets and lacks a unified protocol or rationale (L316-317), making fair comparison and generalization less convincing.\n- The technical descriptions are insufficiently detailed, particularly for modules such as BBS, SPA, and DVI (Page 5). As currently written, the paper would be difficult to reproduce.\n- There is no evaluation on Long-RVOS benchmark, leaving uncertainty about how the generative method performs on long-term temporal reasoning or consistency over extended sequences.\n- The paper does not include failure case analysis or discussion of limitations, which would help clarify the method’s robustness and boundaries of applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OHXHFQJBcF", "reviewer": "Reviewer_RnKe", "summary": "This paper proposes FlowRVS, a one-stage framework that reframes Referring Video Object Segmentation (RVOS) as a text-conditioned continuous flow matching from a video sequence to its target mask sequence. Instead of leveraging previous  “locate-then-segment” paradigm, FlowRVS learns a velocity field (via flow matching) that deterministically deforms video latents into mask latents under language guidance. To adapt existing video generation models to RVOS task, the paper proposes three techniques to stabilize the convergent video→mask flow: Boundary-Biased Sampling (BBS), Start-Point Augmentation (SPA), and Direct Video Injection (DVI).  The proposed FlowRVS achieves new SOTA on MeViS and strong zero-shot results on Ref-DAVIS17, leveraging the Wan2.1 backbone.", "questions": "- How is the VAE fine-tuned in practice? Is it fine-tuned on the Video-frames→Mask task, or purely as an autoencoder on mask images?\n\n- For the alternative paradigms: how concretely are the direct mask prediction (and velocity prediction) variants based on Wan implemented? Are these obtained by supervised post-training on Wan2.1?\n\n- Regarding the SPA modification (L247): “we transform the initial video latent $z_0$ through a stochastic encoding and normalization process.” What exactly is the implementation of this stochastic encoding and normalization? How sensitive are the results to the type and degree of stochasticity?\n\n- Table 3 suggests a fine-tuned VAE decoder is crucial. In Table 2, did all underperforming alternative paradigms also use a fine-tuned VAE decoder? If yes, please state this explicitly; if not, could their poorer performance are due to the lack of VAE fine-tuning?\n\n- In Table 2, the final performance drops when BBS = 0.75. Do you have explanations or analysis for this behavior?\n\nReferences\n[1] Wiedemer, Thaddäus, et al. “Video models are zero-shot learners and reasoners.” arXiv:2509.20328 (2025).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper presents a novel perspective by formulating video segmentation as video sequence -> mask sequence via an ODE, thereby directly leveraging the capabilities of modern video generation models.\n\n- The overall exposition is clear and logically structured, making the method easy to follow. The design choices appears to be reasonable.\n\n- Experiments demonstrate that harnessing video generation models can be effective for RVOS. The findings are interest and could be related to recent work on zero-shot visual understanding with video generators (e.g., studies on Veo3 [1]), further highlighting emergent visual understanding abilities inside video generation models.", "weaknesses": "- The manuscript claims (L493) to provide “detailed implementation details and hyperparameters for reproduction,” yet the main text (and appendix) gives very little in the way of concrete implementation. In particular:\n\n(a) how the three alternative paradigms are instantiated; and\n\n(b) the formal specification and practical realization of the three proposed improvements.\n\nThis lack of detail undermines completeness and reproducibility. Please see “Questions” for specific questions.\n\n- Although RVOS accuracy improves, the paper does not report or discuss the impact on inference speed and computational cost when introducing a video generation model into the segmentation pipeline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aGfH6y7TV2", "reviewer": "Reviewer_bX1k", "summary": "This paper introduces FlowRVS, a novel framework designed to address the Referring Video Object Segmentation (RVOS) task by reframing it as a flow matching problem. The method proposes a Flow Matching Model that predicts continuous deformation fields. This design inherently promotes superior temporal coherence and avoids the compounding errors typical of sequential prediction stages. Extensive experiments conducted on standard RVOS benchmarks, including Ref-YouTube-VOS and Ref-DAVIS, consistently demonstrate that FlowRVS achieves state-of-the-art results", "questions": "1. What is the training and inference computational overhead (e.g., GPU hours, FLOPs per frame) for FlowRVS compared to SAMWISE?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The core idea of replacing the sequential segmentation pipeline with a unified Flow Matching Model represents a significant conceptual leap for RVOS.\n2. The quantitative comparisons presented in the paper showcase substantial performance gains over existing state-of-the-art techniques.\n3. The paper is well-written and logically structured, with clear figures.", "weaknesses": "1. The definition of the two-stage 'locate-then-segment' design is potentially misleading; many modern Transformer-based (e.g., ReferFormer) and MLLM approaches (e.g., LISA)  already bypass explicit bounding box output and generate segmentation masks directly. \n2. The discussion in Section Introduction neglects relevant previous methods, such as SAMWISE and others, which explicitly incorporate a 'temporal segmenting' stage. \n3. The related work section provides insufficient discussion regarding Generative Modeling in VOS or RVOS, specifically lacking recent efforts like \"Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation,\" which is highly relevant to flow-based generation. \n4. The network structure and update strategy for the Flow Matching Model would be significantly clearer and more reproducible if they were enhanced with formalized mathematical expressions and equations. \n5. Despite stating that the foundation model used is Wan2.1, the overall architectural logic of the proposed work, particularly as inferred from the high-level design figures (e.g., Figure 3), shares a similar idea with the ControlNet framework. The authors should explicitly clarify the differences or similarities with a conditional generation framework like ControlNet.\n\nIn summary, despite the compelling performance gains, the cumulative impact of these missing discussions regarding prior work and the lack of technical detail in the model's description remains a significant issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)