EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning
Authors: Xuan Ju
Tianyu Wang
Yuqian Zhou
He Zhang
Qing Liu
Nanxuan Zhao
Zhifei Zhang
Yijun Li
Yuanhao Cai
Shaoteng Liu
Daniil Pakhomov
Zhe Lin
Soo Ye Kim
Qiang Xu
Venue: ICLR 2026 Oral (2026)
Keywords: Video Editing
Content Generation
Artificial Intelligence
Primary area: generative models

Abstract: Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While image generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due to architectural limitations and data scarcity. In this work, we introduce EditVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and outputs with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video editing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that EditVerse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while exhibiting emergent editing and generation abilities across modalities.

EDITVERSE: UNIFYING IMAGE AND VIDEO EDITING AND GENERATION WITH IN-CONTEXT LEARNING
Xuan Ju
Tianyu Wang!
Yugian Zhou
He Zhang'
Qing Liu'
Nanxuan Zhao
Zhifei Zhang
Vijun Li
Yuanhao Cai
Shaoteng Liu
Daniil Pakhomov
Zhe Lin
Soo Ye Kim
Qiang Xu
AAdbee Research
CUHK
Johns Hopkins University
Shenzhen Loop Area Institute
Video Editing
Originll
Video
Video
Change
Remove
Add
Camera
Magical
Change
Change
Taska
VIDEO
Inpainting
Change
Crnginal
Reforence
orginal
Video
Wieather
Video
Mask
video
to-Video
Image Editing
Poss
Objest
5tyl0
Camaro
Figure 1:
The strong video editing performance of Edit emerges from a unified framework trained on diverse set of mixed image and video data. This teaser visualizes a selection of supported image and video editing tasks (Instructions in the Appendix). More results in our Project Page
'This work was done when Xuan Ju was an intern at Adobe Research. Corrrsponndig author.

---

ABSTRACT
Recent advances in foundation models highlight a clear trend toward unification and scaling, showing emergent capabilities across diverse domains. While im- age generation and editing have rapidly transitioned from task-specific to unified frameworks, video generation and editing remain fragmented due architectural limitations and data scarcity. In this work, we introduce EddiVerse, a unified framework for image and video generation and editing within a single model. By representing all modalities, i.e., text, image, and video, as unified token se- quence, Verse leverages self-attention to achieve robust in-context learning, natural cross-modal knowledge transfer, and flexible handling of inputs and out- puts with arbitrary resolutions and durations. To address the lack of video editing training data, we design a scalable data pipeline that curates 232K video edit- ing samples and combines them with large-scale image and video datasets for joint training. Furthermore, we present Veiseeeee,,,,,, the first benchmark for instruction-basse video editing covering diverse tasks and resolutions. Extensive experiments and user studies demonstrate that Verse achieves state-of-the-art performance, surpassing existing open-source and commercial models, while ex- hibiting emergent editing and generation abilities across modalities.
INTRODUCTION
Recent advancements of foundation models in computer vision and large language models highlight clear trend toward unification and scaling (Achiam et al.] [203] Zhou et al. [0244] Deng et al.)
2025), showing that joint training on diverse datasets can unlock emergent intelligence. Specifically
image generation and editing, there is also shift from domain-specife models (Zhang et al.
2023b Ju et al., 2023b Li et al.) 2024) toward universal models (Labs et al. 2025 Chen et al,,
2025ch that unify diverse generation and editing tasks under a generalized and scalable framework.
However, unlike the image domain, the exploration of unified video generation and editing remains
limited (Jiang al. 2025 Ye et al. 20256). This stems from two primary challenges: (1) Archi- tectural Limitations: Existing video generation models, mostly based on cross-attention (Polyak)
2025 or MMDiT (Yang et al. 2024c; Kong et al. (024)) architecture, typically designed for specific tasks such as texteoo-video generation. Adapting them to support various video generation and editing tasks introduces substantial design and scaling challenges. For example, VACE (Jiang et 2025) uses an additional branch that accepts unedited videos and masks as input, transforming a text-to-video model into video inpainting model. However, it relies on masks to localize the editing regions and requires task-specific input configurations, making it less practical for real-world use. To unlock emergent abilities with in-context learning, fully unified framework must be able to process diverse input modalities (e.g., text, image, video) and types (e.g., duration, resolution) with a consistent and flexible representation. (2) Data Scarcity and Diversity:
al.) 2025 Wan et al.
Unlike the vast and varied datasets readily available for image editing Yuu et al. 2024 Ye et al., 2025a Chen et al. 2025b), high-quality and diverse video editing datasets are significantly scarce.
To address this challenge, we propose EditVerse, unified framework that enables image and video editing and generation within a single model, leveraging full self-attention to enable robust in- context learning and effective knowledge transfer between images and videos. Our design considers two aspects: (1) In-Context Learning: We represent all modalities (text, image, and video) as a uni- fied one-dimensional token sequence, which is then concatenated and fed into the model as a long sequence. This design enables the use of full self-attention with strong in-context learning capabili- ties (Ju et al., (2055) to jointly model and align different modalities. As result, Verse achieves enhanced text comprehension. improved image and video editing quality, and most importantly, nat- ural rross-modal knowledge transfer between images and videos, which effectively alleviates the limitations caused by the scarcity of video editing data. (2) Flexibility: We use an interleaved de- sign for text, image, and video, inspired by the native generation architecture of multimodal large language models (MLLM), which are well-suited for supporting diverse tasks and interactive gen- eration. This design enables the model to process image and video inputs and outputs with arbitrary resolution, temporal duration, and sequential position, thereby providing enhanced flexibility. To further distinguish positional and modal information, we introduce a four- dimensioinl Rotary Posi- tional Embedding (RoPH) that incorporates sequential, temporal, height, and width dimensions.
2

---

While careful model design is crucial, simply training it image editing data is insufficient to enable the model to perform various video editing tasks. Based on the observation that open-source instruction-based video datasets (Zi et al. 2025) are inadequate in both volume and quality, devise a data pipeline that first generates video editing samples with task-specific models, then filters high-quality samples from the generated samples. For unified training, we mix such curated video editing data (232K) with 56K samples filtered from Señorita-2M as well as 2M image generation samples, 6.M image editing samples, and 4M video generation samples.
At last, due to the absence of instruction-based video editing benchmarks encompassing diverse tasks and mixed resolutions, we introduce Edit to enable a more comprehensive eval- uation. It contains 100 videos, evenly divided between 50 horizontal and 50 vertical formats, with each video paired with two editing prompts different editing tasks. Each data instance includes an editing instruction, source prompt, and a target prompt, spanning 20 distinct video editing categories. Comprehensive evaluations (both automated and user studies) demonstrate that Edit- Verse achieves state-of-the-art performance compared to existing open-source methods as well as commercial models. Moreover, experiment results show the model's capacity for knowledge trans- fer from image to video domain and reveal emergent abilities arising from our proposed design.
RELATED WORK
Instruction-base Image and Video Editing Datasets. In recent years, the field has witnessed a surge in large-scale, open-source datasets for instruction-based image editing. Increasingly sophis- ticated data annotation pipelines have been designed and continuously improved, advancing from earlier methods doing large-scale annotation using editing models with lower success rates (e.g., InstructPix2Pix (Brooks et al.] 2023) and HQ-Edit (Hui et al.] (024)) small-scale manual labeling (e.g., MagicBrush (Zhang et al., 2023ap), to advanced techniques that leverage well-trained task- specific models and pipelines to generate better quality data at large scale (e.g.. UltraEdit (Zhao) cct al. 2024), OmniEdit (Wei et al. 2024), AnyEdit (Yu et al. 2024), SEED-Data-Eue
(Ge et al. 2024 and Edit (Yang et al., 2024b)). Data quality improvement further boosts the perfor- mance of instruction- based image editing models (OpenAl 2024 Labs et al. 2025), which are then served as data source of open-source datasets (e.g., ShareGGPT-4o-maaa (Chen et al. 2025b)).
However, video editing datasets progress at a slower pace. Ins V2V (Cheng 2023) uses Prompt- to-Prompt (Hertz al. 2022) and large language model (LLM) to create its video editing datasets, where the low performance ceiling of Prompt-to-Prompp leads to poor dataset quality. Although VIVID-IOM (Hu et al. 2024) provides collection of videos with corresponding textual instructions and mask annotations, it lacks paired ground edited videos, making it unsuitable for training Instruction-based video editing models. Señorita-2M (Zi et al.) 2025) builds an instruction- based video editing dataset using task-specific diffusion models. However, when compared to datasets in image editing, exhibits notable limitations in both quality and editing diversity. In conclusion, datasets for instruction-based video editing are significantly less mature than the image domain, necessitating architectural innovation to transfer editing capabilities from image to video.
Image and Video Editing. The success of diffusion models has led to rapid progress in image and video editing. Since most pre-trained models are designed for text-to-image and text-to-video generation, early research explores training- free image and video editing techniques based on these
models, often by manipulating attention maps or latent spaces (Hertz et al. 2022 Cao al. 2023 Ju
et al. 2023a Wang al.) [023a] Qi et al., 2023 Liu al. (024b) Yoon et al. 2024). Despite their simplicity, such techniques frequently yield unsatisfactory results characterized by a lack of precise control and low quality. Consequently, the field has shifted mainly to data-driven criinin-bbased methods. For image editing, methods such as InstructPix2Pix (Brooks et al. 2023) and subsequent works (Sheynin et al. 2024) concatenate the unedited image latent with the noisy latent along the channel dimension. directly fine-tuning text-to-image models for editing tasks. Later studies (Ju al. 2025 Xiao et al.] 2025; Cai et al. 2025a Zhang al., 2025b) find that sequential concate- nation benefits model learning by using self-attention to improve in-context learning, which is a design choice that also aligns with the architectures multimodal LLMs' native image generation (e.g.. BAGEL (Deng et al. 2025), transfusion (Zhou et al. 2024)). While similar techniques can be employed for video editing. investigations into instruction based video editing are relatively rare. EVE (Singer al. 2024a) trains adapters top of frozen text-to-image models to enable video editing ability. Ins V2V (Cheng et al. (202)) extends InstructPix2Pix (Brooks et al. 2023) to a video version. GenProp (Liu al. 2025a) propagates the edits in the given first frame to the following
3

---

frames. Recent work UNIC Ye et al. 2025b) concatenates conditions sequentially, similar image editing architecture designs, and supports six editing tasks with task-aware positional embeddings. However, these methods still fall short in supporting flexible instruction- based video editing tasks.
3 METHOD
As illustrated in Figure 2 Edit Verse employs transformer architecture with full self-attention (Chen ict al. 2025c Ju et al. 2025). All text and vision inputs are tokenized and concatenated into uni- hed sequence in interleaved manner, then fed into the model and processed with self-attention blocks (Section 3.1. To accommodate this interleaved design, we design a four-dimensional Ro- tary Positional Embedding with spatial (height and width), sequential, and temporal dimensions (Section 3.2). For training and inference, Verre predicts the visual velocity (Esser et al., 2024 Lipman 2022) that guides the generation of images videos through denoising procedure
(Section 3.3
Subsequent sections detail our framework and the insights behind our design choices.
Text T5 Taken
Clean/Naity Image/Viden VAE Takan
Start End of Vision Taken
Image
IIsSSSSSS
N
Sequential Fostional Emboddiing
Linear
Linear
Linear
Linear
Linear
ANNSSSS
Temporal Enthadling
N x 3D Full Attention Blocks
(with fully concatenated interieaved text and visual latent)
Spatial Embediing
NeS
Predicted Velocity
Rotary Embedding
Figure 2: Overview of VditVerse. We design unified framework for image and video editing and generation, which processes text and vision inputs into a unified sequence. The right part of the figure shows our positional embedding design. This framework leverages full self-attention to facilitate robust in-context learning and effective knowledge transfer among modalities.
3.1 INTERLEAVED TEXT AND VISION INPUT
Following prior works (Kingma & Welling 2013), we encode the RGB pixel-space videos and im- ages into a learned Autoencoder (VAE) capable of both feature extraction and reconstruction. Specifically, given in- put image or video Joisions the VAE compresses into a continuous- with downsampling ratios rT, TH, Tw. Then, the vision features are patchified into long token sequence with a x x 2 kernel to get Xpision E IR Leissinn XCvision (Lvision is the vision token number, Crision is the channel dimension of vision feature). For given text input Iteats first generate text tokens using the Flan-T5-XXX (Chung et al., (002) encoder. Then, we retain only the tokens that correspond directly to the input text, discarding the rest, yielding a final representation Xtert E R frces Creet (Liext is the token count of Lezts Ctext is the channel dimension of Flan-Te T5- XXL), which saves computation while preserving the necessary information from text input.
compressed latent space by training a convolutional Variational
latent representation
Clean/Noisy
& End Vision
More Examples
depicts scene
beach
Generate video
by remove lighthouse
Edit the video following the first frame
Remove the parrot
Generated result:
in the mask
Concatenated Token Sequence
Figure 3: Examples for the interleaved text and vision pattern. Verse is capable of processing image and video inputs and outputs of arbitrary resolution, duration, and sequential positions.
4

---

To handle instructions composed of arbitrary combinations of text, images, and videos, we unify all modalities into a single interleaved sequence representation (shown in Figure 3 First, we project the tokens from each modality into shared embedding space using separate single-layer linear projectors. This maps both text and visual inputs to the model's s hidden dimension, C, yielding two distinct embedding matrices: tteat_atign E R and Xgision_aling align Reovsoo XC, Then, concatenate the projected embeddings to construct unified input sequence, X RLxC, where L denotes the total number of text and vision tokens. The sequence preserves the original interleaved order of text and visual elements from the instruction. To explicitly indicate the location of vision tokens (images and videos) within an interleaved sequence, we add learnable "start of vision" token and learnable "end of vision" token at the beginning and the end of each vision token segment.
3.2 ROTARY POSITIONAL EMBEDDING
To distinguish text, image, and video from each other and to indicate their sequential positions, we design a special Rotary Positional Embedding (RoPE) that incorporates sequential, temporal and spatial (height and width) dimensions (shown in Figure 2 For each of these four positional dimensions, we apply a separate RoPE computation. (1) Sequential dimension: This dimension captures the global position within the overall sequence, starting from 0. The value is incremented by for each text token and image/video frame, up to the end of the sequence. (2) Temporal dimension: This dimension is used exclusively for video frames encode their temporal order within video clip. It begins at 0 and increases by for each subsequent frame. For text and image inputs, this dimension remains 0. (3) Height and Width Dimensions: For images and video frames, the height and width dimensions correspond to the pixel coordinates, increasing incrementally from the top- left to the bottom-right corner (Polyak al. 2025). The increment values reflect the number of pixels along the height and width axes. For text tokens, both dimensions are set to O. The sequential, temporal, height, and width dimensions each compute separate RoPE, which are assigned RoPE embedding dimensions of 12, 4. 56, 56 respectively. To better support variable-length input, we use the NTK- interpolation (Peng et al. (023) in RoPE calculation for context window extension.
3.3 TRAINING AND INFERENCE PARADIGM
Given an interleaved sequence X1 = Concau(X (x xx ...., x(")), where each x represents a clean image. a video, or a text segment, and the total number of visual or textual segments, we randomly select one image or video x as the generation target, optimizing with the Flow Matching (Lipman et al. 2022) training objective. In the diffusion process with the formulation of Flow Matching, noise sample X ~ 1), is progressively denoised into clean data x(" with X = + (1 x)xx()) where timestep i E [0, 1]. The learnable model is trained predict the velocity V, axx()) which can be further derived as: V ax(()) = x(1) x( Thus,
with input sequence X, Concat(X ...... x-(1) ..... x(")), the model with parameter θ is optimized by minimizing the mean squared error loss L between the ground truth velocity and the model prediction, where Xo- Concat(X ................................ x() ...,,,(()))
C - E, X, Iue (X,,t) t) - (X1 xxo)2
During inference, the difffusion model first samples x! NN(0,, 1), then uses an ODE solver with a discrete set of N timesteps to generate X, from Xo.
DATA PIPELINE
Edit Verse is trained on large-scale data composed of: 1.9M image generation samples (around 2.0B tokens), 3.9. video generation samples (around 68.8B tokens), 6.0MM image editing samples (around 12.6B tokens), and 288K video editing samples (around 10.2B tokens). Notably, the video editing datasets have significantly smaller sample number and are less diverse than image editing datasets. Thus, our architecture is specifically designed to transfer learned editing knowledge from the image domain to the video domain. We summarize the used datasets in Table [ and provide further details below. A more detailed breakdown for each dataset is provided in Appendix Table TO
5

---

Video Editing Data Pipeline. Due to the scarcity and inadequate quality of publicly available video editing datasets, developed our pipeline to gen- erate Veit Editing Data, which can be ap- plied to obtain video editing pairs from any video input. (I) Object Removal and Addition. We first use Grounded-SAM-2 (Ravi et al., 2024; Ren et al. (024) to extract object masks from the video. To improve the success rate of object removal, we filter candidates based on object name, total mask area, and detection confidence score. Next, we apply DiffuEraser (Li et al. (025) to remove the masked objects. We use video pairs before and
Image Datasets
Dataset
#Samples
MagicBrush
9K
SharcGPT-4o--maann
46.K
Object Removal & Addition!
119K
Omnilidit
186K
Edit
ImgBdit
246K
NHR-Edit
358K
UltraEdit
500K
AnyFdit
1.2M(2.5M)
GPT--mage-Edit-1.5
1.5M
Instruction-based Editing?
L8M
BI.I3--o 60K
60
Gen
LLaVA-pretrain
500K
after the removal to construct object removal and
Text-to-Image'
610K
addition data. (2) Object Change. Again, we obtain object masks from Grounded-SAM-22 (Ravi iet al. 2024 Ren et al. 2024). Next, we leverage a Vision-Language Model (VLM) (Wang al. 2024) to imagine plausible transformations of the object.
LLaVA-next fine-tuning
700K
Video Datasets
Dataset
#Samples
Gen Edit
Scñorita-2M
(6K(2M")
We then use VACE (Jiang et al.] 2025) to inpaint the masked region based on the VLM's 's output. To improve the success rate, we apply dynamic adjust-
Verse Editing Data
232K(13M2
Text-to- Videot
223K
Customization
740K
ments to the mask's shape and area, conditioned on the object's size and geometry. (3) Style Trans- fer. Previous style transfer techniques primarily rely on inference-basee video editing methods (Qi et al. 2023), which we found to be unreliable when han- dling more diverse styles (e.g., Minecraft style). To address this, we first apply an image style transfer model to edit the first frame, and then utilize VACE's (Jiang et al. 2025) depth-guided first-frame- to-video feature to generate the full styled video. (4) Camera Change. We select 10 camera movements and use ReCamMaster (Bai et al., (2225) to generate camera change data. (5) Mask Detection. We construct the mask detection dataset by converting object removal, object addition, and object change data using the prompt template: "I want to [edit prompt]. Detect the region that needs to be edited". (6) Propagation. We build the propagation dataset by extracting the first edited frame from style transfer, object removal, object addition, and object change data.
VditVerse Gen Data
3.0M
"Dataset volume before filtering. Internal dataset.
Table 1: Statisties of the training datasets. We mix open-source datasets, internal datasets, and EditVerse datasets for unified training. Detailed information in Table I0]
In addition, we incorporate data from the open-source dataset Señorita- (Zi et al. 2025). How- ever, we observe relatively low success rate in this dataset, necessitating extensive filtering.
Video Generation Data Pipeline. Since we start from a pretrained model capable of text-to-image and text-to-video tasks, we only use small scale of pure text-based generation data (223K sam- ples for -xx--o) to preserve the model's inherent generative capabilities while simultaneously introducing controllability and enhancing its text comprehension via control tasks. For controllable and video-to-control data pairs (including depth,
video generation, we annotate control-to-videe sketch, and pose), where the depth map is annotated with Depth Anything v2 (Yang et al. 2024a), human pose is annotated with RTMPose (Jiang et al.] 2023). ) and sketch is annotated with OpenCV Canny Edge Detection (Itseez] 2015). Moreover, we also include annotations for test-framm-too-idde generation data and video inpainting data annotated with GroundeddSAAM- (Ravi et al. [024 Ren Øt al., 2024). The combined data from control-to to-- to-video, video-to-controd video inpainting are referred to as VditVerse Gen Data in Table [] customization dataset to support reference-bait based generation (Cai et al. 20256).
Lust-frram-ter to-ieeo,, and Additionally, we include a video
Image Editing. After reviewing the data quality of existing image editing datasets, we incorporate 8 high-quality open-source datasets: MagicBrush (Zhang et al., al. 2023a), ShareGPT-4o--mage (Chen et al.] 2025b), OmniEdit (Wei et al.) (004)). ImgEdit (Ye et al. 2025a). NHR-Edit (Kuprashe- vich et al., 2025), UltraEdit (Zhao et al. 2024), AnyEdit (Yu et al. 2024). and GPT-Image-Edit- 1.5M (Wang et al.) 2025). In addition, we incorporate two internal image editing datasets: one focused on image addition and removal, and the other on free-form instruction based image editing.

---

Image Generation. For text-to-image include 610K internal text-to-image samples well as several open-poeneeeoo non--o image understanding datasets (BLIP3-o 60K (Chen et al. (025a). LLaVA- pretrain (Liu et al. 2023), and LLaVA-next fine-tuning (Liu et al. 2024ah) that contain high-quality text annotations, which can improve the editing instructions understanding ability.
Data Filtering. Since the training data is model-generated and contains errors, filtering is vital for curating high-quality examples. We used a VLM (Wang et al., (022)) to filter the dataset by scoring both editing and video quality, The scores covered instruction adherence, context preservation, video sharpness, temporal consistency, artifact presence, object integrity, aesthetics, and physical plausibility. To determine the final filtering criteria, we manually inspected the relationship between the VLM scores and the editing quality. Based on this inspection, we defined a set of score thresholds to select the final training dataset. As shown in Table [ our video editing pipeline achieves retention rate six times higher than Señorita-2M after filterin, demonstrating high editing quality.
EXPERIMENTS
5.1 IMPLEMENTATION DETAILS
Edit Verse is trained on 2B dense transformer architecture similar to LLaMA 3 (Dubey et al. 2024) initially pretrained on text-to-image and text-to-video data to get basic generative capabilities at resolution of 360p. Then, we train the model on our dataset as listed in Section + For each image/video, we resize according to original aspect ratio so that its area falls between 256x256
and 512x512. During training, we use global batch size of 256 and train for around 56K steps. We use AdamW optimizer (Loshchilov et al. 2017) with saper--arameters set to 31 0.9, 3g = 0.95, peak learning rate of - and weight decay of 0.01. We use warm-up of 2K steps and cosine decay learning schedule, decreasing the learning rate to the minimum of le 1 We set the gradient clipping norm to 1.0 and disable gradient clipping during the warm-up stage. Since the training data consist of token sequences with variable lengths, making it difficult to form batches, we adopt the packing strategy introduced in KnapFormer (Zhang al. 2025a). During inference, classifier-free guidance scale of 5.0, applying it only to text conditions. The inference timestep is set to 50 for the balance performance and inference speed.
Remove Object
Change Object
Add Object
Stylization
Inpainting
Propagation
<v1>> Add
<>1>> Remove background poople.
<y1> Replace stop sign with groen go sign.
Inpaint this video <>1>>> with mask <v2>, A woman is excrcising in a bright room.
<vI>> Change watercolor.
edit video following the first frame <|1>.
rainbow in the background.
Change Background
Change
Change Material
Change Camera
Reference Insertion
Reasoning
Add Effect
Color
<v1> Make
<v1> Change back background
<y1> Change the table golden.
<v1> Change the turtle to crystal.
<v1> Change the camera Pan Left.
<v1> Add
<<1>> Insert a paper boat <i1> in the water.
the dirty
time-lapse
animal clean.
Mars surface.
effect sky.
Change Weather
Pose-to-
Depth-to- Video
Sketch-to- Video
Combined Task
Detection
Edit with Mask
Video
<v1> Change the weather to snowstorm.
<v1> Detect the mask of the bird.
< A young woman sits the floor.
<v1> Colerful roses sway in the breeze.
<v1> Three young women stand together.
<v1> Add glass, beach background
<v1> remove the blue parrot shown mask <y2>,
Figure 4: Examples from the proposed Verseeereeenchh EditVerseBench includes 200 editing pairs, evenly distributed across 20 editing categories as well as horizontal and vertical orientations.

---

VIM evaluation
Video Quality
Text Alignment
'emmporl Consistency
Method
Editing Quality
Pick Score 1
Frame t
Video
CLIP 1
DINO t
Attention Manipulation
(Training-free -reiin
TokenFlow
5.26
19.73
25.57
22.70
98.36
98.09
STDF
4.41
19.45
25.24
22.26
96.04
95.22
First-Frame Propagation (w/ End-to-Eno Training)
Señorita-2M
6.97
19.71
26.34
23.24
98.05
97.99
Instruction-Guidee (w/ End-to-End Training)
Ins Ins
5.21
19.39
24.99
22.54
97.15
96.57
Lucy Edit
5.89
19.67
26.00
23.11
98.49
98.38
EditVerse (Ours)
7.65
20.07
26.73
23.93
98.56
98,42
Closed-Source Commercial Models
Runway Aleph
7.44
20.42
27.70
24.27
98.94
98.60
Table 2: Quantitative comparison on EditVerseBench. For open-source research models, we compare two training-free methods (TokenFlow and STDF), one first-frame propagation method (M), and one instruction-guider video editing method (Ins Best results are high- lighted in bold. We also provide the results of commercial model, Runway Aleph. While Edit- Verse lags Runway Aleph in generation quality due to base model differences, our proposed method Edit surpasses it in editing faithfulness (via VLM evaluation editing quality), aligning better with human judgment that is further validated by user studies shown in Figure
5.2 EDITV BRSE BRSEBENCII
Commonly used video editing benchmarks (e.g., V2VBench (Sun et al., 2024), TGVE (Wu et al., 20236 Singer et al. 2024b)) only consist square videos and are primarily designed for training- free editing (Qu et al. 2025 Yatim et (004)) rather than instruction -based editing. Moreover, such benchmarks not adequately cover the diverse editing tasks commonly encountered in real-world video editing scenarios. To address these limitations, we propose Edit VerseBecn comprehensive instruction- video editing benchmark composed of 20 distinct instruction- based video editing tasks. We manually selected 100 videos from free stock website (Pixabay) 2025) that cover variety of scenes, including 50 horizontal and 50 vertical videos. For each video, we randomly select two editing instructions from the 20 editing tasks. This results in total of 200 editing pairs (5 horizontal and vertical videos per editing task). We show one example from each editing category in Figure To evaluate editing performance
our proposed BerseBeBB,,,, we use 6 metrics covering four aspects: VLM evaluation, video quality (frame-wise Pick Score (Kirstain al., 2023)). text alignment (CLIP (Radford et al. (222)) text-image and VICLIP (Wang et al. 2023b) text-video alignment), and temporal consistency (frame-wise CLIP (Radford et al. 2021 and DINO (Caron al. 202T) consistency). Details can be found in the Appendix.
5.3 COMPARISON TO PREVIOUS METHODS
We show comparisons of Edi and TGVE+ (Singer et al. 2024b) in this section. More comparisons (e.g., V2VBench (Sun et 2024 and image editing) are provided in the Appendix.
Comparison on Editeerree.cch. Since InsV2V (Ceng et al. 2023) and Lucy Edit (Team] 2025)
win
tie
lose
the only open-source instruction-based video edit- ing method that exactly matches our setting, we se- lected two well-known training-free methods, To- kenFlow (Qu et al. 2024]. as well as iirst-frame propagation method, Señorita-2M et
51.7%
26.3%
22.0%
Runway Aleph
EditVerse
63.2%
33.9%
Lucy Edit Dev
2025) and STDF (Yatim al.
96.9%
InsV2V
et al. 2025), for comparison on Ed- Bereee. We use the first frame of our results as input to Señorita-2M. Moreover, we also compare to a commercial model, Runway Aleph (Runway) 2025). As shown in Table 2 Vditeerse outperforms Figure 5: User study on Edit VerseBeeenn previous research models on all metrics, demonstrating the effectiveness of our proposed method. Figure [ shows visual comparisons on Edit VerseBBrr..n. We further conduct a user study to assess human judgments of editing performance. The evaluation criteria include (i) instruction alignment,
97.0%
TokenFlow
98.6%
STDF
8

---

(ii) preservation of unedited regions, and (iii) overall video quality. We collected 3, 000 pairwise ratings comparing Edit Verse against each of the other methods, with the results summarized in Fig- demonstrating the state-of- the-art performance of our proposed Edit We find the user study result is more aligned with the VLM evaluation metric in automatic evaluation.
ure
Transform the person's hair into realistic flames. The fire should flicker and flow upward, following the motion of the person's head. (a) Source Video
(b) TokenFlor low
(c) STDF
(e) Scñorita-2M
Error Accumulation
(1) Runway Aleph
Poor
(g) Ours
InsV2V
Preservation
Figure 6: Visualization of VditVerse and other video editing methods. Vditeerse shows stronger
context preservation and edit faithfulness. Complete comparisons are the Appendix.
Following Movie Gen (Polyak et al. 2025), we eval- uate Verte on TGVE+ (Singer al., 2024b). Specifically, we follow previous works and measure (i) ViCLIP text- video direction similarity, which evaluates the alignment between changes in cap- tions and corresponding changes in the videos, and (ii) ViCLIP out output similar- ity, which measures the similarity between the edited video and the output caption. As shown in Table Edit Verse surpasses pre- vious methods on both metrics. It is worth noting that all TGVE+ videos are square, whereas our training data does not include any square video editing samples.
Comparison on TGVE+
Method
ViCLIPa
VICL.Pout out
Tume-A- --ideo (Wu et al.
2023a
0.131
0.242
TokenFlow (Qu el ar 2023
0.128
0.237
STDF Yatim et 2024 Fairy
0.093
0.227
0.197
InsV2V Cheng
2023
0.174
0.236
SDEdit (Meng el al
0.131
0.241
EVE (Singer of al.
20248
0.198
0.251
Movie Edit (Poyykk al. 2025
0.225
0.248
EditVerse (Ours)
0.225
0.252
Table 3:
Quantitative comparison on TGVE+. Re-
sults show superior performance of EditVerse
5.4 ANALYSIS OF EMERGENT ABILITY
Emergent ability one of the most exciting phenomena observed in large-scale model training, aris- ing data and model capacity increase. In this section, specifically analyze this phenomenon.
Demonstration of emergent ability. We show the emergent ability of video editing in two aspects: (1) the model can perform editing tasks that included the training data, the model's performance can even surpass the ground-truth quality.
present in the training data, and (2) for tasks
The video editing training data covers only limited set of tasks, including camera changes, style transfer, mask detection, object modification (addition, removal, or replacement), and propagation. However, shown in Figure our model is capable of performing tasks beyond the training distri- bution (e.g., change material, change weather, add effects). Furthermore, it can also handle multiple tasks (e.g., reference insertion by combining customization with inpainting).
We also find that Edit can surpass the ground-truth training data in both quality and success rate by leveraging knowledge from the image generation/editing and video generation domains. We show two examples for object removal and object change in Figure 7 Change the squirel small mouse
Transfrmm black-and white painting
(n) Original Video
(b) Ground Truth
(c) EditVerse (Ours)
Figure 7: Compare EditVerse generated results with ground truth. Results show Edit Verse can surpass ground-truth truth data quality by extracting knowledge from image and video generation data.

---

The source of emergent ability. We further analyze the source of emergent ability by performing ablations on the training data. We find that removing either image generation/editing data or video generation data negatively impacts video editing quality. Specifically, image generation/cditing data helps the model better understand editing instructions and perform more diverse edits, while video generation data improves temporal consistency and motion modeling. Figure Tablef4illustraal the differences with and without image generation/cditing and video data. Interestingly, EditVerse is able to perform some video editing tasks even without being trained on video editing dataset.
Add small golden with delicate jewels top of the girl's head.
Replace strawberry with small, animated dragon.
(a) Original Video
(b) Image
(c) w/o Video Gen
(d) Full Data
Figure 8: Visualization of ablation on training data. Image data plays a critical role.
Training Datasets
VLM evaluation
Video Quality
Text Alignment
Temporal Consistency
Image
Video Gen
Video Edit
Editing Quality
Frame
Video
Pick Score
CLIP
DINO
V
X
3.62
18.64
22.31
20.44
93.48
90.27
X
/
5.76
19.41
25.22
22.37
98.26
97.83
X
/
6,52
19.81
25.78
22.63
98.24
97.97
6.40
19.72
25.37
22.51
98.77
98.60
/
6.95
19,99
26.26
23.81
98.68
98.44
Table Ablation study on training data. We run 20K steps with the same setup as in Section 5.1
Results indicate that both image and video generation data are crucial to video editing performance.
5.5 ABLATION STUDY ON MODEL DESIGN
Compared with previous approaches (Chen al., 2025c). our model contains two key designs: the interleaved formulation and the special positional embedding. Therefore, as shown in Table we performed ablations by (i) removing the interleaved formulation (placing all images and videos the end of the sequence) and (ii) removing the sequential dimension RoPE. Results show that both the model's performance, especially for the text alignment and
designs have large influence editing quality. This is because the temporal consistency and video quality are partly inherited from the base model, while text alignment and editing quality largely depend the in-context learning ability coming from the model design. Only the interleaved input format combined with sequential positional embedding can best enable the model be aware of the relationships among different modalities (e.g., knowledge transfer of image and video), thereby achieving optimal performance.
Model Design
VLM Evaluation
Video Quality
Text Alignment
Temporal Consistency
Interleave Sequential
Editing Quality
Pick Score
Frame
Video
CLIP
DINO
X
6.42
19.89
25.77
22.74
98.62
98.43
6.84
19.92
26.19
23.51
98.69
98.39
6.95
19,99
26.26
23.81
98.68
98.44
Table 5: Ablation study on interleaved formation and sequential RoPE. We run 20K steps with the same experimental setting detailed in Section 5.1 for the ablation to save compute.
CONCLUSION
This paper introduced Editee unified framework designed address the architectural and data-scarcity challenges in universal video generation and editing. By representing text, images, and videos as single interleaved token sequence, our model leverages full self-attention for ro- bust in-context learning, enabling flexible inputs/outputs of arbitrary resolution and duration, while facilitating knowledge transfer from the data-abundant image domain to the video domain.
We further developed data pipeline for obtaining high-quality video editing samples and proposed EditVerreBench, benchmark covering diverse editing tasks. Results show that Verse achieves state-of-the-as performance. These findings validate that unified architecture data limitations via cross-modal learning, revealing emergent abilities and paving the way for more general multimodal foundation models. Limitations and future work are discussed in the Appendix.
mitigate video
10

---

Acknowledgement. This project was supported in part by the Innovation and Technology Fund (UD-Ia-MHP//13/244) Hong Kong S.A.R.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt. Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Adobe Inc. Adobe stock: Stock photos, royalty-free images, graphics, vectors & videos. https:
distock.adoke com/ 2025.
Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlled generative rendering from single video. arXiv preprint arXiv:2503. 11647, 2025a.
Qingyan Bai, Qiuyu Wang, Hao Ouyang, Yue Yu, Hanlin Wang, Wen Wang, Ka Leong Cheng, Shuailei Ma, Yanhong Zeng, Zichen Liu, et al. Scaling instruction-bast based video editing with high-quality synthetic dataset. arXiv preprint arXiv:2510.15742, 2025b.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix:
Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18392-18402, 2023.
Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fengbin Gao, Peihan Xu, et al. Hidream-il: A high-efficient image generative foundation model with sparse diffusion transformer. arXiv preprint arXiv:2505. 22705, 2025a.
Yuanhao Cai, He Zhang, Xi Chen, Jinbo Xing, Yiwei Hu, Yugian Zhou, Kai Zhang, Zhifei Zhang, Soo Ye Kim, Tianyu Wang, et al. Omnivcus: Feedforward subject-driven video customization with multimodal control conditions. arXiv preprint urXiv:2506.2336/ 2025b.
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Mas- actrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 22560-22570 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models-architeecture, training and dataset. arXiv preprint arXiv:2505.0095 2025a.
Junying Chen, Zhenyang Cai, Pengcheng Chen, Shunian Chen, Ke Ji, Xidong Wang, Yunjin Yang. and Benyou Wang. Sharegpt-4o-image: Aligning multimodal models with gpt-4o-level image generation, 2025b. uttppsss:://xxx wttp:///.gabbbb///eeee 18095
Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, et al. Unireal: Universal image generation and editing via learning real-world dynamics. In Proceedings of Computer Vision and Pottern Recognition Conference, pp. 12501-12511, 2025c.
Jiaxin Cheng, Tianjun Xiao, and Tong He. Consistent video-to-video transfer using synthetic dataset. arXiv preprint arXiv:2311.002 2023.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery. Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts. Denny Zhou, Quoc V. Le, and Jason Wei. Scaling instruction- finetuned language models, 2022. URL htt.thttp:://aet sttttpss//axxiv.orrt

---

Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang. Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505.14663, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. aiviv-247, 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024.
Yuying Ge, Sijie Zhao, Chen Li, Yixiao Ge, and Ying Shan. Seed-data-edit technical report: A hybrid dataset for instructional image editing, arXiv preprint arXiv:2405.04007. 2024.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
Xuanhua He, Quande Liu, Zixuan Yc, Weicai Ye, Qiulin Wang, Xintao Wang. Qifeng Chen, Pengfei Wan, Di Zhang, and Kun Gai. Fulldit2: Efficient in-context conditioning for video diffusion transformers. arXiv preprint arXiv:2506.044 04213, 2025.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prome image editing with cross attention control. arXiv preprint arXiv:2208. 2006:2200..01666, 2022.
Jiahao Hu, Tianxiong Zhong, Xuebo Wang, Boyuan Jiang, Xingye Tian, Fei Yang, Pengfei Wan, and Di Zhang. Vivid- 10m: A dataset and baseline for versatile and interactive video local editing. arXiv preprint arXiv:2411.15200 2024.
Mude Hui, Siwei Yang. Bingchen Zhao, Yichun Shi, Heng Wang. Peng Wang. Yuyin Zhou, and Cihang Xie. Hq-edit: A high-quality dataset for instruction-based image editing. arXiv preprint arXiv:2404.099990 2024.
Itseez. Open source computer vision library. http:///gggthbb..//////eee/e///e
2015.
Tao Jiang, Peng Lu, Li Zhang, Ningsheng Ma, Rui Han, Chengqi Lyu, Yining Li, and Kai Chen. Rtmpose: Real-time multi-person pose estimation based mmpose, 2023. URL https:// arxiv.org abs 2203. 07399
Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. arXiv preprint arXiv:2503.075: 075598 2025.
Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion-based editing with lines of code. arXiv preprint arXiv:2310.01 100.01106, 2023a.
Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd: A native cadleton-guided diffusion model for human image generation. In Proceedings of the IEEE/CVF 2023b.
International Conference Computer Vision, pp.
Xuan Ju, Weicai Ye, Quande Liu, Qiulin Wang, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, and Qiang Xu. Fulldit: Multi-task video generative foundation model with full attention. arXiv preprint ar arXiv:2503. 19907, 2025.
Diederik P Kingma and Max Welling, arXiv:1312.611 14, 2013.
Auto-encoding variational bayes.
arXiv preprint
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems, 36:36652-36665 2023.
12

---

Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXhv:2412.03601, 2024.
Maksim Kuprashevich, Grigorii Alekseenko, Irina Tolstykh. Georgii Fedorov, Bulat Suleimanov, Vladimir Dokholyan, and Aleksandr Gordeev. Nohumansrequired: Autonomous high-quality image editing triplet mining. arXiv preprint arXiv:2507.1411 2025.
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing latent space. arXiv preprint arXiv:2506.1574 15742, 2025.
Xiaowen Li, Haolan Xue, Peiran Ren, and Liefeng Bo. Diffueraser: A diffusion model for video inpainting, arXiv preprint XiX::2501.10011 2025.
Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming-Ming Cheng, and Ying Shan. Pho- tomaker: Customizing realistic human photos stacked id embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8640-8651 2024.
Yaron Lipman, Ricky TQ Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jac Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https:
1lava-v1. qithub....uuu........ ht/bllcg//224-011----1111
Shaoteng Liu, Yuechen Zhang. Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8599-8608, 2024b.
Shaoteng Liu, Tianyu Wang, Jui-Hsien Wang, Qing Liu, Zhifei Zhang, Joon- YounY Lee, Yijun Li, Bei Yu, Zhe Lin, Soo Ye Kim, et al. Generative video propagation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 17712-17722, 2025a.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang. Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et Steplx-edit: A practical framework for general image editing. arXiv preprint arXiv:2504. 251.:25004.1711,,, 2025b.
Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization adam. arXiv preprint arXiv:1711.05100 5(5):5, 2017.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint 1xXiv2208.001033 2021.
Chong Mou, Qichao Sun, Yanze Wu, Pengze Zhang, Xinghui Li, Fulong Ye, Songtao Zhao, and Qian Hc. Instructx: Towards unified visual editing with milm guidance. arXiv preprint 1x.i:25100004885 2025.
OpenAl. Hello got- Blog post, May 2024. hel 1o-ogpt-444/
URL https://openai.rm catieeaai.commmnnin
Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint 0rX0v:2309.00071, 2023.
Pixabay. Pixabay: Free images, videos, music, and more. Accessed: 2025-09-11.
htttss:/ com/ 2025.
13

---

Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih Yao Ma, Ching- Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sethi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Ja- gadeesh, Kunpeng Li, Luxin Zhang, Mannat Singh, Mary Williamson, Matt Le, Matthew Yu, Mitesh Kumar Singh, Peizhao Zhang, Peter Vajda, Quentin Duval, Rohit Girdhar, Roshan Sum- baly. Sai Saketh Rambhatla, Sam Tsai, Samanch Azadi, Samyak Datta, Sanyuan Chen, Sean Bell, Sharadh Ramaswamy, Shelly Sheynin, Siddharth Bhattacharya, Simran Motwani, Tao Xu, Tianhe Li, Tingbo Hou, Wei-Ning Hsu, Xi Yin. Xiaoliang Dai, Yaniv Taigman, Yaqiao Luo, Yen- Cheng Liu, Yi-Chiao Wu, Yue Zhao, Yuval Kirstain. Zecheng He, Zijian He, Albert Pumarola, Ali Thabet, Artsiom Sanakoyeu, Arun Mallya, Baishan Guo, Boris Araya, Breena Kerr, Car- leigh Wood, Ce Liu, Cen Peng. Dimitry Vengertsev, Edgar Schonfeld, Elliot Blanchard, Felix Juefei-Xu, Fraylie Nord, Jeff I Liang, John Hoffman, Jonas Kohler, Kaolin Fire, Karthik Sivaku- mar, Lawrence Chen, Licheng Yu, Luya Gao, Markos Georgopoulos, Rashel Moritz, Sara K. Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2025. URL URL BRLLhtts:///axiiii org/ abs 2410. 13720
Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang. Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15932-15942, 2023.
Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang. Yi Jiang, Yiming Gao, Hu Ye, Daniel K Du, Ze- huan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal understanding and generation. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp.
2545-2555. 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PmLR, 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Va- sudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Fe- ichtenhofer. Sam 2: Segment anything in images and videos, 2024. URL https://axxx org/abs/240BB 00714
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024.
Runway, Introducing runway aleph.
July 25 2025. Accessed: 2025-09-
10.
Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8871- 8879, 2024.
Uriel Singer, Amit Zohar, Yuval Kirstain, Shelly Sheynin, Adam Polyak, Devi Parikh, and Yaniv Taigman. Video editing via factorized diffusion distillation. In European Conference Computer Vision, pp. 450-466. Springer, 2024a.
Uriel Singer, Amit Zohar, Yuval Kirstain, Shelly Sheynin, Adam Polyak, Devi Parikh, and Yaniv Taigman. Video editing via factorized diffusion distillation. In European Conference on Computer Vision, pp. 450-466. Springer, 2024b.
Wenhao Sun, Rong-Cheng Tu, Jingyi Liao, and Dacheng Tao, Diffusion model-basse video editing: A survey. arXiv preprint arXiv:2407.07/ 2024.
14

---

DecartAl Team. Lucy edit: Open- Open-weight text-guided video editing. 2025.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. XrXiv preprint wwr.iv:5503.03144 2025.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Yang Fan, Kai Dang. Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- model's perception of the world at any resolution. arXiv preprint arXiv:2409.1219, 2024.
Wen Wang, Yan Jiang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chun hua Shen. Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303.17599 2023a.
Yi Wang. Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-tex text dataset for multimodal understand- ing and generation. arXiv preprint M.ii:2307.06944 2023b.
Yuhan Wang, Siwei Yang, Bingchen Zhao, Letian Zhang, Qing Liu, Yuyin Zhou, and Cihang Xie. Gpt-image-edit-1.5 m: A million-scale, gpt-generated image dataset. arXiv preprint arXiv:2507.21033, 2025.
Cong Wei, Zheyang Xiong, Weiming Ren, Xeron Du, Ge Zhang, and Wenhu Chen. Omniedit: Building image editing generalist models through specialist supervision. In The Thirteenth Inter- national Conference on Learning Representations, 2024.
Cong Wei, Quande Liu, Zixuan Ye, Qiulin Wang, Xintao Wang, Pengfei Wan, Kun Gai, and Wenhu Chen. Univideo: Unified understanding, generation, and editing for videos. arXiv preprint arXiv:2510.08377, 2025.
Bichen Wu, Ching Yao Chuang, Xiaoyan Wang, Yichen Jia, Kapil Krishnakumar, Tong Xiao, Feng Liang, Licheng Yu, and Peter Vajda. Fairy: Fast parallelized instruction-guii video-to-vide synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- 2024.
Yao
nition, pp.
Jay Zhangjie WU, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7623-7633, 2023a.
Jay Zhangjie Wu, Xiuyu Li, Difei Gao, Zhen Dong, Jinbin Bai, Aishani Singh, Xiaoyu Xiang, Youzeng Li, Zuwei Huang, Yuanxi Sun, al. Cvpr 2023 text guided video editing competition. arXiv preprint arXiv:2310. 16003, 2023b.
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. In Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 13294-13304 2025.
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv:2406.0944 2024a.
Ling Yang, Bohan Zeng, Jiaming Liu, Hong Li, Minghao Xu, Wentao Zhang, and Shuicheng Yan. Editworld: Simulating world dynamies for fnstlwctionnfolloiinng image editing. arXiv preprint arXiv:2405. 14785, 2024b.
Xiangpeng Yang, Ji Xie, Yiyuan Yang, Yan Huang, Min Xu, and Qiang Wu. Unified video editing with temporal reasoner. arXiv preprint 2rXiv:2512.07469, 2025.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint 24Xv:2440.00077 2024c.
15

---

Danah Yatim, Rafail Fridman, Omer Tal,Tal, Yoni Kasten, and Tali Dekel. Space-time diffusion features for zero-shot text-driven motion transfer. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 8466-8476 2024.
Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. Imgedit: A unified image editing dataset and benchmark. arXiv preprint arXiv:2505.20277 2025a.
Zixuan Ye, Xuanhua He, Quande Liu, Qiulin Wang, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, Qifeng Chen, and Wenhan Luo. Unic: Unified in-context video editing. arXiv preprint NXiv:25066042166 2025b.
Jaehong Yoon, Shoubin Yu, and Mohit Bansal. Raccoon: A versatile instructional video editing framework with aeto-generated narratives. arXiv preprint arXiv:2405.18406, 2024.
Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. Anyedit: Mastering unified high-quality image editing for any idea. arXiv preprint arXiv:241..15736 2024.
Fan Zhang, Shulin Tian, Ziqi Huang, Yu Qiao, and Ziwei Liu. Evaluation agent: Efficient and promptable evaluation framework for visual generative models. arXiv preprint arXiv:2412.09645, 2024.
Kai Zhang, Lingho Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guider image editing. Advances in Neural Information Processing Systems, 36:31428-31444 2023a.
Kai Zhang, Peng Wang, Sai Bi, Jianming Zhang, and Yuanjun Xiong. Knapformer: An online load balancer for efficient diffusion transformers training. arXiv preprint wrXiv:2508.06001 2025a.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 1886-38477 2023b.
Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang. In-context edit: Enabling instructional image editing with in-context generation large scale diffusion transformer. arXiv preprint arXiv:2504.20690, 2025b.
Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-based fine-grained image editing scale. Advances in Neural Information Processing Systems, 37:3058-3093, 2024.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint 2tti::2408.1103333 2024.
Bojia Zi, Penghui Ruan, Marco Chen, Xianbiao Qi, Shaozhe Hao, Shihao Zhao, Youze Huang, Bin Liang, Rong Xiao, and Kam-Fai Wong. Se'," norita-2m: A high-quality isstruction-based dataset for general video editing by video specialists. arXiv preprint 44X1v:2502.06773,, 2025.
16

---

A APPENDIX
A.1 IMAGE AND VIDEO COPYRIGHTS
T are from pixabay (Pixabay 2025), stockbusters stock.adobe.ccooo (the first video on the top), andreybiling stock.adobe.com (the second video the top), and Mara Zemgaliete sstock.adobe com (the third video on the top). Comparison images in Figure [ are from ImgEdit- Bench Ye et al. 2025a).. Example videos in FigureßJare from pixabay (Pixabay 2025) and black- boxguild www.k.toooooooo adobe.coo (the first video in "More Examples"). Example videos in Figure 4 and 8 are from pixabay (Pixabay 2025). licensed from the website.
Adobe Stock (Adobe Inc. 2025) videos are officially
A.2 EVALUATION DETAILS
Automatic Evaluation. To provide a comprehensive and robust evaluation of instruction- video editing models EditVerseBench, employ suite of six metrics spanning four aspects: overall editing quality evaluated by a Vision-Language Model (VLM), video quality, text alignment, and temporal consistency.
Overall Editing Quality Evaluated by VLM: To evaluate the overall editing performance, we employ a state-of-the-art Vision-Language Model (VLM), GPT-4o OpenAl (2024), to serve as an automated judge. This provides scalable, human-like assessment that cap- tures nuances of editing quality, such as semantic correctness and artifact presence, which are often missed by other metrics. Our method uniformly samples three frames from each source and edited video pair. For each sample, the VLMM receives the source frame, the edited frame, and the text instruction. It is prompted to score the edit from 0 (worst) (best) across three key criteria: Prompt Following, Edit Quality, and Background Consis- tency, and then sum them together to get the overall score for this frame. The final VLM score for the entire video is the average of these three frame scores.
Video Quality: We employ PickScore (Kirstain et al. 2023). which shows strong cor- relation with human judgment of image quality and prompt alignment. We calculate the PickScore for each frame and average these scores across the entire video.
Text Alignment: Text alignment evaluates how well the edited video reflects the given text instruction. We measure this at both the frame level and the video level.
CLIP Text-Image Alignment: This metric assesses the semantic alignment between the editing instruction and each frame of the output video. We encode the text instruction using the CLIP text encoder and each frame using the CLIP vision encoder to get feature vectors, respectively. The final score is the average cosine similarity across all frames.
VICLIP Text- Video Alignment: Frame-wise alignment doesn't capture the temporal aspects of the instruction. Therefore, we use ViCLIP (Wang et al.) 2023b) to compute an embed- ding for the entire video clip and measure its cosine similarity with the text instruction's 's embedding. This measures how well the video as a whole corresponds to the prompt.
Temporal consistency measures the smoothness and coherence of the edited video, penalizing flickering, jarring transitions, and inconsistent object appear- ances between frames. We assess this using feature similarity between adjacent frames.
Temporal Consistency:
Frame-wise CLIP Consistency: We use the ViT-L/14 vision encoder from CLIP (Radford) et al., 202T) to extract features of each frame in the edited video. The consistency score calculated as the average cosine similarity between the features of all adjacent frames.
Frame-wise DINO Consistency: To capture more fine- fine- grained structural and textural con- sistency, we repeat the same procedure using features extracted from a pre-traines DINOv2 model (Caron et al. 2021). DINO's self-supervised training allows it to capture object de- tails that might be overlooked by CLIP. Similarly, the consistency score is calculated as the average cosine similarity between the features of all adjacent frames.
User Study. To validate our automated metrics and directly measure human perceptual preferences, we conducted comprehensive user study. The user study was outsourced to a professional ex- ternal vendor, who recruited 20 annotators coming from diverse non-expert backgrounds. Each comparison pair was independently evaluated by 3 different annotators, and each annotator labeled 150 comparison pairs in total. To faithfully capture end-user preferences, we intentionally kept the instruction minimal and user-centric, providing only the brief prompt shown below, without addi- tional technical guidance. This setup was chosen reflect how typical users would judge the outputs
17

---

rather than imposing task-specific expertise. Although we did not compute formal inter- annotator agreement statistic, the redundancy of three independent judgments per pair helps mitigate noise and increases the robustness of the aggregated preferences. We recruited detail-oriented participants to evaluate the performance of different Al video editing models. Using a web-based interface, participants were shown pairs of edited videos, labeled "Result 1" and "Result 2", each generated by different models from the same source video and text instruction. Their task was to compare the two videos and choose among "Result is better," "Result 2 is better," or "They are about the same" across three evaluation criteria: (1) Text-Instruction Aligmment: Which video better follows the provided instruction? (2) Preservation of Unedited Regions: Are unmodified parts of the video accurately preserved, with minimal distortion or artifacts? Ideally, edits should only affect the in- tended object or region. Select the one that preserves better. (3) Aesthetic Quality: Which video more visually appealing in terms of realism, smoothness, and overall perceptual quality? A video is considered the winner of a comparison if it achieves majority of wins across these three crite- ria. We find the user study shows Pearson Correlation of 0.84 with automatic VLM evaluation, indicating a very strong positive correlation between the user study and VLM rankings.
Instructions of Figure 1 We list the editing instructions that were used in Figure [ in top- to-bottom, left-to-right manner: (1) Add a pair of sparkling feathered wings to the person who is running. (2) Turn the man into running cartoon leopard. (3) Turn the person into translucent, crystal-glass-ikk form. (4) Remove the woman. (5) Transform the woman's 's dress into a golden, fluid-like form with flames. (6) Turn into cartoon form. (7) Change the water to blue. (8) Change the camera pose to Pan Left. (9) Change the woman's 's slip dress to red and add a gentle snowfall effect. (10) Turn the grass into reflective water surface. (11) Dramatically transform the scene by adding animated fiery embers and gentle flame wisps subtly dancing along the edges of the rose petals, giving the impression that the flower is being ignited by magical fire without harm, creating surreal and striking contrast of beauty and intensity. (12) Insert paper boat in the water [source image] A graceful white swan glides silently across the still surface of a clear lake, its long neck curved in gentle arch and its feathers shining with a soft pearly sheen in the sunlight. Beside it, an orange paper boat drifts lightly, its sharp folds and pointed bow creating small ripples as floats. (13) Two vibrant blue parrots
perched closely together to be pecking or searching for food in the crevice of the wood. The background shows a sunlit, green outdoor area with other birds visible in the distance, giving the scene lively and natural atmosphere. (14) Change the weather to a heavy snowfall. (15) Detect the mask of the bird. (16) A young beautiful woman wearing a white hijab and long white top sits quietly on the floor. She is reading from an open book, which rests on an intricately carved wooden stand. Her expression calm and focused as she moves her finger along the lines of text, absorbed in her reading. The peaceful setting, with soft light and a tiled background, suggests a moment of reflection or prayer. (17) A quiet tree-lined path stretches into the distance, bathed in soft sunlight. Green leaves form a canopy overhead, while brown and yellow leaves scattered across the ground. The scene feels calm and peaceful, inviting a slow walk or a moment of reflection in nature.
tree stump. They appear
A.3 ADDITIONAL EXPERIMENTS
Image Editing. We present comprehensive evaluation of Edit for the task of image edit- ing using the ImgEdit- Lnngdii-Bench benchmark, as summarized in Table 6 The results demonstrate that Verte achieves highly competitive performance, surpassing wide range of existing ap- proaches (Deng et al. 2025 Liu et al. 20256). This highlights the effectiveness of our method.
Method
Add
Adjust
Extract
Replace
Remove
Background
Style
Hyhrid
Action
Overall?
MagicBrush
2.84
1,97
1,58
1.75
2.38
1.62
1.22
1.83
Instruct-P2P
2.45
1.83
1.44
2.01
1.50
1.44
3.55
1.20
1.46
1.88
AdyEdit
3.18
2.95
1.88
2.47
2.23
2.24
2.85
1.56
2.65
2.45
UltraEdit
3,44
2.81
2.13
2.96
2.83
3.76
1.91
2.98
2.70
ICEdit
3,58
3.39
1.73
3.15
2.93
3.08
3.84
2.04
3.68
3.05
StepIX-Edit
3.88
3.14
1.66
3.40
2.41
3.16
4.63
2.64
2.52
3,06
UniWorld-Vi
3.82
3.64
2.27
3.47
3.24
2.99
4.21
2.96
2.74
3,26
BAGEL
3,81
3.59
1.58
3.85
3.16
3.39
4,51
2.67
4.25
3.42
Edit Verse (Ours)
3.81
4.71
3.80
3,42
OmniGen2
3,57
3.06
1.77
3.74
3.20
3.57
4.81
2.52
4.68
3,44
Kontext-dev
3,83
3.65
2.27
4.45
3.17
3.98
4.55
3.35
4.29
3.71
Ovis-U1
3,99
2.66
4.38
4.15
4.86
3.43
4.68
3.97
-ma-ee-Image
4.61
4.33
2,90
4.35
3.66
4.57
4.93
3.96
4.89
4,20
Table 6: Quantitative comparison on ImgEdit-Bench (Ye et al. 2025a).
18

---

Video Generation. We evaluate the video generation capability of Vdit the VBench bench- mark ZZang et al., 2024), shown Table 7 As shown, Vdit achieves highly competitive performance compared with wide range of both open-source and commercial models. Notably, even though Edit Verse is trained on diverse tasks beyond video generation and is built with a rela- tively small model size, it can still match surpass the performance of several larger-scale systems.
Models
# Params.
Total
Quality Score
Semantic Score
ModelScope
1.7B
75.75
78.05
66.54
La
3B
77.08
78.78
70.31
OpenSoraPlan V1.3
77.23
80.14
65.62
Show-1
6B
78.93
80,42
72.98
AnimateDiff-V.
80.27
82.90
69.75
Gen-2
80.58
82.47
73.03
Pika-1.0
80.69
82.92
71.77
VideoCrafter-2.0
80.44
82.20
73.42
EditVerse (Ours)
2B
80.97
83.47
70.97
CogVis VidcoX
5B
81.61
82.75
77.04
Kling
81.85
83.39
75.68
Step-Vidco- Video-T2V
30B
81.83
84.46
71.28
Gen-3
82.32
84.11
75.17
Table Comparison with text-to-video models on the VBench (Zhang et al. (2024). Params. is the number of total parameters. EditVerse shows competitive performance with a small model size.
Image Generation. We evaluate the image generation capability of Edit Verse using the GenEval benchmark (Ghosh et al. al.] (203) shown in Table which is designed comprehensively assess text- to-image models across multiple aspects of visual reasoning and compositional fidelity. Our method achieves
performance when compared against a wide range of both source and commercial systems, highlighting better semantically aligned generation.
Method
Single Obj-
Two Obj.
Counting
Colors
Position
Color Attri.
Overall
LlamaGen
0.71
0.34
0.21
0.58
0.07
0.04
0.32
LDM
0.92
0.29
0.23
0.70
0.02
0.05
0.37
SDy1.5
0.97
0.38
0.35
0.76
0.04
0.06
0.43
PixArt-Alpha
0.98
0.50
0.44
0.80
0.08
0.07
SDv2.1
0.98
0.51
0.44
0.85
0.07
0.17
0,50
DALL-E
0,94
0.66
0.49
0.77
0.10
0.19
0.52
Emu3-Gen
0.98
0.71
0.34
0.81
0.17
0.21
0.54
SDXL
0.98
0.74
0.39
0.85
0.15
0.23
0.55
DALL-E33
0.96
0.87
0.47
0.83
0.43
0.45
0.67
Infinity
0.85
0.49
0.57
0.73
SD3-Medium
0.94
0.72
0.89
0.33
0.60
0.74
FL.XX.1-dev
0.98
0.93
0.75
0.93
0.68
0.65
0.82
Edit Verse (Ours)
0.99
0.95
0.81
0.82
0.68
0.64
0.82
1 use LLM-rewritten prompts.
Table Comparison with text-to-image models on the GenEval Zhang et al. 2024
Video Editing, We provide quantitative comparison V2VBench (Sun al. 2024 in Table Noted that all V2VBench videos are square, whereas our training data does not include any square video editing samples. Our method achieves the best competitive results across metrics.
A.4 DETAILED TRAINING DATA
Table 10 provides detailed statisties overview of the whole training datasets that are used in our work, along with their respective ratio in the training process. The table is organized by task type, image editing, image generation, video editing, and video generation. For each dataset, we report the total number of samples, the ratio applied when constructing the training mixture, and brief description highlighting the data quality, coverage, and characteristics. The training data comprises mixture of high-quality open-source data, curated internal datasets, and filtered synthetic datasets. This combination allows us to balance scale, quality, and diversity, ultimately supporting unified training across both editing and generation tasks for images and videos.
19

---

Frames Quality
Semantic Consistency
Object Consistency
Frames Text Frames Pick Score
Video Text
Motion
Method
Alignment
Alignment
Alignment f
Network and Training Paradigm
Tune-A-Video
5.001
0.934
0.917
27.513
20.701
0.254
-5.599
SimDA
4.988
0,940
0,929
26.773
20.512
0,248
-4.756
VidToMe
4.988
0.949
0.945
26.813
20.546
0.240
-3.203
VideoComposer
4.429
0,914
0,905
28.001
20.272
0.262
-8.095
MotionDirector
4,984
0,940
0,951
27.845
20.923
0,262
-3,088
EditVerse (Ours)
4.957
0.959
0.960
28.587
21.117
0.273
-3.015
Attention Feature Injection
Video-P2P
4,907
0.943
0.926
23.550
19.751
0.193
5.974
Vid2Vid-Zero
5.103
0.919
0.912
28.789
20.950
0.270
-4.175
Fate-Zero
5.036
0,951
0,952
25.065
20.707
-1.499
TokenFlow
5.068
0.947
0.943
27.522
20.757
0.254
-1,572
FLATTEN
4.965
0.943
0.949
27.156
20.745
0,251
-1,466
FRESCO
5.127
0.908
0.896
25.639
20.239
0,223
-5.241
Diffusion Latent Manipulation
Tex
5.097
0.899
0.894
29.124
20.568
0.265
11.226
Pix2Video 5.075
0,946
0.944
28.731
21.054
-2.89
ControlVideo
5,404
0,959
0,948
28.551
20.961
-9.396
Rerender
5.002
0.872
0.863
27.379
20.460
0.261
-4,959
RAVE
5.077
0.926
0.936
28.190
20.865
0,255
-2.398
Table 9; Quantitative comparison on V2VBench (Sun et al. 2024). Methods are grouped into three categories: (i) Network and Training Paradigm, (ii) Attention Feature Injection, and (iii Dif- fusion Latent Manipulation, Local best are bold. Global best are underlined.
A.S LIMITATION AND FUTURE WORK
While EditVerse presents significant step toward unified video and image generation and editing. we acknowledge several limitations that open avenues for future research.
Observed Failure Cases. Despite its strong overall performance, Verte is not immune to failure cases including artifacts, flickering, low motion, logical flaws, wrong editing position, and blurred editing region. Figure shows examples of commonly seen failure types of Editeerse.
Insert old-fashioned treasure chest half-buried among the rocks at the man's feet, with a faint golden glow emanating from slightly open lid.
Remove the lamp in the background entirely from the video, ...to create a natural, uncluttered look.
Original Video
Original Video
Edited Video
Edited Video
(a) Wrong Position
(b) Blurred Editing Area
Figure 9: Failure case examples Vditee (a) The model fails add object (treasure chest) at the correct position (at the man's 's feet). (b) Generation of blurry artifacts within the edited region.
Computational Cost. Our reliance on full self-attention mechanism across a unified one- dimensional token sequence, while powerful for in-context learning, leads significant compu- tational overhead. The concatenation operation results in long sequence lengths, particularly for high-resolution or long-duration videos, which translates to high FLOPs and prolonged training and inference time. Future work could explore more efficient attention mechanisms reduce the com- putational burden without compromising the model's 's cross-modal learning capabilities. Our work was not specifically designed to improve efficiency, and the reported memory usage and inference time reflect the model's 'ss raw, unoptimized performance. There are several practical ways to improve it, which we plan explore in future work: (1) Using higher-compression VAE. In model,
20

---

Dataset
#Samples
#Ratio
Information
Image Editing
Manually annotated with real image.
MagicBrush
8,802
High-quality, oditing calegories.
Generated by GPT 4o. 14 editing categories.
ShareGPPT-oo-Tmage
46,489
10
Most are high- qughituulity, but some cases contain noise.
Object Removal & A
118,972
4
Manually captured photos with object-preseen
and object- absent scenes. High-quaity.
OmniFEdit
185,500
2
Good-quality but contains large noise some editing calegories.
Generated by lask specific models. 7 editing categories.
Generated by segmentation and inpainting. 13 editing categories. Fair quality. Need filtering.
Imggddi"
245,986
Generated with designed pipeline using internal
NIR--Edit
358,463
5
inage editing model. High-quulity. oditing calegories.
Geucrated by a specially designed editing model. Fair quality, 9 editing categories.
UltraEdit
500,000
1
Generated by task-specilic pipelines. 25 editing categories. Fair quality. Need fillering.
AnyEdit
1,244,033
Re-process OmniEdit, UltraEdit, and HQ-Edit with GPT-lo. Most are high-quality, but
GPT-Image-Ediee E.Mtte-E
1,500,000
1
contain noise. An internal instruction-based image editing dataset.
Instruction-hased Editing
1,824,90 969
Sum
6,033,214
Image Generation
BLIP30-60k
60,000
Text-to-maage instruction tuning datasct distilled from GPT-4o.
LLaer-rrtraain
500,000
Text-to-Image data re-captioned using Qwen2-VL. (from text--o-mmag--MM)
Text-to-Image!
609,950
Internal high-quality text-to-image dataset.
LLx fine-tuning
700,000
1
Tex--t-Image data generated by Flux-dev (from lex-tto-image-MMM MM).
Sum
1,869,950
Video Editing
Camera Change
8,000
20
Camera change pair generated with RcCamMaster
Trunsfer Trunsfer
10,327
10
Style trunsfer data puir generated with Step X. Edit and VACE.
Editing region detection with prompt
Mask Detection
15,741
5
"I want to [edit prompt). Detect the region that needs to he edited"
Contain object removal. object addition, and object replacement.
Object replacement data pair generated with VACE. Contain mask version and w/o mask version in training.
Object Chnage
31,482
10
CG Removal Additionn
38,900
2
Rendered vidcos with object-present and object-absent scenes.
Generated with lusk specific models. 5 editing calegories.
Señorita-2M
55,711
2
Low quality. Need fllteiing
Containing oditing propagation for object removal, object addition. object replacement, and style transfer.
Propagation
$9,826
10
Object removal and addition puirs generated with DilluEraser. Contain w/ mask version and wio mask version in training.
Object Removal & Addition
67,516
10
Sum
287,503
Video Generation
Depth-to-
182,097
2
Depih-to Depih-to-vider video dataset, Depih is detected with Depth Anything v2.
Video-to-Depth
182,097
Video-to-depth dataset. Dopth detected with Depth Anything v2.
Sketch-to-Vidde Vidden
207, 797,
2
Sketch-to-videe dataset, Sketch is detected with OpenC'V Canny.
Video-to-Sketch
207, 747,749
2
Viden-to-sketch dataset, Sketch detected with OpenCV Canny.
First Frame-to-Vider
217,038
5
First frame-to-video dataset.
Pose-to-Video
233,068
2
Pose to videa dalaset. Pose is detected with RTM-
Video-to-Powe
233,068
Video-to-pose datasct. Pose detected with RTM-Pose.
Text-to-Video
223,494
10
Internal high-quality text-to-vidon dataset.
Customization
140
1
High-quality identit-to-video dataset from OmniVCus (Cai et al. 20256
Video inpainting data pair generated with Grounded SAM
Video Inpainting
1,495,020
2
Contain w/ mask version and wlo musk version in training.
Sam
3,921,491
Internal datiaeets,
We filter these datasoss to imprroe their quality.
Table 10: Detailed Statistics of the training datasets. We combine high-quality open-source datasets, internal datasets, and Verte datasets for unified training. This table presents the dataset name, sample counts, training ratios, and key details for each dataset.
21

---

VAE with relatively low compression (8x spatial downsampling), which leads to large number of visual tokens. Recent VAEs can achieve 16x spatial compression. If we replace our VAE be reduced to about one quarter, which directly lowers the attention cost. (2) Dynamic token selection. We can introduce a dynamic token selection mechanism that adaptively keeps only the most important context tokens and prunes redundant ones. This can reduce the effective sequence length for full attention, as explored in recent work FullDiT2 (He et al. 2025). (3) Distillation for faster inference. We can further apply model distillation and step distillation to reduce both the number of diffusion steps and the model size, which can noticeably speed up generation. (4) Future work could explore more efficient attention mechanisms (e.g., linear attention, Mamba attention) to reduce the computational burden without compromising the model's cross-modal learning capabilities.
with 16x spatial compression rate models, the token length
1000
20 steps
40
Inference Time (saconds)
50 steps
GPU Memory (GB)
750
500
20
250
20000
40000
60000
80000
100000
20000
40000
80000
80000
100000
Token Length
Token Length
Figure 10: GPU memory usage and inference latency as functions of token length.
Token Length
17652
34342
63662
110822
Inference Time (second/20 steps)
21
48
129
349
Inference Time (second/50 steps)
52
118
323
873
GPU Memory (MB) TFLOPs/second
29511
32768
36663
44824
67.9
58.2
39.4
25.4
Table 11: Inference efficiency across different token lengths
To assess the efficiency of our model with extended token length (as shown in Figure 10 and Ta- ble [T]. we analyze the impact of token length on both efficiency and GPU memory usage. Ex- periment results demonstrate that resource consumption growth remains within manageable and predictable range. Specifically, GPU Memory usage exhibits a strong linear relationship with token length. As the sequence length scaled from 17,652 up to 110,822 tokens, the peak memory footprint increased modestly from 29.5 GB to 44.8 GB. This predictable and relatively slow scaling rate confirms that the extended context itself does not impose an unconstrained memory ceiling. Similarly, inference time shows a systematic, controllable increase with token length.
Image Editing Performance. While our unified model demonstrates strong generalization and performs on par with many image editing models, it does not currently achieve state-of-the-art per- formance in the image domain. Targeted optimizations, such as employing a more sophisticated data-mixing strategy or fine-tuning the model on high-quality, image-only editing datasets, could be explored to boost its performance and close the gap with specialized, ff-t--oftthe-ar image editors.
Dataset Quality. Although our data curation pipeline is crucial for enabling instruction- video editing, the resulting dataset contains inherent noise. The editing instructions are often concise (averaging around 10 words) and may lack the detail required for highly complex or nuanced edits. Furthermore, the automated methods used to generate editing pairs have an estimated success rate of around 65% inevitably introducing imperfect or failed edits into the training corpus. Future efforts could focus on developing more advanced data generation and filtering techniques as mentioned in concurrent works (Wei et al. 2025 Mou et al. 2025 Bai et al. 2025b Yang et al. (025).
Generalist vs. Specialist Models. Our work highlights the potential of unified models, but it is plausible that for specific, well-defined tasks with abundant high-quality data (e.g.. inpainting), dedicated specialist model might still yield superior results.
22


Reviews:
[{"review_id": "NSkeiBjsa5", "reviewer": "Reviewer_U1BY", "summary": "This paper introduces EditVerse, a unified multimodal framework for text-guided image and video generation and editing. The model represents text, image, and video modalities as a single interleaved token sequence and applies self-attention for in-context learning and cross-modal reasoning. To address the lack of video editing data, the authors design a large-scale automated data generation pipeline. Extensive experiments demonstrate that EditVerse achieves strong performance and exhibits emergent editing capabilities that generalize beyond the training set.", "questions": "* How is the VLM used for data scoring and filtering? Was it used off-the-shelf or fine-tuned for editing relevance? What is its reliability or correlation with human judgment?\n* How does model performance and runtime scale with sequence length (e.g., multi-video input or long textual instructions)? Can efficiency be improved beyond full-sequence attention?\n* Have you compared the single-layer modality projectors against deeper or non-linear alternatives?\n* The reported improvements are modest; how sensitive are the evaluation metrics, and do the differences translate into perceptual gains?\n* Given that the data pipeline heavily depends on pretrained vision models, how do you ensure the resulting dataset and EditVerse itself do not inherit or amplify their biases?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Comprehensive experiments with solid quantitative and qualitative evaluations.\n* Clear presentation and well-structured writing.\n* The unified token representation for text, image, and video is conceptually elegant.\n* Interesting empirical insight: the model demonstrates emergent abilities on unseen editing tasks, suggesting strong cross-modal generalization.", "weaknesses": "* While the unified sequence formulation is elegant, it primarily extends existing transformer-based multimodal modeling paradigms. The contribution lies more in engineering integration and data scaling than in introducing fundamentally new learning principles.\n* The model requires ~30 GB GPU memory and ~118 seconds to edit a single 360p video on an A100 80 GB GPU. This raises serious scalability concerns for long-duration or high-resolution videos, as well as practical deployment constraints.\n* The approach concatenates all modality tokens into one long sequence, resulting in $O(L^2)$ attention cost. It is unclear how EditVerse maintains efficiency with long inputs, multiple video clips, or multi-turn text–video interleaving. No efficiency analysis (FLOPs, latency, or memory growth curves) is provided.\n* The use of single-layer linear projectors for modality alignment may be oversimplified. There is no ablation on alternative projector depths or modality-specific encoders to validate whether this minimal design is sufficient for cross-modal alignment.\n* The video editing dataset is entirely generated via an automated pipeline using pretrained models (Grounded-SAM-2, VACE, ReCamMaster, etc.). This synthetic data may not reflect the diversity and imperfections of real-world edits, leading to overfitting on artificial editing patterns.\n* Each data generation stage depends on prior model outputs, so artifacts such as inaccurate masks, poor inpainting, or unrealistic motion can cascade. The paper provides no quantitative analysis or quality control to assess data noise accumulation. \n* The paper mentions using a VLM to assign quality scores for filtering generated data, but it is unclear whether the VLM was adapted for the data filtering, how accurate its scores are, or how thresholds were chosen.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uccGndgTRX", "reviewer": "Reviewer_ijpS", "summary": "This paper proposes EditVerse, a unified framework for image and video generation and editing. The core idea is to represent text, image, and video as a single interleaved token sequence, processed by a full self-attention transformer. A 4D ROPE position embedding is proposed to encode spatial, temporal, and sequence positions. To address video editing data scarcity, the authors build a synthetic video editing pipeline to collect 0.2 million video editing data for model training. Experiments shows that their model can suprpass existing open-source and commercial models across various tasks.", "questions": "Refer to weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written, and the proposed framework is clearly presented and easy to understand.\n\n2. The visualized qualitative results are impressive and effectively demonstrate the model’s capabilities.", "weaknesses": "1. The main weakness of this work lies in the lack of substantial technical novelty. The proposed pipeline appears highly similar to recent multimodal generative frameworks (e.g., Bagel, OmniGen). The novelty mainly resides in engineering integration rather than in proposing a fundamentally new modeling formulation.\n\n2. In my opinion, the key contribution of the paper is to show that a unified video editing/generation framework can be implemented. However, the authors do not commit to releasing the engineering details, dataset collection pipeline, or the collected dataset itself. Without open sourcing or releasing enough details for reproduction, their engineering contribution becomes less impactful.\n\n3. The justification for the necessity of the proposed interleaved representation remains insufficient. Although the authors include some ablations, it is still unclear why interleaving tokens is fundamentally better than other existing designs (e.g., cross-attention conditioning, multi-encoder structures).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7iXwVbCPzE", "reviewer": "Reviewer_hzXL", "summary": "The paper presents EditVerse, a unified video editing model that aims to handle diverse edit types (style change, object manipulation, temporal edits, localized edits, etc.) with a single model. The core modeling idea is to serialize visual content (both images and multi-frame videos) into an interleaved token sequence, and to use a multi-dimensional rotary positional embedding (RoPE) capturing spatial, temporal, and sequential position.\n\nOn the evaluation side, the paper introduces EditVerseBench, which focuses on real-world, non-square aspect ratios (horizontal and vertical videos) across multiple editing categories, arguing that prior benchmarks mostly assume square crops and narrow edit types. They also describe a data generation and VLM-based filtering pipeline to curate higher-quality training dataset. Quantitative comparisons are reported using automatic VLM metrics, frame-level visual quality and temporal consistency metrics, plus a human pairwise preference study.\n\nOverall, the paper addresses a practically relevant problem and proposes a promising integration of a unified model design, a new benchmark, and a data curation pipeline. Several points require clarification, in particular the transparency of the evaluation setup, the reporting of the human study, and ablations on key components, but these issues appear to be fixable.", "questions": "1. **Human evaluation protocol**\n\n   More details are needed on the human preference study: how many annotators were involved, their backgrounds, how annotation was distributed per person, and whether any quality control or inter-annotator agreement was assessed. Such information would increase the credibility of the human study results.\n\n2. **Clarification on the Qwen2-VL–based data filtering process**\n   The description of the VLM filtering process remains vague. Could you specify the exact score threshold used to retain samples, provide an overview of score distributions before and after filtering (ideally across data sources), and comment on whether Qwen2-VL frequently misjudged quality (e.g., bad edits retained or good edits discarded), even approximately? Greater transparency here would strengthen the argument for the data pipeline.\n\n3. **Human rating in the main results table**\n   For the core quantitative comparison (e.g., Table 2), it would be valuable to include a human rating or user preference column alongside the automatic metrics, similar to prior works such as TokenFlow, Tune-A-Video, and FateZero. This would provide a more direct comparison from a human perspective and make the main results more informative, rather than relying solely on automatic metrics.\n\n4. **Benchmark construction and possible bias**\n\n   Since EditVerseBench excludes square videos entirely, it would be helpful to clarify the rationale for not including them alongside horizontal and vertical formats. This concern is reinforced by the fact that in Table 6, EditVerse does not always outperform training-free baselines on V2VBench, where some simpler methods achieve competitive or better results. It would be useful to discuss in which scenarios such lightweight baselines remain preferable in practice, and how the proposed benchmark design avoids unintentionally favoring EditVerse.\n\n5. **RoPE ablations**\n   As the multi-dimensional RoPE is positioned as a key architectural contribution, more direct evidence isolating its effect would be useful. Even a small-scale study comparing variants that remove temporal RoPE, remove spatial RoPE, or use only standard sequential RoPE would help substantiate the importance of each component.\n\n6. **Data filtering transparency**\n    The data pipeline uses a VLM to assign 0–10 quality scores (instruction adherence, temporal consistency, artifact severity, etc.) and then thresholds these scores to filter large editing datasets.\n\n   - Please show the score distribution before vs. after filtering.\n   - How was the threshold chosen, and how sensitive is final model performance to that threshold?\n   - Do you have an estimate of “false positives” (bad edits that passed) and “false negatives” (good edits that were filtered out)?\n      Since the dataset is part of the claimed contribution, a brief audit would strengthen the narrative.\n\n7. **Reproducibility of automatic evaluation**\n   To facilitate reproducibility, would you consider releasing the exact prompt templates and inference configurations used for the VLM-based automatic evaluation? Without these details, the community may find it difficult to replicate the automatic scores.\n\n8. **Minor fixes for clarity**\n\n   - The training loss formula should be corrected to:\n     $$\n     \\mathcal{L} = \\mathbb{E}_{t,X_0,X_1} \\left\\| u_\\Theta(X_t, t) - (X_1 - X_0) \\right\\|^2.\n     $$\n\n   - The naming and ordering of the RoPE components in the text (around lines 200–221) do not match Figure 2. The text first describes “Height and Width Dimensions” before sequential and temporal, while Figure 2 uses the order spatial → temporal → sequential. Please unify terminology and ordering to avoid confusion.\n\n   - In Table 9, the symbol used to mark the method is “†”, but the note below the table refers to “‡ uses LLM-rewritten prompts.” The marker in the table and the footnote symbol should be consistent.\n\n   - The citation for “TokenFlow” is incorrect. The paper currently cites *“Tokenflow: Unified Image Tokenizer for Multimodal Understanding and Generation”*, which is unrelated to video editing. It should cite **“TokenFlow: Consistent Diffusion Features for Consistent Video Editing”**. The incorrect reference appears in line 348–349, Table 3, and line 818.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper addresses a practically relevant problem by aiming to unify diverse instruction-based video editing tasks within a single model. The proposed interleaved visual token representation together with the multi-dimensional RoPE design is technically interesting and potentially reusable beyond this specific application. The introduction of a benchmark with realistic horizontal and vertical video formats and multiple editing categories is valuable for evaluating real-world usage, and the experimental comparisons include both strong instruction-based methods and training-free baselines, which provides a relatively comprehensive view of performance.", "weaknesses": "The human evaluation lacks essential reporting details such as annotator composition, annotation protocol, and reliability, making it difficult to assess the credibility of the results. The design choices of the proposed benchmark may be perceived as favoring the method, and the paper does not sufficiently analyze scenarios in which simpler, training-free baselines still outperform the model. In addition, the contribution of the positional encoding components is not well-isolated through ablations, and the automatic evaluation pipeline, including data filtering and VLM-based scoring prompts, is not fully transparent, which limits the reproducibility of the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rvcL0IKCyS", "reviewer": "Reviewer_3VqN", "summary": "This paper introduces EditVerse, a novel unified framework designed to perform both image and video generation and editing within a single model. The authors identify two primary bottlenecks in video editing: (1) architectural limitations of existing models, which are often task-specific, and (2) the scarcity of high-quality, instruction-based video editing data.", "questions": "1. Could the authors provide more details on the scaling properties of EditVerse? How do inference time and VRAM usage scale with (a) video duration (e.g., 3s vs. 10s vs. 30s) and (b) resolution (360p vs. 720p)? Is the full self-attention approach feasible beyond the short, low-resolution clips shown?\n2. The dimension allocations for the 4D RoPE (56H, 56W, 12Seq, 4Temp) are specific. Could the authors provide a brief rationale or ablation for this design choice? For example, why is the sequential dimension (12) given more embedding space than the temporal dimension (4)?\n3. How does the model perform on editing tasks or concepts that are *not* present in its synthetic \"teacher\" models? For instance, if VACE is poor at \"making an object transparent,\" can EditVerse still learn this concept purely from the image-editing data and apply it successfully to video, or is its video performance on this task limited by VACE?\n4. The \"wrong position\" failure case in Fig 9a suggests limitations in complex spatial reasoning. The VLM evaluation (Table 2) is high, but this metric averages 3 frames. How well does this VLM metric capture these kinds of high-level logical or instruction-misalignment failures, as opposed to per-frame artifacts or quality?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The core design of an interleaved 1D token sequence for mixed image/video data, combined with a full self-attention mechanism and the novel 4D RoPE, is a powerful and original solution for multimodal in-context learning.\n2. The paper's standout strength is its clear demonstration that a unified model can learn video editing from data-abundant *image* editing datasets. The ablation in Table 4 (showing the model *can* edit video with zero video-edit data, albeit poorly) and the major quality drop from removing image-edit data (Fig 8) provide powerful evidence for this hypothesis.\n3. The authors address the ecosystem problem by not only building a model but also creating the data (232K video-edit samples) and the evaluation tools (EditVerseBench) necessary to make progress, and they are releasing the benchmark.", "weaknesses": "1. The most significant weakness, which the authors acknowledge in the appendix, is the computational cost. Using full self-attention on a 1D-tokenized video (where the sequence length $L$ includes all frames) leads to $O(L^2)$ complexity. The reported 118 seconds for a single 360p video on an A100 is very slow and will likely scale quadratically or worse with resolution and duration, making it impractical for real-world use on high-resolution or long-form videos.\n2. The video editing data is synthetically generated by a pipeline of specialist \"teacher\" models (VACE, DiffuEraser, etc.). While this is a clever solution, it means the model's performance may be capped by the quality and biases of these teachers. It's unclear if the model can perform edits that its teacher models are incapable of.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BYBQn8z56U", "reviewer": "Reviewer_72vX", "summary": "This paper presents EditVerse, a unified framework for image and video generation and editing. By representing images, videos, and text as one-dimensional sequences of interleaved tokens, EditVerse demonstrates promising capabilities in in-context learning and cross-modal knowledge transfer. EditVerse achieves state-of-the-art results on both the EditVerseBench and TGVE+ benchmarks. Furthermore, it exhibits impressive emergent abilities when trained at scale with unified multi-modal data. Although interactive editing and generation scenarios were not evaluated in this work, I believe this approach, with appropriate scaling, could demonstrate significant potential and enable substantially broader applications. In summary, this is a technically solid and well-presented paper.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. **Unified Design**.\nThe paper presents a clean solution : treat text, images, and videos as interleaved token sequences with a 4D positional embedding (sequential, temporal, height, width). This simple yet effective design handles arbitrary resolutions and durations naturally, which is quite elegant.\n2. **Knowledge Transfer between Different Modality**.\nThe key insight is using abundant image editing data to help with scarce video editing data. Through full self-attention and mixed training (6M image editing + 288K video editing samples), the model learns to transfer knowledge across modalities. \n3. **Solid Experimental Results**.\nThe paper presents comprehensive experiments across image/video generation and editing tasks, demonstrating that EditVerse achieves state-of-the-art performance with only 2B parameters. The ablations also clearly show both image and video data matter—images help with instruction understanding, videos help with temporal consistency. What's interesting is the emergent behavior: the model can handle tasks it wasn't explicitly trained on, and sometimes even beats the ground truth by combining knowledge from different domains.\n4. **Strong Results  with Potential**.\nWith just 2B parameters, EditVerse achieves SOTA on EditVerseBench and TGVE+, matching or beating much larger models and commercial systems. The results hold up in both automatic metrics and user studies across 20 different editing tasks. I believe this approach has significant room to grow—with more data and parameters, it could unlock even broader applications.", "weaknesses": "1. **Background Drift in Local Edits**.\nI noticed some background preservation issues in local editing tasks—there seems to be temporal drift in unedited regions. Could you elaborate on when and why this happens? Understanding these failure cases would be helpful.\n\n2. **Why Text-Only Localization Work So Well?**\nMost video editing models need masks for precise object editing, but EditVerse seems to work with just text instructions. This is quite impressive but also surprising. What makes this possible? Is it the scale of training, the unified architecture, or the cross-modal learning? A deeper analysis here would really strengthen the paper's contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)