Matting Anything 2:  Towards Video Matting for Anything
Authors: Chenyi Zhang
Yiheng Lin
Yunchao Wei
Hongsong Wang
Caifeng Shan
Fang Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Video Matting
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Video matting is a crucial task for many applications, but existing methods face significant limitations. They are often domain-specific, focusing primarily on human portraits, and rely on the mask of first frame that is challenging to acquire for transparent or intricate objects like fire or smoke. To address these challenges, we introduce Matting Anything 2 (MAM2), a versatile and robust video matting model that handles diverse objects using flexible user prompts such as points, boxes, or masks. We first propose Promptable Dual-mode Decoder (PDD), an effective structure that simultaneously predicts a segmentation mask and a corresponding high-quality trimap, leveraging trimap-based guidance to improve generalization. To tackle prediction instability for transparent objects across video frames, we further propose a Memory-Separable Siamese (MSS) mechanism. MSS employs a recurrent approach that isolates trimap prediction from potentially interfering mask memory, significantly enhancing temporal consistency. To validate our method's performance on diverse objects, we introduce the Natural Object Video Matting dataset, a new benchmark with substantially greater diversity. Extensive experiments show that MAM2 possesses exceptional matting accuracy and generalization capabilities. We believe MAM2 demonstrates a significant leap forward in creating a video matting method for anything.

Published as a conference paper at ICLR 2026
MATTING ANYTHING 2:
TOWARDS VIDEO MATTING FOR ANYTHING
Chenyi Zhang', Yiheng Lin', Yunchao Wei", Hongsong Wang', Caifeng Shan', Fang Zhao' "Nanjing University, BBiijing Jiaotong University, 'Southeast University
zcheny1007@gmaii com, 231202980bjtu edu.cn, en, wychaol 98 9870gmai1 com hongsongwang@seuu edu. ofshan@n ju.edu.cn. fzhaoln ju edu.cm
ABSTRACT
Video matting is crucial task for many applications, but existing methods face sigmificant limitations. They are often domain-specific, focusing primarily on hu- man portraits, and rely the mask of first frame that is challenging to acquire for transparent or intricate objects like fire or smoke. To address these challenges, we introduce Matting Anything 2 (MAM2), versatile and robust video matting model that handles diverse objects using flexible user prompts such as points, boxes, or masks. We first propose Promptable Dual-mode Decoder (PDD), an effective structure that simultancously predicts a segmentation mask and corre- sponding high-quality trimap. leveraging trimap-based guidance improve gen- eralization. To tackle prediction instability for transparent objects across video frames, we further propose a Nemory-Separall Siamese (MSS) mechanism. MSS employs recurrent approach that isolates trimap prediction from potentially interfering mask memory, significantly enhancing temporal consistency. To vali- date method's performance on diverse objects, we introduce the Natural Ob- ject Video Matting dataset, a new benchmark with substantially greater diversity. Extensive experiments show that MAM2 possesses exceptional matting accuracy and generalization capabilities. We believe MAM2 demonstrates a significant leap forward in creating video matting method for anything. The code is available
INTRODUCTION
Video matting, the process of precisely extracting the foreground alpha matte from video sequence, is a critical enabling technology for a myriad of applications. It is fundamental to the visual effects industry for seamless cinematic composition, powers virtual backgrounds in video conferencing, and facilitates realistic object integration in augmented reality experiences.
Despite sigmificant advancements in the field, existing video matting methods still exhibit several critical limitations: i) Domain Specificity: The vast majority of recent video matting models are predominantly human-centric Lin al. (202Tb): Huynh et al. (2024); Yang et al. (2025): Li et al.) (2024a); Ke et al. (2022); Ge al. (2025), focusing almost exclusively human portrait. Research into matting for more general natural scenes remains largely underexplored, a stark contrast to the well-developed state of natural image matting. Concurrently, illustrated in Table I compre- hensive benchmark for evaluating the generalization capabilities of video matting models on diverse natural scenes is conspicuously absent. ii) Reliance Mask: Popular methods often follow the standard semi-supervised Video Object Segmentation (VOS) framework Yang et al. (2021): : (Callee) al.) (2017); Yang et al. (2025). This framework necessitates user interaction the first frame, typ- ically by providing mask to specify the target object. However, a high-quality mask cannot always be easily obtained via an interactive segmentation model, particularly for certain types of objects. For transparent objects (e.g., smoke, fire), many regions exhibit a high degree of transparency that difficult for both the human eye and the model to discern, making such a mask challenging to acquire In this case, bounding box is clearly a more efficient form of interaction, as the user only needs to roughly enclose the object, eliminating the need for pixel-level mask correction.
"Corresoonding author.

---

Published as a conference paper at ICLR 2026
Box Prompt
Matting Anything 2
Prompt
Figure 1: Matting Anything 2 is video model that can be directly driven by user prompts and is able to process diverse objects addition to human portraits.
Table I: Statistics of popular video matting benchmarks. Count is calculated based on the number of distinguished foregrounds. Average Duration refers the average frame count of the clips. Domain refers to the categories of objects.
Test Set
Numher
Average Duration
Domain
Vid0 Lin et al. (b02
25
100
human
VideoMatting 108 Zhang et al. (2021b
human, cloth, smoke human
YoutubeMatte Yang et al. 2025
32
100
animals, bubble, cloud, fire, water, frost, plant...
Natural Video Matting
50
164
To address these challenges, we introduce Matting Anything 2 (MAM2), a video matting model that handle diverse objects. We developed MAM2 by building upon the foundation of SAM2 Ravi jet al.] (2024). thereby inheriting its excellent interactive capabilities. This allows MAM2 to ac- cept various forms of interaction, including points, boxes, and masks. Furthermore, paradigm of recent video matting models, which requires user interaction only on the first frame, minimizing the user's interaction cost as much as possible, as shown Fig.
follow the
To endow MAM2 with strong generalization capabilities, we propose the Promptable Dual-mode Decoder (PDD). contrast the original mask decoder of SAM2, PDD can simultaneously predict an object's segmentation mask and its corresponding trimap, which serves strong guidance for the final alpha matte prediction. We adopted this technical approach, motivated by the demonstrated dominance of trimap-based methods Dosovitskiy et al. (2020): Hu et al. (2024) in the field of natural image matting. By strengthening the guidance from the mask for the trimap prediction, PDD is able to generate high-quality trimaps for common objects.
However, we found that simply using PDD to predict per-frame trimaps for transparent objects results in unstable quality. This issue is primarily caused by the decoding mechanism of SAM2. For frames without user prompt (i.c., all frames after the first), the decoding process relies mainly on embedding the mask memory from the previous frame into the image features. Yet, for transparent objects, decoding trimap based on mask memory is particularly challenging. This is because large transparent areas require the prediction of more unknown regions the trimap rather than foreground regions. This, in turn, increases the discrepancy between the trimap and the mask, which means that the disparity between the ideal features required for their respective decoding also grows. To resolve Siamese (MSS) mechanism. MSS employs a recurrent
this, we propose the Memory-Separabee approach bypass the interference that mask memory causes during trimap decoding: after the segmentation mask is predicted by PDD, this mask is used as a prompt to drive the PDD second time to generate the trimap. Crucially, this second decoding pass utilizes pre-saved image features that have not undergone the mask memory embedding, Experiments show that MSS significantly improves the stability of trimap predictions for transparent objects. Furthermore, since the two passes share parameters, this siamese architecture adds additional parameters.
To validate the generalization performance of our method on natural scenes, we introduce a new, advanced benchmark: the Natural Object Video Matting (NOVM) dataset. In contrast to existing video matting test sets, NOVM exhibits significantly greater domain diversity, encompassing cate- gories beyond portraits such plants, fire, water, and more, shown in Table
2

---

Published as a conference paper at ICLR 2026
We conducted extensive experiments both existing human portrait benchmarks and newly proposed NOVM dataset. Compared to the state-of-the-art model, MAM2 reduces the MAD from 39.44 to 14.72 on NOVM (Natural Object Video Matting) and from 2.05 to 1.16 Youtube- Matte Yang et al. (202) (human video matting). Therefore, MAM2 is not a model specifically optimized for transparent objects; it also outperforms existing methods in human matting. Further- more, MAM2 is also a powerful image matting method. We also evaluate its performance on image matting tasks, where it achieves competitive results. We believe that Matting Anything holds immense value for practical applications.
RELATED WORKS
2.1 VIDEO OBJECT SEGMENTATION
Video Object Segmentation (VOS) tasks are primarily divided into several categories: unsuper- vised VOS, semi-supervised VOS, referring
et al. 2025); Cuttano et al. (2025); ; Li et al.
(203bb) and interactive VOS. Unsupervised VOS Lee et al. 2202 Li et al. (2024c); Cho et al.
2024): Zhuge et al.] (024)) does not require any user-provided annotations for guidance, allowing
the model perform segmentation automatically. However, this type of methods suffers from the
inability to specify a target object and often exhibits lower accuracy and consistency.
Consequently, plenty of work has been dedicated semi-supervised VOS methods Caelles et al. (2017). These methods require the user to provide an object mask for the first frame, and the model then segments the object in all subsequent frames based on this initial mask. Numerous classic architectures have been proposed to address this task. These include mask propagation- bbsed meth- ods Oh a (2018); Garg
Garg & Goel (00TT), which use the mask from the previous frame as guidance for the current one to achieve coherent segmentation throughout the sequence, and memory-based methods Oh et al. (2019); Cheng & Schwing (2022); /Zhou et al.] (2024), which rely on feature matching between the current and historical frames ensure temporal consistency.
Recently, the introduction of SAM2 has drawn significant attention to interactive VOS methods. This category of approaches aims to achieve video segmentation through more user-friendly inter- actions, such as clicks. Moreover, users can refine the segmentation results based on the model's s predictions. In partieular, SAM2's compatibility with multiple prompt types and its strong gener- alization capabilities have inspired a considerable amount of subsequent work Ding et al. (2024);
Cuttano al. 2025 Yang al. (2024).
2.2
VIDEO MATTING
Video Matting methods similarly be categorized based the type of user guidance they require. Automatic video matting models Ke et al. (2022): Lin et al. (2022). can predict precise alpha mattes without any user input, but they lack the ability to specify a target object. In contrast, approaches that follow the VOS paradigm Yang et al. (2025): : Huynh et al. (2024) require user-provided mask in the first frame select the target for matting. However, both of these method categories are typically limited to processing human or animal portraits.
Alternatively, other methods utilize trimap instead of a mask of the first frame for guidance Seong et al. (2022); Huang & Lee (2023). Guided by the stronger prior information provided by trimap, these methods not only achieve high matting accuracy but have also shown the potential handle objects beyond portraits. However, it is evident that the user interaction cost of providing trimap significantly higher than that of mask.
Additionally, there is a special type of background- video matting methods Lin et al. (2021)); Sengupta et al.) (2020); Xu et al. (2021). These methods require the user to provide an image of clean background without the foreground object, auxiliary information to achieve precise matting. However, in practical applications, a clean background is often unavailable. Furthermore, these methods impose strict requirements on background consistency, making them unable to adapt to temporal changes. Therefore, compared to the aforementioned methods, background based matting is less practical to deploy in real-worke scenarios.
3

---

Published as a conference paper at ICLR 2026
3 METHODOLOGY
Uphnweight Mucter
Matting Duto
Preditte Apta Haza
Trmmp
Therep coes
Encoder
Segnenntation
Matting
Data
Stage
Stage
Frame
Stamese Hochanism
Saparaite
Sutvective
Rainable
Promptable
(c) Memory-Separanee
Dual-mode Decoder
Hosk Ourput Taker
Mechaniem
Hook
Image
Feature Hemory
Memory
Feature with Hemory
Signood
Teaturm
Comy
Mocorry Bank
Mesk Augment Fewm.um
Rorurd Prampe
POD First Pass
Saconc/The
Conceter 602
Trimap TutputTolion
Det Product
Paramatter Sharing
Trimpp
Mark Dww.tadiinnnnnno..
Figure 2: Architecture of Matting Anything 2. MAM2 first predicts the object's trimap based on the user's prompt, and then uses the trimap to predict the final alpha matte.
3.1 OVERALL FRAMEWORK
As shown in Fig. MAM2 is capable of predicting alpha mattes for an entire video, requiring user interaction on only the first frame. Users can provide prompts directly to MAM2, eliminating the need for an auxiliary interactive segmentation model to generate mask for the first frame. Furthermore, MAM2 inherits the strong interactivity of SAM2, allowing to support various prompt types such as boxes, points, and masks, which ensures high user-friendliness.
We implement this inference process using a progressive decoding pipeline. As depicted in Fig- 2, the pipeline begins with image encoder of SAM2 finetuned by LoRA that extracts the image feature video frame. Subsequently. the
Siamese (MSS) mechanism utilizes two sequential Promptable Dual-mode Decoder (PDD) passes to predict the target's mask and trimap based on the user prompt. Finally, a lightweight trimap- based matting module predicts the final alpha matte. MAM2 is compatible with trimap-based matte model of various architectures. We choose MEMatte Lin et al. (2025) as our lightweight matter here due to its efficiency, with details provided A.
3.2 SELECTIVE SUPERVISION SCIIEME
The task of Video Matting (VM) suffers from significant data scarcity, compelling most ap- proaches Yang et al. (2025): Huang Leee (203)) to supplement their training data with Image Matting (IM) or Video Object Segmentation (VOS) data. Thus, propose selective supervision scheme to better utilize the knowledge contained within these heterogeneous data sources. The im- plementation of this strategy is enabled by MAM2's capability to concurrently generate multiple outputs, including the mask, trimap, and alpha matte.
To facilitate this specialized learning, partition the training procedure into stages. Let the model parameters è be partitioned into f for the main components (encompassing all parameters excluding the lightweight matter module) and 0 for the lightweight matter module.

---

Published as a conference paper at ICLR 2026
The first stage is designed to optimize 8main using data from IM, VM, and VOS. Accordingly, the loss for this main stage, Lmain, is formulated as follows:
Lmain
Ivoo Lmask (M, ym) + (IvM + IMM Liiimap (T, yr)
(I)
where M and T are the mask and trimap outputs, with corresponding ground truths уM and yr. The indicator functions llvos, livM, and LIM selectively activate the appropriate loss term based on the type of data randomly loaded by the dataloader from the mixed training set. This stage trains MAM2 to robustly produce coarse mask or a trimap.
In the second stage, only the lightweight matter parameters matttrr are optimized. For this stage, we define a separate loss, Lmmatter:
I.matter
TIM Lalpha (ox, ya)
where is the final predicted alpha matte and Yo is its ground truth. This stage exclusively uses image matting data for supervision. This is because this stage focuses on learning fine-grained detail perception, which demands high-fidelity annotations. Notably, the annotation quality of image matting data is significantly superior that of video matting data, an observation also noted in MatAnyone Yang et al. (2025).
Further details concerning the assignment of specific data sources to the optimization of different model parameters can be found in Appendix AA2 The composition of each category of datasets can be found in Appendix [A.3] The detailed formulation of each loss function and the sampling strategy
for the training data are provided in Appendix A.5
3.3 PRELIMINARY
To facilitate a better understanding of method, begin by providing brief preliminary the decoding mechanism of SAM1& &2 Kirillov et al.] (2023); Ravi et al. (2024) before detailing the model architecture of MAM2. To ensure consistency with the task setting of this paper, all subsequent descriptions are based on the standard semi-supervised VOS setting, where the user provides a prompt only for the first frame of a video sequence.
As SAMI is an image segmentation model that requires user-providde peer-provided prompts, it utilizes Prompt- able Mask Decoder. This decoder takes the features extracted by image encoder and the user's prompt as input to predict the segmentation mask. This process can be formally formulated as:
M SDecoder (F, Puser)
where M is the predicted segmentation mask, F represents the image feature, and Puser is the user- provided prompt.
However, since SAM2 is required to predict masks for every frame based on sparse prompts, memory mechanism is introduced to address this challenge. For subsequent frames, which lacks prompt, the memory mechanism embeds the mask prediction from the previous frame into the current image features via memory attention. This embedded memory functions as an implicit prompt, replacing the absent user prompt and driving the Promptable Mask Decoder's operation. The per--eeam per-frame decoding process of SAM2 can be formally described as:
[ DDeoder (F from mem
P Puser) DDecuderr (F' (F (),
if if
M
>
where t is the frame index, M is the mask prediction at frame t, Ft. Foon-mem represents the image feature of frame t without memory embedding, F'nem denotes the image feature of frame t embedded with memory of mask predictions of previous frames, and represents the absent user prompt.
3.4 PROMPTABLE DUAL-MODE DECODER
MAM2 operates as progressive decoding pipeline; consequently, it necessitates the preliminary prediction of the target's trimap to facilitate the subsequent prediction of the final alpha matte. To address this requirement, we propose the Promptable Dual- mode Decoder (PDD). Distinet from the original SAM2 decoder, the PDD is capable of simultaneously predicting both segmentation mask
5

---

Published as a conference paper at ICLR 2026
and trimap for every frame. We denote the PDD the function fPDD. Similar Equation 4 frDD can be described as:
(M',7") SPDD Fr non-mem: Puser); (PDD FL Fimm 0);
if 1 0
(5)
if t
where M and T represent the mask and trimap prediction at frame t, respectively.
However, predicting an additional trimap based on the original SAM2 decoder is non-trivial. While line of work has focused on refining the coarse masks predicted by SAMI into finer and more detailed ones Ke et al. (2023; ; Liu et al. (2024), these masks, whether coarse or fine, remain se- mantically consistent. The trimap we aim to predict, however, is fundamentally a different form of semantic representation. To address this distinction, SEMatte Xia et al. (2024), SAM-based image matting method, proposes employing a fully independent parallel branch prior to the final decoding to predict this distinct representation. However, our findings indicate that while simple parallel structure facilitates the simultaneous prediction of masks and trimaps, the resulting trimaps are of- ten noisy, frequently exhibiting jagged artifacts along the boundaries, illustrated in the second row of Fig.
Video Frames
w/ Parallel
w/ PDD
Figure 3: PDD improves the quality of the trimap prediction. Zoom in to observe details.
We attribute this instability to the fact that simple parallel branch fails to fully leverage SAM's robust semantic understanding and the exceptional stability of its mask predictions. Therefore, we consider it essential to incorporate strong guidance from SAM2's 's mask predictions into the parallel branch. Motivated by this, we propose the trimap decoding flow in PDD, as illustrated in part (b) of Fig. 2
First, the predicted mask is normalized with a sigmoid function and then processed by convolu- tional layer generate mask augment feature. Then, the mask augment feature is concatenated with the original segmentation feature and feature. Subsequently, lightweight fusion mod- ule performs mask-guidee enhancement these concatenated features. Finally, the final trimap obtained by computing the dot product between the fused features and the trimap output token. As illustrated in Fig. 3 this simple design leads quality of the predicted trimap. This, in turn, enhances the quality of the final predicted alpha matte. This simple design yields improvements of 24% and 29% on the natural object and human portrait benchmarks, respectively, as shown in the top two rows of Table
significant improvement in the stability and
In addition, PDD inherits sam2's excellent compatibility with multiple prompt types, which makes the implementation of Memory-Separable Siamese Mechanism possible, as will be discussed in detail the next section.
3.5
SEPARAE SIAMESE MECHANISM
A "matting anything" method must do more than just competently handle portraits objects where fine details are concentrated at the boundary. A true test of its capability lies in processing challeng- ing transparent objects like fire and bubbles, which are characterized by extensive transparency and complex details.
When applied our PDD-equipped MAM2 to these objects, we discovered strange temporal collapse. While the trimap for the first frame is predicted accurately, from the second frame onward, found that unknown regions in the trimap were prone to being misclassified as foreground, as

---

Published as a conference paper at ICLR 2026
shown in Fig. 4 For our lightweight matter, which relies the trimap predicted by PDD to predict the final alpha matte, such false positive errors of foreground are known to be catastrophic, particularly for objects with large-scale transparency.
w/o MSS
w/ MSS
Video
Predicted Trimap
Predicted
Alpha Matie
Extracted
Foreground
Frame-O
Frume-1
Frame-20
Frame-O
Frame-1
Frame-20
Figure 4: The entire region of the bubble should be predicted as unkmown in the trimap. MSS effec- tively mitigates the collapse of trimap prediction for subsequent frames when handling transparent objects.
As discussed in Section 3.3 and Equation 5 the decoding process for the first frame driven by the user prompt, whereas for subsequent frames, it relies on the memory of the previous mask predic- tion embedded within the image features. Consequently, this anomalous temporal collapse can be attributed to specific disparity: while PDD performs accurate trimap decoding when guided by user prompts, fails to do so when guided by the memory embedded in the image features. Therefore, identify the root cause as the process of embedding the memory of previous mask predictions posit that this embedding operation induces a significant shift in image feature space, which severely interferes with the trimap decoding process. Corroborating idence for this is observed the erroneous trimap predictions of subsequent frames, which exhibit tendency resemble binary segmentation masks rather than trimaps. Notably, these predictions of their actual transparency- as
into the image features.
display a strong bias towards classifying object definite foreground rather than the unknown category required for trimaps.
We consider this phenomenon to be logical. Given that masks and trimaps represent distinct seman- tic concepts, inconsistency between the ideal feature spaces required for their respective decoding is to be expected. In essence, while subsequent frames rely on mask memory for decoding, the em- bedding of this memory simultaneously interferes with trimap prediction.
To overcome this dilemma, we introduce the Separab- Siamese Mechanism (MSS), recurrent approach trimap decoding. For subsequent frames, once PDD computes a segmentation mask, we leverage this mask as a eseudo-prompp to drive a second PDD pass. Crucially, this second pass is performed on a preserved, memory-free version of the image feature that was saved prior to the memory attention.
Consequently, the decoding of subsequent frames is effectively realigned to rely on memory- features and "user prompt", thereby avoiding the interference caused by the shifted image feature on trimap decoding, as indicated by the purple connecting line in part (c) in Fig. 2 The trimap generated from this second pass is then selected as the final output. This process can be formally formulated as:
M = nı (fPDD i 9))
(6)
T कं (JPDD non-mom: M³)
(7)
where nº denotes the projection function that extracts the z-th element from the output tuple (e.g., T extracts the mask and TT extracts the trimap).
As illustrated in Fig. 4 MSS substantially enhances the robustness MAM2 on challenging objects characterized by large-scale transparency. Furthermore, since the mask used drive the second pass

---

Published as a conference paper at ICLR 2026
is decoded from features with memory, temporal consistency of the trimap can be transmitted and maintained by this mask. addition, since the PDD weights are shared between two passes, MSS siamese architecture, adding no additional parameters. Mcanwhile, because PDD is built upon SAM's lightweight mask decoder, the computational overhead of this second pass is negligible.
EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Consistent with recent video matting methods Yang et al. (2025), MAM2 is trained on multiple types of data: video object segmentation Ding et al. (2023). video matting Zhang et al. (2021b). image segmentation Qin et al. (2022), and image matting Xu et al. (017); Qiao al. (2020) Li et al. (202)]; Ma et al. 2023); Cai et al. 2022). No private or proprietary data was used to train MAM2.
The entire training set composed of publicly accessible datasets. The specific datasets used for
training are detailed in Appendix A.3 A..3
Omain finetuned from pretrained SAM2 weights for 100 epochs using the AdamW optimizer with an initial learning rate of 4 x 10-1 The LoRA rank for the encoder set to 16. matter. are trained from ViT-Small model initialized with DINO pretrained weights. This component trained for approximately 3,500 iterations using AdamW with an initial learning rate of x 10 The total number of trainable parameters 44.7M. Further training details are available in the Appendix A
4.2 NATURAL OBJECT VIDEO MATTING DATASET
To evaluate the performance of methods on diverse objects, we introduce Natural Object Video Mat- ting (NOVM), the first video matting benchmark composed of a rich variety of natural objects. The construction of NOVM began with the collection of After Effects assets that included preexisting alpha mattes, allowing them to be used directly as assets in content creation. Subsequently, this col- lection filtered to discard assets with insufficient detail, well as all cartoon-styles styledd clips, which we noted constituted a significant portion of the initial set. Finally, the retained assets are composited onto high-resolution and dynamic backgrounds to produce the final video clips and corresponding alpha matte clips.
The final NOVM dataset contains 50 clips, each featuring distinet foreground object action and a dynamic background. Most importantly, NOVM covers a highly diverse range of object domains, presenting formidable challenge existing video matting models. We provide several examples of NOVM in Fig and the breakdown of NOVM in Table
4.3 VIDEO MATTING
Table 2: Quantitative comparison with other video matting methods in interactive mode on NOVM and YoutubeMatte Datasets.
NOVM (natural objects)
YoutubeMatte (human)
Method
Prempt
MAD
MSE L
GRAD 1
dtSSD |
MAD 1
MSE I
GRAD 1 dtSSD J
TCVOM Zhang al. 222
Trimap
56.18
38.90
153.95
1.57
0.40
FTP HMMHlnngg
2023
Trimap
37.98
19.90
78.06
4.24
2.26
1.10
5.63
1.70
MaGGGe Haynh et al
A124
Mask
50.04
35.23
108.01
4,90
2.37
0.98
7.69
1.77
MatAnyone Yang
Mask
39.44
25.63
89.60
0.76
Matting Anything (Ours)
Mask
15.19
4.27
26.45
2.80
1.16
0.24
3.12
1.21
Matting Anything (Ours)
Box & Point
14.72
3,70
23.54
2.65
1.16
0.24
3.07
1.20
We first compare the performance of the methods when prompted by a mask of the first frame, following the setting semi-supervised VOS task, as presented in Table 2 We select NOVM and YoutubeMatte Yang al. (2025) as test datasets. The former is used to evaluate the model's performance on diverse objects, while the latter assesses its performance on typical human portraits. MAM2 demonstrates significant advantage across all metrics, including MAD and MSE for overall prediction, Grad for detail fidelity, and dtSSD for temporal consistency. More importantly. MAM2 can even perform matting without the need for user-provided mask; it can be driven directly by
8

---

Published as a conference paper at ICLR 2026
user-provided points or boxes. Even in this mode, MAM2 continues exhibit exceptionally strong performance.
Following the MatAnyone Yang al. (2025), also evaluated the performance of MAM2 in an automatic human matting mode. This mode operates by using an automatic human matting model to obtain the matte for the first frame. For fair comparison, both MAM2 and MatAnyone utilize RVM to generate this first-frame matte. As shown Table MAM2 also demonstrates the strongest overall performance in automatic matting.
Table Quantitative comparison with other video matting methods in automatic mode on Youtube- Matte and VM240K Datasets.
YoutubeMatte (human)
VM240K (human)
Method
MAD
MSE +
GRAD +
dtSSD
MAD 4
MSE
GRAD
dtSSD
MODNet 2022
15.29
12.68
8.42
2.74
11.13
5.54
15.30
3.08
RVM et 2022
4.37
2.25
15.1
2.28
6.57
1.93
10.55
1.90
Large
3.50
1.19
12.64
2.08
5.81
0.97
9.65
1.78
MatAnyone Yang
3.70
2.35
11.45
1.81
5,66
1.68
5.75
1.27
Matting Anything (Ours)
1.19
0.27
3.17
1.23
5.10
1.10
4.15
1.26
4.4 IMAGE MATTING
Recently, a wave of image matting methods built upon the foundation of SAM 1/2 has emerged, including ZIM Kim al. (2025), Matting Anything Li et al. (2023a), and SEMatte Xia et al. (2024). MAM2 not only shows strong performance in video matting but is also powerful image matting model. For image matting, MAM2 can also be driven by efficient user prompts such as points and boxes. We provide a comparison with other image matting methods in Table 4 Specifically, in addition to visual prompt, SDMatte requires a flag to specify whether the matting target is transparent object. Therefore, to ensure fair comparison with other methods, we report the results for SDMatte without this text prompt. provide visualization results of image matting in Appendix C.2 Notably, the model parameters of MAM2 for the video matting and image matting tasks are identical.
Table 4: Quantitative comparison of image matting methods with various prompt types on AIM- 500. N/A indicates that the model does not support Click mode. The denotes different version of SDMatte.
Method
Prompt
MSE
SAD
Grud
Comn
Prompi
MSE
SAD
Grad
Conn
Matting Anything ci al. 2024b
Box
11.60
36.66
21.04
18.99
Point
7.52
186.50
37.4 .8.8
40.38
SmartMatting Ye ct 2024
Box
7.65
25.33
27.16
13.54
Point
30.20
66.27
46.63
18.77
SEMarte Xia èt 2024
7.65
16.06
13.64
Point
N/A
N/A
N/A
N/A
SDMatte Huang of 2125
Box
10.04
29.35
24.06
15.62
Point
11.93
33.57
29.15
18.15
SDMatte Tuung al. 2025
Box
4.91
19.81
15.84
11.97
Point
N/A
N/A
N/A
N/A
Matting Anything 2 (Uur))
Box
4.24
18.07
13.88
11.01
Point
5.68
20.78
14.63
10.90
4.5 ABLATION STUDY
We present the ablation studies for PDD and MSS here. The first two rows of Ta the ablation study for PDD, where Parallel refers to the simple parallel structure for trimap prediction, as described in See 3.4 In conjunction with Fig. 3 is evident that PDD substantially improves the stability of the predicted trimap, thereby enhancing the final matting quality.
Rows 2, 3, and 4 detail the ablation study for MSS. this context, MCS (Memory-Consistent Siamese) represents the strategy that simply performs second pass of PDD based on memory- embedded image features. The results indicate that the critical factor for MSS's significant improve- ment is not extra decoding pass (as evidenced by the lack of improvement from MCS). Instead, the key is the utilization of memory-free image features during the second PDD pass, which prevents interference from mask memory in the trimap decoding process.

---

Published as a conference paper at ICLR 2026
Table 5: Ablation study of different components of Matting Anything
NOVM
Youtube
Parallel
PDD
MCS
MSS
MAD
MSF L
GRAD І
dISSD
MAD 4
MSF.
GRAD
diSSD
26.19
13.21
43.29
3.14
1.54
0.52
3.49
1.30
18.55
6.77
29.77
2.91
1.16
0.24
3.08
1.19
20.23
8,59
39.26
2.85
1.18
0.26
3.14
1.19
14.72
3.70
23.54
2.65
1.16
0.24
3.07
1.20
Video Framess
FIP-VM
MAM2 (Owns)
MalAmyyon
Video Fnimes
FTP-VM
MAM2 (Ows)
T
Mat.Aayone
T
Figure 5: Visual comparison with other video matting methods. Zoom in observe details.
4.6
VISUALIZATION
provide visual comparison of MAM2 with MatAnyone Yang et al. (2025) and FTP- VM Huang Lee (2023), which represent the state-of-the-arr in mask-guided and trimap- guided video matting, respectively. MAM2 provides higher-quality matting results even when driven directly by user-provided points and boxes.
In Fig.
We provide extensive visualization results in Appendix C.
Specifically, Figs. 8 and demonstrate
performance on the NOVM dataset, Fig. [0 on the YouTubeMatte dataset, and Figs. I and 12 on real-world videos. We also provide several results in MP4 format in the supplementary material.
CONCLUSION
In this paper, we introduce Matting Anything 2, a powerful model designed for matting any object in videos. We propose Promptable Dual-mode Decoder to enable seamless, interactive workflow and Memory-Separable Siamese mechanism to enhance generalization for complex objects by re- solving memory conflicts without adding parameters. To facilitate robust evaluation, we also present the Natural Object Video Matting dataset, a new benchmark with significant domain diversity. Ex- perimental results demonstrate that MAM2 significantly outperforms existing methods, establishing a new trate-of-the-ae for both natural and portrait scenes. believe this work holds immense potential for significant practical applications.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (No. 62476124, 62172090), Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China UYB2025XDXMM902) Natural Science Foundation of Jiangsu Province (No. BK20242015, BK20230833), and the Gusu Innovation and Entrepreneur Leading Talents (No. ZXL2025322).
REFERENCES
Sergi Caelles, Kevis-Kokite Maninis, Jordi Pont-Tuset, Laura Leal-Taixé, Daniel Cremers, and Luc Van Gool. One-shot video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 221-230, 2017.
Huanqia Cai, Fanglei Xue, Lele Xu, and Lili Guo. Transmatting: Enhancing transparent objects
matting with transformers. European Conference on Computer Vision (ECCV), 2022.
Ho Kei Cheng and Alexander G Schwing. Xmem: Long-term video object segmentation with atkinson-shiffrin memory model. In European conference on computer vision, pp. 640-658. Springer, 2022.
Suhwan Cho, Minhyeok Lee, Seunghoon Lee, Dogyoon Lee, Heeseung Choi, Ig-Jae Kim, and Sangyoun Lee. Dual prototype attention for unsupervised video object segmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19238- 19247, 2024.
Claudia Cuttano, Gabriele Trivigno, Gabriele Rosi, Carlo Masone, and Giuseppe Averta. Samwise: Infusing wisdom in sam2 for text-driven video segmentation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3395-3405, 2025.
Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Philip HS Torr, and Song Bai. MOSE: A new dataset for video object segmentation in complex scenes. In ICCV, 2023.
Shuangrui Ding, Rui Qian, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Yuwei Guo, Dahua Lin, and Jiaqi Wang. Sam2long: Enhancing sam 2 for long video segmentation with training- free memory tree. arXiv preprint arXiv:2410.1624 2024.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 1929, 2020.
Shubhika Garg and Vidit Goel. Mask selection and propagation for unsupervised video object seg- mentation. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision, pp. 1680-1690, 2021.
Yongtao Ge, Kangyang Xie, Guangkai Xu, Li Ke, Mingyu Liu, Longtao Huang, Hui Xue, Hao Chen, and Chunhua Shen. Generative video matting. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Pupers, pp. 1-10, 2025.
Qiqi Hou and Feng Liu. Context-awarr image matting for simultaneous foreground and alpha timation. In Proceedings of the IEEF/CVE International Conference on Computer Vision, pp- 4130-4139, 2019.
Yihan Hu, Yiheng Lin, Wei Wang, Yao Zhao Yunchao Wei, and Humphrey Shi. Diffusion for natural image matting. In European Conference on Computer Vision, pp. 181-199. Springer, 2024.
Longfei Huang. Yu Liang, Hao Zhang, Jinwei Chen, Wei Dong, Lunde Chen, Wanyu Liu, Bo Li, and Peng- Tao Jiang. Sdmatte: Grafting diffusion models for interactive matting. In Proceedings of the IEFF/CVF international conference computer vision, 2025.
11

---

Published as a conference paper at ICLR 2026
Wei-Lun Huang and Ming-Sui Lee. End-to-end video matting with trimap propagation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14337- 14347, 2023.
Chuong Huynh, Seoung Wug Oh, Abhinav Shrivastava, and Yoon-oung Lee. Maggie: Masked guided gradual human instance matting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Patlern Recognition, pp. 3870-3879, 2024.
Lei Ke, Minggiao Ye, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, and Fisher Yu. Segment anything in high quality. In NewrIPS, 2023.
Zhanghan Ke, Jiayu Sun, Kaican Li, Qiong Yan, and Rynson WH Lau. Modnet: Real-time trimap- free portrait matting via objective decomposition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 1 1140-1147 147, 2022.
Beomyoung Kim, Chanyong Shin, Joonhyun Jeong, Hyungsik Jung, Se- Yun Lee, Sewhan Chun, Dong-Hyun Hwang, and Joonsang Yu. Zim: Zero-shot image matting for anything. Proceed- ings of the IEEE/CVF International Conference Computer Vision, pp. 23828-23838, 2025.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.0200 30..00440.........00000 2023.
Minhyeok Lee, Suhwan Cho, Seunghoon Lee, Chaewon Park, and Sangyoun Lee. Unsupervised video object segmentation via prototype memory network. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5924-5934, 2023.
Jiachen Li, Jitesh Jain, and Humphrey Shi. Matting anything. arXiv: 2306.05399, 2023a.
Jiachen Li, Roberto Henschel, Vidit Goel, Marianna Ohanyan, Shant Navasardyan, and Humphrey Shi. Video instance matting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. (668-6677) 2024a.
Jiachen Li, Jitesh Jain, and Humphrey Shi. Matting anything. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1775-1785 2024b.
Jizhizi Li, Jing Zhang, Stephen J Maybank, and Dacheng Tao. Bridging composite and real: to- wards end-to-end deep image matting. International Journal of Computer Vision, 130(2):246- 266, 2022.
Ping Li. Yu Zhang, Li Yuan, Huaxin Xiao, Binbin Lin, and Xianghua Xu. Efficient long-short temporal attention network for unsupervised video object segmentation. Pattern Recognition, 146: 10078, 2024c.
Xiang Li, Jinglu Wang, Xiaohao Xu, Muqiao Yang, Fan Yang, Yizhou Zhao, Rita Singh, and Bhik- sha Raj. Towards noise-tolerant speech-referring video object segmentation: Bridging speech and text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 2283-2296, 2023b.
Tianming Liang, Kun-Yu Yu Lin, Chaolei Tan, Jianguo Zhang, Wei-Shi Zheng, and Jian-Fang Hu. Referdino: Referring video object segmentation with visual grounding foundations. arXiv preprint arXiv:2501. 14607 2025.
Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian I. Curless, Steven M Seitz, and Ira SamellmacherSShlizemmen Real-time high-resolution background matting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8762-8771, 2021a.
Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian I. Curless, Steven M Seitz, and Ira Kemelmacher-Shlizz Shlizerraaii...an....m Real-time high-resolution background matting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8762-8771 2021b.
Shanchuan Lin, Linjie Yang, Imran Saleemi, and Soumyadip Sengupta. Robust high-resolution video matting with temporal guidance. In Proceedings of the IEEÉ/CVF Winter Conference on Applications of Computer Vision, pp. 238-247, 2022.
12

---

Published as a conference paper at ICLR 2026
Yiheng Lin, Yihan Hu, Chenyi Zhang, Ting Liu, Xiaochao Qu, Luoqi Liu, Yao Zhao, and Yun- chao Wei. Memory efficient matting with adaptive token routing. Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 5298-5306 2025.
Mengzhen Liu, Mengyu Wang. Henghui Ding, Yilong Xu, Yao Zhao, and Yunchao Wei. Segment anything with precise interaction. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 3790-3799, 2024.
Qin Liu, Zhenlin Xu, Gedas Bertasius, and Marc Niethammer. Simpleclick: Interactive image segmentation with simple vision transformers. In Proceedings of the IEEE/CVE International Conference Computer Vision, pp. 22290-22300 2023.
Sihan Ma, Jizhizi Li, Jing Zhang, He Zhang, and Dacheng Tao. Rethinking portrait matting with pirvacy preserving. International Journal of Computer Vision, 2023. ISSN 1573-1405.
Seoung Wug Oh, Joon- Young Lee, Kalyan Sunkavalli, and Seon Joo Kim. Fast video object seg- mentation by reference- -eereenceguuided mask propagation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7376-7385, 2018.
Scoung Wug Oh, Joon- Young Lee, Ning Xu, and Scon Joo Kim. Video object segmentation us- ing space-time memory networks. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9226-9235, 2019.
Yu Qiao, Yuhao Liu, Xin Yang. Dongsheng Zhou, Mingliang Xu, Qiang Zhang, and Xiaopeng Wei. Attention-guided hierarchical structure aggregation for image matting. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Xuebin Qin, Hang Dai, Xiaobin Hu, Deng-Ping Fan, Ling Shao, and Luc Van Gool. Highly accurate dichotomous image segmentation. In ECCV, 2022.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.007/4 2024.
Soumyadip Sengupta, Vivek Jayaram, Brian Curless, Steven M Seitz, and Ira Kemelmacher- Shlizerman. Background matting: The world your green screen. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 229122292000 2020.
Hongje Seong, Seoung Wug Oh, Brian Price, Euntai Kim, and Joon- Young Lee. One-trimap video matting. In European Conference Computer Vision, pp. 430-448. Springer, 2022.
Konstantin Sofiiuk, Ilya A Petrov, and Anton Konushin. Reviving iterative training with mask guid- ance for interactive segmentation. In 2022 IEEE international conference on image processing (ICIP), pp. 3141-3145. IEEE, 2022.
Jingwei Tang, Yagiz Aksoy, Cengiz Oztireli, Markus Gross, and Tunc Ozan Aydin. Learning-basee bbaed sampling for natural image matting. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 5055-3063. 2019.
Ruihao Xia, Yu Liang, Peng-Tao Tao Jiang. Hao Zhang, Qianru Sun, Yang Tang, Bo Li, and Pan Zhou. Towards natural image matting in the wild via real- prior. arXhv:2410.0659) 2024.
Ning Xu, Brian Price, Scott Cohen, and Thomas Huang. Deep image matting. Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2970 2979, 2017.
Yong Xu, Baoling Liu, Yuhui Quan, and Hui Ji. Unsupervised deep background matting using deep matte prior. IEEE Transactions Circuits and Systems for Video Technology, 32(7):4324 4337, 2021.
Cheng- Yen Yang. Hsiang- Wei Huang, Wenhao Chai, Zhongyu Jiang, and lenq-Neng Hwang. Samu- rai: Adapting segment anything model for zero-shot visual tracking with motion-awe aware memory. arXiv preprint 2rXiv:2411.11922 2024.
13

---

Published as a conference paper at ICLR 2026
Peiqing Yang, Shangchen Zhou, Jixin Zhao, Qingyi Tao, and Chen Change Loy. Matanyone: Stable video matting with consistent memory propagation. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7299-7308, 2025.
Zongxin Yang, Yunchao Wei, and Yi Yang. Associating objects with transformers for video object segmentation. Advances in Newral Information Processing Systems, 34:2491-2502, 2021.
Jingfeng Yao, Xinggang Wang, Shusheng Yang, and Baoyuan Wang. Vitmatte: Boosting image matting with pre-trainn tre-trained plain vision transformers. Information Fusion, 103:102091, 2024.
Zixuan Ye, Wenze Liu, He Guo, Yujia Liang, Chaoyi Hong, Hao Lu, and Zhiguo Cao. Unifying tomatic and interactive mating with pretrained vits. In Proc. IEEE/CVF Conference on Computer Vision and Paltern Recognition (CVPR), 2024.
Yunke Zhang, Chi Wang, Miaomiao Cui, Peiran Ren, Xuansong Xie, Xian-Sheng Hua, Hujun Bao, Qixing Huang, and Weiwei Xu. Attention-guidet temporally coherent video object matting. In Proceedings of the 29th ACM International Conference Multimedia, pp. 5128 5137, 2021a.
Yunke Zhang, Chi Wang, Miaomiao Cui, Peiran Ren, Xuansong Xie, Xian-Sheng Hua, Hujun Bao, Qixing Huang, and Weiwei Xu. Attention-guided temporally coherent video object matting. In Proceedings of the 29th ACM International Conference Multimedia, pp. 5128- 5137, 2021b.
Junbao Zhou, Ziqi Pang, and Yu-Xiong Wang. Rmem: Restricted memory banks improve video ob- ject segmentation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 18602-18611. 2024.
Yunzhi Zhuge, Hongyu Gu, Zhang, Jinqing Qi, and Huchuan Lu. Learning motion and temporal cues for unsupervised video object segmentation. IEEE Transactions on Neural Networks and Learning Systems, 36(5):9084 9097, 2024.
A IMPLEMENTATION DETAILS
A.1 LIGHTWEIGHT MATTER
We employ lightweight trimap-based matter, MEMatte Lin et al. (2025), to predict the alpha matte. This matter integrates the trimap predicted by PDD with the original RGB image to produce the final alpha matte.
MEMatte standard trimap-basee based matting method bsaed on ViT Dosovitsi et al. (2020) back- bone. Its core advantage lies in an adaptive token routing mechanism that drastically reduces the number of tokens participating in computation within the global attention blocks of its ViT backbone. Because the computational complexity of the attention mechanism scales quadratically with the number of tokens, MEMatte can significantly lower the memory usage and latency during inference.
The target compression degree is set to 0.25, with a maximum token number of 12,000. For the router and lightweight token refinement module, the linear layers are initialized with truncated normal distribution (std=0.02), while the LayerNorm layers have biases set to zero and weights set The remaining modules initialized with the teacher model.
During training, data augmentations include random affine transformations, random cropping, ran- dom jitter, random horizontal flipping, and composition, among others. The inputs are randomly cropped into 1024 x 1024 patches. Essentially, MAM2 retains the same model architecture as MEMatte, with the only modification being the training data.
A.2 SELECTIVE SUPERVISION SCHEME
As illustrated in Fig. 6 we employ a two-stage training pipeline utilizing a Selective Supervision
Scheme.
14

---

Published as a conference paper at ICLR 2026
During Stage we deploy mixed dataloader that randomly samples training batches from hetero- geneous sources sources-speeciieeeeee sources--specfficllly Segmentation Data and Matting Data. This stage focuses on opti- mizing the main model parameters, @ muin= with the supervision signal adapting dynamically to the sampled data type. For Segmentation Data, the model is supervised via the Segmentation Loss on the predicted mask. Conversely, when Matting Data is sampled, the model minimizes the Trimap Loss. The ground truth trimap required for this supervision is generated by degrading the ground truth alpha matte through standard dilation, erosion, and quantization operations. This approach is a well-established technique in image matting for synthetically generating trimaps.
Subsequently, Stage exclusively optimizes the lightweight matter parameters, maneer loading only Matting Data to refine the final alpha matte under the guidance of the Matting Loss.
Stage 1 7 O main
When foeding Segmentstion dats:
Segmentation
Seginentation
MAM2-8
Predicted Trimap
When loading Matting
Prodicted Maak
Matting
Trimap
Degradation
Prodictec Ti mmp
GI Inimap
Stage 2 O matter ):
Exclusively loading Matting deta:
Marting
MAM2 .....
Matting Loss
Predicted Alpha Hutt
Figure 6: Pipeline of Selective Supervision Scheme.
A.3 TRAINING DATA
The field of video matting has long faced the issue of data scarcity. Consequently, individual meth- ods employ their own strategies to augment training data by sourcing it from other fields, such as video segmentation, image segmentation, image matting. As the field has evolved, the datasets used by different methods have diverged, making it exceedingly difficult to ensure identical training data settings for fair comparison. We can ensure that MAM2 is trained exclusively on available public datasets. We list the composition of the training data for recent video matting methods in Table 6 should be noted that the relatively small image counts for PIOK. AM-2K, DIS5K, and DUTS are due to the fact that MAM2 utilizes only subset of these datasets for training, rather than the full sets. For instance, while the P10K training set contains 9,421 images, MAM2 employs only 1, 1,000 of them.
Minbah
Ieage
Vakeo
Vestubeii
MOSE
COCO
SPD
DUTSS
DE6KK
JIM
Bus6s
AM-O
T-00
LHIMAOK
PINK
VM
2011
Staodde
<0013
333
MAN
17500
1536
Table 6: Composition of training datasets for different methods.
We have summarized the sampling probabilities for each dataset type, along with other fundamental configurations, in Table Num Frames denotes the number of frames loaded at once; this setting is only applicable video data.
15

---

Published as a conference paper at ICLR 2026
Sample Probability
Batch Size
Num Frames
Image Segmentation
0.25
Image Matting
0.25
Vidco Segmentation
0.25
Video Matting
0.25
Table 7: Settings for training with multiple datasets.
A.4 ABLATION OF TRAINING SETS
As shown in Table recent matting methods utilize diverse training configurations. Due to the non-public nature of certain datasets and training codes, it is infeasible to align the training settings across all methods. However, to demonstrate that the performance improvements of our method not derived from our specific training set, aligned and retrained MAM2. As evidenced in Table 8 MAM2 continues to exhibit significantly superior performance. We selected FTP-VM for this validation because it employs the most straightforward training data composition, consisting only of VM108, YouTube VIS, and D-646.
training data with that of FTP-VM
Table 8: Ablation study different training sets.
NOVM (natural objects)
Youtube (human)
Method
Training Data
MAD
MSF.
GRAD ) diSSD
MAD 1
MSF. |
GRAD J.
diSDD
TCVOM Zhane et
2021a
Respective Sets
56.18
38.90
153.95
3.84
1.57
0.40
1.52
PTP-VM Tonng 2025
Respective Sets
37.98
19.90
78.06
4.24
2.26
1.10
5.63
1.70
MaGGle Havas et
2029
Respective Sets
50.04
35.23
108.01
4.90
2.37
0.98
1.77
MalAnyein Yang
2025
Respective Sets
39.44
25.63
89.60
4.10
2.05
0.76
9.67
1.75
Matting Anything (Ours)
Same as FTP-VM
18.01
4.89
26.26
2.76
1.08
0.22
2.64
1.20
Matting Anything (Ours)
Respective Sets
14.72
3.70
23.54
2.65
1.16
0.24
3.07
1.20
A.5 Loss FUNCTION
The training of MAM2 involves three loss functions: Lmasks Luimap- and Lalpha.
Among these, Lmaas is inherited from the original SAM2, which is a combination of focal and dice loss for the mask, MAE loss for the predicted IoU score, and a cross cross-entropy loss for object prediction.
For Luimap- adopt the Normalized Focal Loss, loss function commonly used in interactive segmentation Sofiiuk et al. (2022): Liu et al. 2023) which is defined as follows:
NFL(i.)) Eis (1 pio) (1 Pi,)) log
(8)
where Pt,j denotes the confidence at (i.j) of the predicted trimap e #xx3
Lalpha loss is combination of separate /1 loss Yao et al. (2024), 12 loss, Laplacian loss Hou & Liu 2019), and gradient penalty loss Tang et al. (2019), tormulated as follows
Lalpha Liseparate 11 + L1z + Liaplacion Lgradient-
B NATURAL VIDEO MATTING DATASET
Table 9: Breakdown of the NOVM dataset.
Category
Animals
Bubble
Cloud
Explosion
Fire
Frost
Plant
Slime
Vehicles
Water
Sum
Quantity
4
5
4
7
50
Proportions
I 8%
8%
8%
10%
8%
8%
14%
4%
8%
14%
100%
The most distinctive feature of our proposed NOVM dataset is its category diversity, as illustrated in Table Furthermore, have balanced the number of samples per category ensure uniform
16

---

Published as a conference paper at ICLR 2026
class distribution. We provide several examples from the NOVM dataset in Fig. 7 which demon- strate that the domain diversity of its objects far surpasses that of existing vidco matting benchmarks, and that its alpha mattes are of exceptional quality.
Figure 7: Examples of NOVM dataset.
VISUALIZATION
C.I
VIDEO MATTING
We present the visualization results of MAM2 on the video matting task in Fig. and displaying both the predicted alpha mattes and the extracted foregrounds. Additionally, provide visualiza- tion results on the YouTubeatte dataset Fig. I0 and on real-world videos Fig. II and [2] We provide visual comparison with other methods Youtube Fig. 13 Given the limited
17

---

Published as a conference paper at ICLR 2026
fine details in public matting benchmarks, have supplemented the last of Fig. L with the video characterized by large-scale motion and complex hair structures. Several visualization results in MP4 format are also provided in the supplementary material.
Video Frames
Prompi
Predicted Alpha Motte
>
Extracted Foregreund
Video Frames
Prompt
Prodicted Alpha Matte
Extracted Foreground
Frames & Prompt
YOMR
YOUR
YOUR
Prodicted Alpha Matte
Extracted Foreground
Video Frames & Prompt
Predicted Alpha Matte
Extracted Fareground
& Prompt
Predicted Alpha Matte
Extracted Foreground
Figure 8: Visualization results of MAM2 on NOVM.
18

---

Published as a conference paper at ICLR 2026
C.2 IMAGE MATTING
We present MAM2's visualization results of MAM2 on the image matting task in Fig. 14
C.3 VIDEO EDITING
Since video matting empowers creators with control the layer and object levels, it is extensively applied in various creative editing workflows. In Fig. [5] we present several simple video edit- ing effects realized using MAM2, including background replacement, trajectory control and object scaling. These examples represent merely the tip of the iceberg regarding matting-basee editing; experienced and creative practitioners can undoubtedly leverage matting to generate far more so- phisticated and visually stunning effects.
D LATENCY
We compared the FPS (Frames Per Second) and parameter count of MAM2 against recent methods in Table TO with all tests conducted on an NVIDIA RTX 5880 GPU. While MAM2 is not as com- putationally efficient, it achieves significantly higher matting accuracy. In practical deployment, methods requiring a first-frame mask necessitate the deployment of auxiliary models to generate this mask. For instance, the official MatAnyone demo deploys an additional SAM-ViT-Huge Kir- illov et al.] (2023), while MaGGle utilizes SAM 2-Base Ravi et al.] (2024). Therefore, from the perspective practical deployment, we have included these auxiliary parameters to compare the total parameter count required by each method.
MAD |
Grad |
FPS
Params
Params of Auxiliary Model J
Total Params
MuGGle
50.04
108.01
18.87
30,91
111.71
MatAnyone
39.44
25.63
15.43
35.25
636.00
671.25
Matting Anything 2
14.72
3.70
11.24
256.83
256.83
Table 10: Efficiency comparison with other methods.
E LIMITATIONS
Despite MAM2 achieving promising performance, pioneering attempt video matting across such diverse domains, it inevitably exhibits certain limitations. These issues primarily manifest when the target object possesses high transparency. For instance, the explosion shown in the left part of Fig. 16 as the smoke dissipates, its opacity becomes extremely low. This results in a weak visual signal, causing the model to abruptly fail in capturing a significant portion of the smoke. As illustrated in the third column, while smoke actually persists with high transparency in the upper section of the ring, is missing from the extracted result.
Another issue pertains to the extraction of transparent objects based on the alpha matte. Since the standard practice in matting involves extracting the foreground by multiplying the original image with the predicted alpha matte, the resulting foreground inevitably retains some background color. For example, in the right part of Fig. [6] the shape of the background mountains remains faintly visible along the edges of the extracted bubble, albeit slightly, and becomes even less discernible when composited onto a new background. We consider this an open problem in the field matting worth future exploration: specifically, how to extract clean foreground based the alpha matte.
F
LLM USAGE
LLMs are used for refining English usage, and all content is reviewed by authors.
19

---

Published as a conference paper at ICLR 2026
Video Frimess & Prompt
Predicted Alpha Matic
Extracted Foreground
Vidco Frames & Prompt
Predicted Alpha Matte
Extracted Foroground
Video Frames Prompt
Predicted Alpha Matte
Extracted Foreground
Video Frames & Prompt
Predicted Alpha Matic
Extracted Foreground
Video Frames Prompt
Alpha Matte
Extracted Foreground
Figure 9; Visualization results of MAM2 on NOVM.
20

---

Published as a conference paper at ICLR 2026
Predicte Video Frrmmes
Alpha Mattes
Extracted Foreground
Predicted Video Frames
Alpha Mattes
Extractod Foreground
Video Frames
Predicted Alpha Matces
Extractod Foreground
Frames
Predicto Alpha Maties
Extractod
Foreground
Figure 10: Visualization results of MAM2 on YoutubeMatte.
21

---

Published as a conference paper at ICLR 2026
Video & Prompt
Predicted Alpha
Extracted Foreground
Video Frames
Prompt
Predicted Alphn Matte
Extracted Foreground
Video Frames & Prompt
Predicted Alpha Matie
Extracted Foregreund
Video Fromes & Prompt
Predicted Alpha Matte
Extracted Foreground
Figure 11: Visualization results of MAM2 on real-world videos.
22

---

Published as a conference paper at ICLR 2026
Video Frames
Prompt
Predicted Alpha Matte
Extracted Foroground
Video Frames
Prompt
Predicted Alpha Matte
Extracted Foreground
Figure 12: Visualization results of MAM2 on real-world videos.
23

---

Published as a conference paper at ICLR 2026
FTP-VM
Video
Frames
MatAnyone
MAM2 (Our))
Video Frames
FIP-VM
MalAnyone
MAME
(Oars)
FIR-VM
Videon
MAM2 (Ours)
MatAnyone
Figure 13: Visualization comparison with other methods.
24

---

Published as a conference paper at ICLR 2026
YEEls
Image & Prompt
Predicted Alpha Matte
Extracted Foreground
Image & Prompt
Predicted Alpha Matte
Extracted Foreground
Figure 14: Visualization results of MAM2 on image matting.
25

---

Published as a conference paper at ICLR 2026
Change Background
Exmotedd
Foreground
New DG
New BG
Trajectory Control
T T
T
Exrrated Foreground
New Tnjeccory
Object Scaling
Extracced Foroground
New Sive Position
Figure 15: Simple matting- based video editing effects.
Video Frames
Predicted Alpha Motte
Extructed Foreground
Figure 16: Visualization results of limitations.
26


Reviews:
[{"review_id": "RhvqXXkBDL", "reviewer": "Reviewer_Fzs3", "summary": "This paper tackles the problem of generalized video matting beyond human-centric domains. \nIt introduces Matting Anything 2, a robust model capable of handling diverse objects, including transparent ones, with flexible user prompts such as points, boxes, or masks. \nThe authors propose a Promptable Dual-mode Decoder (PDD) that jointly predicts segmentation masks and trimaps to enhance matting quality and generalization. \nTo address temporal instability for transparent objects, a Memory-Separable Siamese (MSS) mechanism is designed.\nExtensive experiments on diverse exiting benchmark and the newly proposed Natural Video Matting (NVM) dataset demonstrate that MAM2 achieves state-of-the-art accuracy and strong generalization to diverse, real-world scenes.", "questions": "Minor issues include a few typos (e.g., “iamge” in line 406) and missing citations (e.g., MEMatte in line 203 is mentioned without a proper reference). A careful proofread and citation check would improve the paper’s overall quality.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is overall easy to follow and understand. It provides a clear motivation and explains key ideas effectively with well-designed supporting figures (e.g., Figure 3 and 4).\n\n2. The ablation studies are comprehensive and convincingly demonstrate the effectiveness of each proposed component.\n\n3. The authors conduct extensive evaluations across diverse tasks (image and video matting) and environments, which strongly support the generality and robustness of the proposed model.", "weaknesses": "1. Although the model shows superior performance over baselines across multiple tasks, much of the improvement may stem from leveraging the powerful foundation model SAM2. The proposed method benefits greatly from SAM2’s strong generalization and semantic understanding, whereas most baselines are not based on such advanced foundation models. Therefore, a more in-depth comparison with other SAM1/SAM2-based matting models is essential. For the image matting task, paper [A], which also utilizes SAM, would be a particularly relevant and strong comparison.\n\n2. The Natural Video Matting (NVM) dataset is presented as one of the main contributions, but its description lacks sufficient detail. While brief statistics (in Table 1) and a few visual examples (in the supplementary material) are provided, the paper should offer a more detailed breakdown of the dataset composition—such as domain categories and their relative proportions—especially since it emphasizes dataset diversity as a key feature.\n\n3. Since the primary application of video matting lies in video editing, it would strengthen the paper if the authors demonstrated editing results using the generated alpha mattes, rather than only presenting matte outputs.\n\n[A] ZIM: Zero-Shot Image Matting for Anything, ICCV 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yKmsiaw4VG", "reviewer": "Reviewer_Qe9K", "summary": "This paper presents Matting Anything 2, a versatile video matting model designed to overcome the domain-specificity (e.g., human-centric) and restrictive first-frame mask requirements of existing methods. The core technical contributions are twofold. First, a Promptable Dual-mode Decoder (PDD) that jointly predicts segmentation masks and high-quality trimaps, leveraging trimap-based guidance for generalization. Second, a Memory-Separable Siamese (MSS) mechanism that recurrently isolates trimap prediction from interfering mask memory, crucially improving temporal consistency for challenging transparent objects. To validate these contributions, the authors introduce the new, diverse Natural Video Matting (NVM) dataset. Experiments demonstrate that MAM2 significantly outperforms state-of-the-art methods on both diverse natural scenes and human portraits, accepting flexible prompts like points or boxes.", "questions": "The paper presents an extension of SAM 2 to the matting domain, and the quantitative and qualitative results shown are excellent. My primary question, however, concerns the validation scope. All experiments were conducted on synthetic (composited) videos. This raises a question about the method's true capability to video matting \"anything\". To fully substantiate the paper's strong claims, I recommend that the authors provide qualitative results (and quantitative results if possible) on real-world video clips, such as those sourced from YouTube, to demonstrate the model's robustness to non-synthetic artifacts.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper demonstrates compelling quantitative and qualitative results, significantly outperforming previous state-of-the-art methods.\n2. The paper well extends SAM2's promptable, generalist architecture to handle the distinct and more complex task of alpha matting. \n3. The paper is well-written, clearly organized, and easy to follow.", "weaknesses": "1. The name Natural Video Matting is confusing. In matting literature, \"natural\" typically implies real-world, non-composited videos. Since NVM is synthetic (composited from assets), this name is a misnomer and should be revised to avoid ambiguity.\n2. All experiments are conducted exclusively on synthetic (composited) videos. This leaves a significant gap in evaluation, as performance on real-world videos that contain artifacts like complex lighting, sensor noise, and motion blur remains unproven. The matting \"anything\" claim is therefore not fully substantiated.\n3. The paper lacks a dedicated limitations. There is no discussion of potential failure cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qT5Es3K7c7", "reviewer": "Reviewer_Uqg2", "summary": "This paper proposes a generic video matting algorithm for different objects including person, animal, fire, water etc. It proposes a two-branch network to conduct the segmentation and tri-map prediction tasks, respectively. It also proposes a new dataset named Natural Video Matting covering different object categories.\n\nThe generic video matting task is challenging, and the authors take an initial step towards it.", "questions": "see above", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "-\tThis work deals with general objects for video matting, which is more advanced than existing work mainly dealing with humans.\n-\tAccording to table 2, the proposed method (matting anything2) outperforms existing methods on the proposed new dataset and a human matting dataset.", "weaknesses": "-\tThe model architecture is largely Segment Anything 2 (SAM2) with some additional components. The box and point prompt capability are directly from SAM2. The novelty of the proposed framework is limited.\n-\tThe evaluation dataset is small for general object categories. It only contains 50 clips.\n-\tIn supplementary section A, it shows the proposed method and existing method uses different datasets for training the model. The proposed method used more image matting data for training, which may pose an unfair comparison with existing methods. Can authors provide a variant that is trained with same data (like what Matanyone used) to give a fair comparison?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PQxlvdb27C", "reviewer": "Reviewer_TLqf", "summary": "This work introduces Matting Anything 2 known as MAM2, a prompt driven video matting framework that moves beyond human portrait centric settings by supporting points, boxes, and masks, by stabilizing trimap prediction for transparent and complex objects, and by reducing reliance on a first frame mask. The method adds a Promptable Dual-mode Decoder that predicts a segmentation mask and a trimap in one pass, and a Memory Separable Siamese mechanism that stabilizes trimap decoding for transparent or complex objects across time. The authors also introduce a Natural Video Matting benchmark with diverse non human portrait categories. Results report strong accuracy on both natural objects and human videos, and competitive image matting as well.", "questions": "1. Figure 5 appears to report results only on the Natural Video Matting dataset. Could you add a companion figure that compares MAM2 with baselines on additional public video matting test sets?\n\n2. Table 2 shows MAM2 performance with box and point prompts. In Table 4 these entries appear in faint text, which suggests they may be out of scope or not directly comparable. Could you create a separate table summarizing results for models that use both prompts box and point under the same data and evaluation protocol, reporting MAD and GRAD, so the two prompt setting can be compared fairly with single prompt settings?\n\n3. Could you revise Figure 4 to also display the matted image together with the trimap for the same frames and prompts so readers can directly see how trimap quality translates into the visual result?\n\n4. There are some typographical errors. For example, Figure 7,8,9 caption contains “visualizaiotn” instead of “visualization”. Could you proofread the paper and update all figure texts and captions, ensuring correct spelling, consistent capitalization?\n\n5. Section 3.4 is difficult to read and follow. Could you provide a clearer rewrite during rebuttal?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "1. Clear problem motivation regarding portrait bias and reliance on a first frame mask for selection. The paper explains why transparent targets such as smoke or fire make mask prompting difficult and why lighter prompts like box or points are preferable for such cases.\n\n2. Architectural idea with practical value. PDD extends the SAM2 mask decoder to produce both a mask and a trimap in one pass, and it leverages the strong mask quality of SAM2 as a spatial prior. In practice this yields cleaner boundaries and a more stable unknown band, which improves the final alpha matte.\n\n3. Transparent object failure analysis and remedy. The paper identifies temporal collapse where unknown regions drift to foreground for later frames. MSS addresses this by running a second PDD pass to decode the trimap from memory free features using the first pass mask as a pseudo prompt. Parameters are shared between the passes.\n\n4. New test dataset (Natural Video Matting) for generalization.", "weaknesses": "1. The mathematical specification of the pipeline is insufficient.\nThe paper lacks a complete equation level description of the forward computations, especially for the interaction between PDD and MSS.\n\n2. Figures do not explain the full system behavior. Figure 2 leaves ambiguity about how segmentation data and matting data are used across training iterations and stages. It is unclear how features from the MSS pathway connect to the PDD pathway, which parts are trainable at each stage, and whether user prompts for MSS and PDD are shared or distinct. The figure should be redrawn to show the end to end pipeline with data sources, feature flow, prompt flow, and trainable versus frozen modules.\n\n3. The boundary between the SAM2 decoder and the authors’ contributions is unclear.\nThe text and figures do not make it evident what is inherited from the original SAM2 mask decoder and what is newly introduced in this work. The paper should provide a module level figure that clearly labels inherited blocks and newly added blocks, together with explicit annotation of the mask output token, the trimap output token, and all entry points for user prompts.\n\n4. Why the method works is not analyzed.\nThe paper states that the proposed method fixes the failure cases but gives little analysis of the mechanism. In particular, the paper should explain why the Mask Augment Feature and the preserved Feature without Memory lead to stable trimap prediction, with diagnostic evidence or ablations.\n\n5. Experimental fairness and data accounting are not sufficiently documented.\nAccording to Appendix 1, models appear to be trained on different data sources. The manuscript should quantify the training data for each method in comparable units such as number of images, clips, and frames, and include experiments where all methods are trained on the same data to isolate the effect of the proposed design. Table 8 also indicates a parameter gap of nearly nine times between MAM2 and the strongest baseline. This capacity difference makes it difficult to attribute the gains in MAD and GRAD to the proposed architecture rather than to model size.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)