SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation
Authors: Zhenjie Mao
Yuhuan Yang
Chaofan Ma
Dongsheng Jiang
Jiangchao Yao
Ya Zhang
Yanfeng Wang
Venue: NeurIPS 2025 poster ()
Keywords: Referring Image Segmentation
Referential Ambiguity
Mamba
Multi-Modal Understanding
Primary area: applications

Abstract: Referring Image Segmentation (RIS) aims to segment the target object in an image given a natural language expression. While recent methods leverage pre-trained vision backbones and more training corpus to achieve impressive results, they predominantly focus on simple expressions—short, clear noun phrases like “red car” or “left girl”. This simplification often reduces RIS to a key word/concept matching problem, limiting the model’s ability to handle referential ambiguity in expressions. In this work, we identify two challenging real-world scenarios: object-distracting expressions, which involve multiple entities with contextual cues, and category-implicit expressions, where the object class is not explicitly stated. To address the challenges, we propose a novel framework, SaFiRe, which mimics the human two-phase cognitive process—first forming a global understanding, then refining it through detail-oriented inspection. This is naturally supported by Mamba’s scan-then-update property, which aligns with our phased design and enables efficient multi-cycle refinement with linear complexity. We further introduce aRefCOCO, a new benchmark designed to evaluate RIS models under ambiguous referring expressions. Extensive experiments on both standard and proposed datasets demonstrate the superiority of SaFiRe over state-of-the-art baselines. Project page: https://zhenjiemao.github.io/SaFiRe/.

SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation
Zhenjie Mao!
Yuhuan Yang' Chaofan Ma
Dongsheng Jiang'
Jiangchao Yao't Ya Zhang?31 Yanfeng Wang
CCooperative Medianel Innovation Center, Shanghai Jiao Tong University
Schooo of Artificial Intelligence, Shanghai Jiao Tong University
Institute of Artificial Intelligence Medicine, Shanghai Jiao Tong University School of Medicine Huawei Inc.
(zhenjmao, yangyuhuan, chaofanma, sunarker, ya_zhang, wangyanfeng6221 Ga edu, cn
Abstract
Referring Image Segmentation (RIS) aims segment the target object in image given a natural language expression. While recent methods leverage pre-trained vision backbones and more training corpus achieve impressive results, they predominantly focus on simple expressions-short, clear noun phrases like "red car" or "left girl". This simplification often reduces RIS key word/concept matching problem, limiting the model's ability to handle referential ambiguity in expressions. this work, we identify two challenging real-world scenarios: object-distracte expressions, which involve multiple entities with contextual cues, and category-implis implicit expressions, where the object class is not explicitly stated. To address the challenges, we propose novel framework, SaFiRe, which mimics the human wwo-phase cognitive process- first forming global understanding. then refining it through detail-oriented inspection. This is naturally supported by Mamba's cean-then-update property, which aligns with our phased design and enables efficient multi-cycle refinement with linear complexity. We further in- troduce aRefCOCO, new benchmark designed to evaluate RIS models under ambiguous referring expressions. Extensive experiments both standard and pro- posed datasets demonstrate the superiority of SaFiRe over kate-offthe--a baselines. Project page: attps://zhenjiet github.... io/SaFiRe/
Introduction
Referring Image Segmentation (RIS), aiming to segment the target object in an image based on the textual description, is fundamental task in multi-moda understanding. Most studies solve the RIS task by designing image-text alignment modules to bridge the modality gap [ With the success of image foundation models [416], recent works further improved the performance of RIS by leveraging the pre-trained image features 7 and introducing more training corpus (10+2).
Despite significant process, current RIS methods primarily focus on narrow scenario, where most of the referring expressions consist of a single, simple noun phrase that clearly identifies the target object, e.g., "left girl" [13, "red car" [13]. etc. We summarize this the simple expression pattern. Under this paradigm, the RIS task is often over-simplified and modeled as a "key word/concept matching problem".. The model only needs extract the key words or concepts from the given text, then matches them with the image patches. Most existing RIS methods adopt this simplified
Conresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Simple Expression
Object-distracting Expression
-ategory-implieit Expression
Image & GT
"Compured to the blue-shirt man, he closer to the two giraffes."
"Taller man."
"He is the taller one.
One Object, Divergent Attention. Attention maps under three types of referring expressions, all targeting the same object. (a) Simple expression yields accurate focus. (b) bbjtattdistracttnng expression misguides attention to irrelevant regions. (c) Category- implicit expression leads to dispersed attention. This highlights the challenge of referential ambiguity for "key word/concept matching" method and motivates our saccade-fixation framework.
Figure I: Referential Ambiguity:
formulation. For example, LAVT 7 and ReLA 14 treat RIS as a process of matching individual words to specific pixels, matching keywords for each image location and segmenting the regions most relevant. VLT [15] introduces multiple queries to focus on various word pairs, and it reduces the rich semantics of language to a set of discrete choices. Similarly, MagNet [66] tries to emphasize key words by masking several input textual tokens and matching them with visual objects.
Although these methods demonstrate reasonable performance to some extent in this simple expression scenario, we notice that, in real-world applications, referring expressions often exhibit referential ambiguity, where the mapping between the text and the target object is indirect, underspecified, or contextually entangled. We summarize this into two challenging cases. (1) One is object-distracting expression, where the referring sentence contains multiple noun phrases but only one is the true referent, e.g., "compared to the blue-shirt man, he is closer to the two giraffes". This introduces misleading contextual cues that can divert attention away from the actual target, increasing the difficulty of accurate segmentation. (2) The other is pategory-implice expression, where the target object lacks an explicit categorical attribute, e.g., "he the taller one". This common in spoken language, where pronouns and comparative or other distinct adjectives are frequently used without specifying object categories, making difficult to determine the referent based solely on class-level semantics. As shown in Fig. [1] we visualize the attention maps under three types of expressions that refer to the same object. While the model successfully highlights the target under the simple expression (a), its focus becomes diffuse and uncertain under the object-distractin (b) and category- implicit (c) expressions. The referring expression these cases is not ideally with a single object or clear category, resulting in limited comprehension and weak visual-textual interaction for the existing methods that rely on key word/concept matching.
To address this issue, we investigate how humans interpret the correspondence between complex textual descriptions and visual representations, and how they identify objects within images. Ac- cording to the research in cognitive psychology [17] [8]. humans engage in a two-phase process. The first phase, which can be summarized as global semantic understanding, involves an initial overview of the sentence and general scan of the image. The second phase, referred to as cross- modal refinement, occurs when the individual focuses on finer details, re-examining the sentence and systematically inspecting each region of the image to extract relevant information. This dual-phase process underscores how humans effectively integrate textual and visual information.
Inspired by this, we propose our network SaFiRe with two alternating operations: Saccade and Fixation, simulating the two human cognition phase respectively. This formulation aligns well with state-space sequence modeling, and we adopt Mamba as the underlying architecture to support our framework in a cognitively inspired and structurally efficient manner. Specifically, the Saccade operation performs rapid "glimpse" of both the text and image. By modulating the image features with the general meaning the text, this operation establishes rough correspondence between the modalities after SSM scanning. While the Fixation operation aims to refine this coarse result by carefully examining the whole image region by region, and continuously reconfirms the text during this process. To this end, this operation alternates cach image region with reiterated text in

---

unified sequence for SSM scanning, enabling Mamba to focus locally while maintaining global context-natien natural-ntuuraly leveraging its order-sensitive and memory design. Furthermore, with the reiteration of the two-phase process, the model is able to progressively refine its cross-modal understanding and accurately identify the target referent.
To sum up, our contributions lie in four folds:
We highlight the limitation of recent RIS methods, identifying their tendency to oversimplify RIS as a key word/concept matching problem, which often fails when confronted with referential ambiguity.
We propose SaFiRe, novel RIS framework inspired by cognitive psychology, which simulates the human cognitive process through alternating two inspection phases to achieve coarse-to to-fine alignment and comprehensive cross-modal understanding.
We design two operations- Saccade and Fixation- Saccade establishes initial semantic correspondence through fast global scanning, and Fixation enables fine-grained region-wise inspection guided by reiterated textual cues for more accurate segmentation.
To evaluate the effectiveness of our approach, we not only conduct experiments on the traditional test-only benchmark, aRefCOCO, which
RefCOCO family of datasets but also introduce features ambiguous expressions. Extensive experiments and ablation studies demonstrate the superior performance of our method over the state-of-the methods.
2 Related Work
Referring Image Segmentation (RIS) is a challenging task that requires identifying and segmenting specific objects in images based natural language expressions. Early methods mainly focus on attention-based vision-language fusion strategies. For example, VLT (15) introduced a cross- attention-based framework with query generation. while LAVT (7) demonstrated that encoder- level cross-modal fusion significantly enhances vision-language alignment. ReLA [14] advanced the field by incorporating fine-grained cross-region modality interactions. Later methods such as CGFormer [1] and MagNet [16] enhance segmentation accuracy by integrating object-level priors and mask supervision. The advent of Large Language Models (LLMs) has also spurred the development of LLM-based RIS models [1). leveraging their powerful multi-modal reasoning abilities to better address the RIS challenges. While existing methods perform well on simple expressions with clear object references, they often treat RIS as key word/concept matching problem, which limits their ability to handle referential ambiguity.
State Space Models and Multi-Modal Extensions. Originally developed for dynamical systems, state-space models (SSMs) have proven effective in deep learning for modeling long-range de- pendencies. S4 [22] demonstrated their scalability, and Mamba (23 further improved efficiency and performance through a novel selection mechanism. Mamba has achieved strong results across vision 66 [4, 25] and language (26) tasks. Recently, its use in multi-modal learning [27/31] has gained interest, mainly structurally aligned settings. However, current approaches offer limited cross-modal alignment. For instance, Coupled Mamba [32] uses shallow fusion strategies, while ReMamber focuses on encoder-level fusion but oversimplifies local interaction as a keyword- matching problem. To address these gaps, we propose framework that leverages Mamba's 's sequential processing to emulate the human-like two-phase grounding process.
(
Method
3.1
Overview
The standard definition of RIS task is to segment the target object based on the input textual description. Given image I and textual description T, we aim to predict binary mask M that indicates the referring target in the image. The overall framework of our proposed can be written as:
SaFiRe (I, T) M.
For encoders, we use Swin-Transformer 4 and BERT respectively,
33
to extract image and text features
Fdd (I) MRDDeHHKE HxWxC F = (T) E R
(2)
3

---

Festuee Enoding
Saccade Saccade-Fixttion
Mask
SFLayer
Layer
Head
"Cunpared to ether Ixbiyy bears It Raaepoors more centered."
SaFiRe overviow
Sacoade operation
Fixation operation
Telover
L.C.
F
Tindswy
Text
F
M.P
Tost
Wegilted Avrtage
2
Tend
Tind say
SSM Scanning
graug
SSM Scanning
F
F
Toxt WXX
3
2
Text soc
Fent seg
Figure 2: Overview the SaFiRe Architecture. For each SFLayer, consists of Saccade oper- ation and Fixation operation. The Saccade operation corresponds to the phase of global semantic understanding. enables the model to rapidly sean both visual and textual inputs, establishing coarse-level alignment between the two modalities. The Fixation operation mirrors the cross- rrosl refinement phase. allows the model to attend to specific local visual regions while re-cxamining the textual input, facilitating the extraction of fine-grained, task-relevant information.
Then, we decode the image and text features through a series of our carefully designed SFLayers:
F++1 F++1. F
4 (2)
SFLayer (F F)).
i (,,,,,,3).
Finally we predict mask with processed image features:
M
2 musk (F). F2 F F.).
(4)
SFLayer Design. Our SaFiRe framework simulates the two-phase human cognitive process by applying specialized module called the Saccade-Fixation Layer (SFLayer). Fach SFLayer consists of two sequential operations: Saccade and Fixation. The Saccade operation corresponds to the phase of global semantic understanding. It enables the model rapidly scan both visual and textual inputs, establishing a coarse-level alignment between the two modalities. In contrast, the Fixation operation mirrors the cross-modal refinement phase. allows the model to attend specific local visual regions while re-examining the textual input, facilitating the extraction of fine-grained, task-relevant information. Formally, Eq. can be expanded in detail as:
F Fé+1
4 (6)
ayll (F F()
Prixation Saccade F())
By reiteratively alternating between these two operations across multiple layers, the model pro- gressively enhances its multi-modal understanding. This saccade-fixatios reiteration across layers serves the core mechanism for accurate target localization. For clearer view of this process, the architectural details of our framework are shown in Fig. [ and we detail the design of Saccade and Fixation operations in Sec. 3.2 and Sec. 3.3 respectively.
3.2 Saccade: Quick Glimpse of Image-Text Correspondence
Our Saccade operation aims to take a quick glimpse of both text and image information, and establish rough correspondence between them. This step is designed to mimic the initial phase of human referential understanding- -namee a coarse and global semantic aligument between modalities before any fine-grained reasoning occurs. We implement this by modulating image features with global textual semantics via a lightweight yet effective operation. Concretely, we adopt a DiT- inspired (4 Norm Adaptation, the pooled textual representation adjusts the statistics of image features, contrast to class-based conditioning in the original design.

---

To be more specific, the Norm Adaptation operation pools a global vector from textual sequence and adapts the image's 's scale and bias globally according to this vector. In our practice, with image feature Fo E Hx XC and text feature F, E TR IR, we apply average pooling and linear projection on text side to generate global scale, bias and shift factors:
[a, 9. r) =
E R
The image feature is then adapted with the three factors:
F² 7 VSSM(a F,, + 3) + F, E HR
where V-SM(-)) denotes the visual SSM feed-forward layer in VMamba [6.
This global adapting operation performs quick and general interaction between the two modalities, making the image features more prepared for the subsequent Fixation operation.
3.3 Fixation: Detailed Examination of Cross-Modal Alignment
For Fixation operation, we aim to foster image-text communication in a more delicate manner, by checking the image region by region according to the text description to determine the target more accurately. We find that the SSM mechanism is especially suitable for this. The hidden states of SSM updated by previous input and its scanning mechanism is vivid simulation of human behavior: it focuses more on the recent input sequence while keeping long-term global memory in mind.
Thus, we utilize the most recent VSSM architecture VMamba [6. and came up with a novel Fixation operation on top of it. The Fixation operation follows a "group-sean-recover
pattern, where first (1) groups the image sequence into non-overlapping local regions via window splitting, and then (2) scans each region and the text alternately with SSM, effectively modeling fine-grained cross-modal interactions within a unified multi-modal sequence, finally (3) recovers the processed unified multi-modal sequence back into separate image and text representations for the next operation. We here detail each of them in order.
Group. We aim to divide image sequence into several regions while maintaining their local structural priors. We draw inspiration from Swin [ and use similar split mechanism with several non- overlapping windows. To be specific, with input image sequence F. Hx and window size w, split the sequence into P HW//" windows:
F F ...... FF F E RR2xC.
(8)
As SSM itself offers global reception field, do not apply sliding window mechanism in Swin. Scan. We aim to perform local and fine-grained scanning while foster cross-modal interaction. There is one key property under SSM mechanism: more recent tokens will be more relevant to the current hidden state than the previous ones. other words, for SSM the input order matters. We make full utilization of this property and reiterate the text sequence multiple times within each image region.
To be specific, with input image feature F. E Hx xC and text feature F, E L we re-arrange a hybrid multi-moda sequence F hibrid in the following manner:
F Kyybrid = [L W0
F,, F2 au Ft, F.P F.J E R(HW+PL)xC
Then, we use an VSSM layer to process this hybrid sequence:
F Faybrid = VSSM( hiybria) = [ F!
F2'
w F2
FP F.P..
(10)
Due to the re-arrangment of hybrid sequence and the built-in SSM mechanism, the model will always be focused on the query target with the help of periodically reiterated text tokens. This encourages the model to focus on the description as a coherent and complete whole, preserving its continuity and full meaning, rather than discretizing it into isolated key words/concepts for global matching.
This separates the unified multi-modal sequence back into distinct image and text features. preparing them for the next layer. For image features we re-arrange them, and for text features, we apply weighted average across all repeated sentences as updated representation. The output feature can be expressed as:
Recover.
Fou
F
F w
ER XX
pout
Σ o=1
F.F Fd' Px
(11)
5

---

Here w [wi,-... up] E Rp is a learnable average weight. F and F? are the output features and then passed to the next SFLayer for further processing.
The Fixation operation achieves the goal of delicate multi-modal understanding by fully utilizing the underlying mechanism of SSM. By repeating the text multiple times within the image sequence, the hidden state space is periodically refreshed, forcing the model to continuously memorize and reinforce its segmentation target. With the help of SSM mechanism, model is able be more focused on the current area while ensures the global receptive field for segmentation.
Discussion: Why Mamba Fits Our Framework. Our framework focuses simulating the se- quential nature human cognition process alterating between global understanding and localized refinement. This requires model that processes information by-ste... Mamba's 's state-space scanning aligns perfectly with this paradigm, as processes sequences in a directional, recurrent manner. Besides, its linear complexity also allows efficient, dense region-text interactions, supporting fine-grained reasoning without the overhead of attention. These advantages make Mamba a struc- turally appropriate and computationally efficient fit for our group-sean-recover Fixation operation and the overall Saccade-Fixation Reiteration framework.
3.4 Segmentation Head
For final mask prediction, combine the multi-level visual output [F F. in a top-down manner, following LAVT [7], as characterized by E). The training process employs combination of Dice loss 335 and Focal loss (36] the composite loss function, as follows:
F*) of each SFLayer
= a Dice + (1 a) Lroaal,
(12)
where the weighting coefficient a is set to 0.5 by default.
3.5 Complexity Analysis
Mamba-based architectures are known for their linear complexity in long sequences processing. However, since we introduce multiple times of text repeating Fixation module, the overall sequence length will be slightly increased. Here we take a concise analysis of the computational complexity our Fixation module. Formally, with image sequence length HW, text sequence length L and local window size w, the complexity is defined by the overall sequence length:
Complexity = O(HW + HW/w² L) = (1 + I./w²) O(HW).
(13)
In practice, we set window size w to 4, and average text length L to around 15. The overall complexity is thus increased by constant factor of 0.9, which is nearly negligible.
4
Experiments
4.1 Datasets and Metrics
Traditional Benchmarks. Following prior works 7 16 we systematically assess model perfor- mance on the widely used RefCOCO benchmarks: RefCOCO 13, ReiCOCO+ [I3] and RefCOCOg 37 These datasets are all grounded the MSCOCO B8 visual corpus but differ significantly in their linguistic properties. RefCOCO and RefCOCO+ feature concise referring expressions, aver- aging 3.6 words per phrase. Notably, RefCOCO+ omits absolute spatial terms (e.g.. "left/right" or ordinal indicators), thereby introducing additional difficulty. In contrast, RefCOCOg features more descriptive and elaborate language, with expressions averaging 8.4 words and incorporating more references to locations and appearances, making it the most linguistically complex and challenging benchmark among the three.
aRefCOCO. We introduce aRefCOCO (ambiguous RefCOCO), test-only benchmark constructed by reannotating subset of images from the RefCOCo L and RefCOCOg (37 test splits with more challenging referring expressions. It specifically designed to evaluate model performance on object- distracting and category-implicit scenarios two aspects that remain difficult for existing models. The referring expressions in aRefCOCO are significantly more complex, averaging 12.6 words per sentence. To increase the difficulty of category-implici references, the benchmark includes a higher frequency of pronouns (e.g.. "it", "he", "she"), which obscure explicit category cues and demand deeper contextual understanding. To better support object-distracting evaluation, expressions exhibit more frequent use of prepositions (c.g., "and", "to", "of"), introducing richer relational structures

---

and more noun phrases that increase the likelihood of semantic confusion. Visually, aRefCOCO images contain 3.1 category distractors per image on average- 87.5% more than RefCOCOg's 1.6-requiing models to resolve denser visual ambiguities for accurate grounding. In terms of scale, aRefCOCO introduces 7,050 text-image pairs generated by Qwen-VI.-Man [39] and meticulously filtered-a a 40% expansion over RefCOCOg's & test split (5,023 pairs) eeieeeisstblliiinngggg a benchmark that challenges existing RIS models. For more details of aRefCOCO, see Appendix
Metrics. We adopt oloU as the main metric and further incorporate mloU and Precision X (Xe {50, 70, 90]) for a more comprehensive evaluation, where Precision @ X means the percentage of test images with an IoU score higher than X%.
4.2 Implementation Details
Our model employs the Swin Transformer-B 4 and BERT-B [3 the vision and language encoders, respectively, following prior work [7 Swin-B is initialized with ImageNet pre- pre-trained weights, and BERT-B B uses the official checkpoints. For SFlayers, of the VSSM from VMamba [6] and set the window size to 4 x 4 in the Fixation operation. The model is trained and-to-end for 50 epochs using the Adam W optimizer with learning rate of 5e-5 and weight decay of le-4, along with a cosine learning rate scheduler. All experiments are conducted on 2 NVIDIA A100 GPUs.
adopt the official implementation
4.3 Comparison with the State-of-the-Arts
Table I: Comparison of RIS Methods Trained on Single Datasets across RefCOCO, RefCOCO+, and RefCOCOg with oloU. indicates UMD partition of RefCOCOg. The best performances are in bold.
Encoders
RefCOCOg (hard)
RefCOCO+ (medium)
RefCOCO (easy)
Methods
Visual
Textual
val(U)
lest(U)
val
testA
lestB
testA
lestB
CRIS 40 VIT 15
CLIP Rio:
CLIP
56.56
57.38
$7.94
64.05
48.42
67.35
71.54
62.16
Swin-B
BERT
63.49
66.22
63.53
68.43
56.92
72.96
75.96
69.60
LAVT
Swin-B
BERT
61.24
62.09
62.14
68.38
55.10
72.73
75.82
68.79
BKINeI
CLIP RM
CLIP
64.21
63.77
64.91
69.88
$3.39
73.22
76.43
69.42
ReLA 14
Swin-B
BERT
65.00
65.97
66.04
71.02
57.65
73.92
76.48
70.18
SIVT
3
SegNeXt-B
BERT
62.75
63.57
64.07
69.28
$6.14
74.02
76.91
70.62
SADLR 4
Swin-B
BERT
63.60
63,56
62.48
69.09
$5.19
74.24
76.25
70.06
DMMI 3
Swin-B
BERT
63.46
64.19
63.98
69.73
$7.03
74.13
77.13
70.16
CGFormer
11
Swin-B
BERT
64.68
64.09
64.54
71.00
$7.14
74.75
77.30
70.64
RISCLIP
44
CLIP-B
CLIP-B
64.10
65.09
65.53
70.61
55.49
73.57
76.46
69.76
MagNel 16
Swin-B
BERT
65.36
66.03
66.16
71.32
58.14
75.24
78.24
71.05
LQMFormer 12
Swin-B
BERT
64.73
66.04
65.91
71.84
57.59
74.16
76.82
71.04
ReMamher
Mamha-B
CLIP
63.90
64.00
65.00
70.78
57.53
74.54
76.74
70.89
SaFiRe (Ours)
Swin-B
BERT
67.07
66.87
66.39
71.96
58.47
75.34
78.52
71.35
Traditional Benchmarks. Tab. presents comparison of our SaFiRe with state-of-the-an methods on the RefCOCo, ReiCOCO+, and RefCOCOg datasets with oloU metric, where all models are trained independently on single datasets. Results show that SaFiRe outperforms all other methods across all datasets. In particular, SaFiRe achieved a great improvement on RefCOCOg, where the task involves more complex and fine-grained textual descriptions compared to RefCOCO.
In addition, Tab. 2 compares SaFiRe with recent state-of-the -f-the---tth-eee- art methods trained under mixed-date configuration. The precise dataset mixtures differ across methods. To avoid conflating performance with training-corpus choices, we report each baseline under its official training configuration. In general, these models draw on subsets and combinations of widely used referring-expresseeee and
segmentation datasets- -ucch as RefCOCO [3]. RefCOCO+ [3). RefCOCOg [37]. COCO (38),
Object365 [54], ADE20K [55], COCO-Stuff [56] PACO-LVIS 57 PASCAL-Part (58), GranD 20
VOC2010 [59]. MUSE (53), gReCCOCO 14), and
[52]. Our training corpus follows mixed-data setup based on the RefCOCO series and COCO-Stuff, which is comparatively smaller. However, the results show that SaFiRe achieves competitive results on all three benchmarks, particularly excelling on the challenging RefCOCOg and RefCOCO+ benchmarks. Notably, it surpasses several SAM-enabled models (e.g., u-LLaVA, GSVA) across multiple splits.
aRefCOCO. We further conduct a zero-shol transfer evaluation of different models on the aRefCOCO. The expressions in aRefCOCO are notably ambiguous and more complex, frequently lacking explicit category indications and containing more distractors, which demands sophisticated mrosl
7

---

Table 2: Comparison of Model Performance Trained Mixed Datascts Three Benchmarks. U indicates UMD partition of RefCOCOg. The hest performances bold. (ft) denotes models further finetuned ReiCOCO/+/g after training. "Textual" and "Visual" refer to the textual encoder and visual backbone. respectively. "LLM" denotes the large language model used as the textual reasoner.
ReiCOCOg (hand)
ReiCOCO+ (mediuni)
Re/COCO (easy)
Methods
Textual / /LL.M
Visual
val(UU)
lesi(U)
lestA
EEVG 45
BERT
Swin-B
71.5
71.9
71.4
75.6
64.6
77.5
79.6
75.3
PrompiRIS
CLIP
SAM
69.2
70.5
71.1
76.6
64.3
78.1
81.2
74.6
OneRef.B
47
BRTT3-B
BEIT3-B
74.1
74.9
74.7
77.9
69.6
79.8
81.9
77.0
C3VG
BETT3-B
BETT3
76.3
77.1
77.1
79.6
72.4
81.4
82.9
79.1
SaFiRe (Ours)
BERT
Swin-B
77.3
78.1
80.5
72.9
82.1
83.7
MagNet [16
BERT
Swin-B
67.8
69.3
68.1
73.6
61.8
76.6
78.3
72.2
PolyFonmer-L.
BERT
Swin I.
69.2
70.2
69.3
74.6
61.9
76.0
78.3
73.3
UNINEXT-L.
50
BERT
ConvNeXt-I.
73.4
73.7
70.0
74.9
62.6
80.3
82.6
77.8
LISA (ft)
19
LLaVA-7B
SAM
67.9
70.6
65.1
70.8
58.1
74.9
79.1
72.3
oloU
GLaMM
20
Vicuna-7B
SAM
74.2
74.9
72.6
78.7
64.6
79.5
83.2
76.9
u-LLaVA
5
Vicuna-7B
SAM
74.8
75.6
72.2
76.6
66.8
80.4
82.7
77.8
PSALM 52
Phi 1.5
Swin
71.0
72.3
68.1
70.7
64.4
78.0
78.1
76.6
GSVA (ft)
Vicuna-1 13B
SAM
74.2
75.6
67.4
71.5
60.9
78.2
80.4
74.2
PixcILM
53
Llama2-13B
CLIP
69.3
70.5
66.3
71.7
58.3
73.0
76.5
68.2
SaFiRe (Ours)
BERT
Swin-B
75.8
78.9
68.3
81.4
83.6
PSALM results are re-evaluated align with standard settings 19
which predicts masks image-text pair evaluation.
Table 3: Performance Comparison on aRefCOCO.
Methods
aRe(COCO
Table 4: Performance Comparison of Different Configurations.
oloUi
mloL:
Pr( 50
Pr6770
Pr699
LAVT 7
31.7
34,9
30.8
19.9
5.5
54.0
67.0
Configuration
RefCOCOg
RefCOCO
ReMamber
9
49.4
57.8
64.2
57.9
27.9
Saccade Fixation
oloU
mloU
aloU
mloU
MagNet
16
54.4
60.7
67.7
61.4
29.9
EEVG 45
50.7
66.9
62.4
29.4
67.07
69.68
75.34
77.02
C3VG
48
62.9
69.1
78.4
72.8
32.8
65.72
68 68.24
74.27
76.12
LISA(7B) 19
49.0
53.8
58.7
49.2
20.6
66.32
68.73
73.60
75.64
LISA++(7B) 660
45.3
50.4
51.5
43.5
19.7
64.48
67.72
72.92
75.17
PSALM
52
60.1
75.7
71.8
38.7
SaFiRe (Ours)
understanding for precise segmentation. For fair comparison, all models have NOT been trained on aReíCoCO. As shown in Tab. 3 our model achieves the best overall performance on aRefCOCO, demonstrating its strong ability to precisely localize referred objects under ambiguous language conditions.
4.4 Ablation Study
Module Ablation. Tab, summarizes the performance different module configurations within our model. The results show that our full model achieves the highest oloU scores on both RefCOCOg (descriptive sentences) and RefCOCO (concise phrases). Compared to independent module usage, the synergistic combination of Saccade and Fixation yields significant gains, highlighting their complementary effects. Removing both modules reduces the model a naive baseline that simply concatenates textual and visual sequences for VSSM scanning, which performs worst among all variants, underscoring the essential role of both Saccade and Fixation operations.
Fixation Window Size. In Fixation operation, the approach is related to sequence arrangement for SSM scanning, where the ordering of input tokens impacts performance. With representing individual image tokens and representing the full sentence feature tokens, we conduct an ablation
Table 5: Performance Comparison Table 6: Performance Compari- Table 7: Comparison Inference Efficiency.
of Different Fixation Window Sizes. son of Different Backbones.
Configuration
RefCOCOg
Backhone
Model
ReiCOCOg
Models
GFLOPs
FPS(inage/s)
Vanilla [I... T]
64.38
ViT-B
bascline
55.55
CGFormer
631
2.78
Repeat [I...I T...
65.50
ViT-B
SaFiKe
59.33 (+3.78)
PSALM 52)
LISA(TB) 19
4880
455
2.19
2.67
Fixate x
Fixate (ours)
67.07
66.49
Swin-B
SaFiRe
67.07
(J8A++(B)
Fixate 8 x
66.20
Swin-I.
SaFiRe
68.16 (+1.09)
SaFiRe (Ours)
384
8.26

---

Exprcssion:
Prediction
FV
P F2 FC
"This donut locuted the second column from the left the botton.
Full Model
w/o Fixation
Image
w/o Saccade
Ground Truth
Figure 3: Layer-Wise Visual Feature Maps. Left->right corresponds to shallow ddee layers. The full model shows balanced activation. Without Fixation, local detail is missing: without Saccade, global focus weakens. The differing activation patterns reflect their complementary roles.
ding green ski polos, learning ski
using
RaMMmmber
PSALMM
Ours
13T
Behlamberr
PSALM
Ours
Ft the carrot positioned most to the right among all the camots.
This teddy bear's Shiit has no text. contrasting with the teddy bear the right.
Figure 4: Visualization Results for aRefCOCO. Compared to the other two methods, our SaFiRe is more capable of comprehending ambiguous referring descriptions.
study of the following configurations in Tab. (1) Vanilla [I. T]: This configuration directly concatenates the image sequence II. and the text sequence T. (2) Repeat [I. [I...I T... .T]: In this configuration, the text sequence repeated multiple times at the end of the image sequence. (3) Fixation x x: This approach reiterates the text sequence once after every x2 image tokens window. For example, Fixation 2 x means the sequence looks like: [IIII IIII T ...... Tab. 5 shows that the Repeat configuration improves upon Vanilla, but the Fixation operation, particularly with a window size 4 x 4, shows a more significant enhancement in performance. This indicates that distributing textual information throughout the visual sequence helps maintain semantic continuity and guides the model to better align specific image regions with the text, rather than treating the two modalities isolated blocks with simple concatenation.
Backbone Ablation. Here we present a comparison of different backbones, as depicted in Table Tab. 6 While our method is not specifically designed for global attention backbones such as ViT-B, still brings notable improvement of +3.78. Furthermore, when applied to stronger window- based backbones like Swin-I., the performance continues to improve, reaching 68.16 16 on RefCOCOg. These results indicate that our method generalizes well across different backbone architectures and particularly benefits from more powerful feature extractors.
Layer-wise Feature Map Analysis. In Fig. 3 we visualize the output visual feature maps of each SaFiRe layer under different model configurations. A clear difference in activation patterns can be observed between the Saccade and Fixation operations. Specifically, the Saccade operation generates broader and more globally distributed activations that align with the overall shape and location described the text. In contrast, the Fixation operation produces sharper, more localized activations, focusing on fine-grained visual details. When used independently, each module exhibits limitations in fully capturing and grounding complex language expressions. However, when combined, their complementary activation behaviors allow the model to achieve accurate identification of the

---

described object. This further confirms that Saccade and Fixation work in synergy to enhance the model's mrosll understanding.
4.5 Inference Efficiency Comparison
We evaluate inference efficiency under identical input settings and report both computational cost (GFLOPs) and throughput (FPS). As shown in Tab. 7 SaFiRe compared against repre- sentative transformer-basee (CGFormer (II)) and LLM-driven models (PSALM [52] LISA [19], LISA++ (60). SaFiRe attains 8.26 FPS with 384 GFLOPs, yielding the highest throughput and the lowest compute among all compared methods. Together with the accuracy results in Tab. T Tab. and Tab. 3 these findings underscore the efficiency advantage of our framework.
4.6 Visualization Results
Fig. compares SaFiRe with two state-of-the RIS approaches MMmma-basse ReMamber [
and LLM-driven PSALM [22 oon the aRefCOCO benchmark. Our method exhibits a stronger
ability to handle referential ambiguity and avoids the confusion observed in the other two methods. For additional examples, see Appendix
4.7 Failure Case Analysis
GT
Prediction
GT
Prediction
Text: "He has short dark hair and hands are crossed in front of him."
Text: "the mug on the tray closer to the plate nearest the chef." (b) Unusually long relational chain
(a) Extremely similar visual semantics
Figure 5: Two Typical Types of Failure Cases.
To guide future works, analyze two typical failure scenarios. (1) Extremely similar visual semantics. Errors arise when multiple candidates share similar visual semantics but only one best fits the intended meaning. As in Fig. 5a both people look "cross-like" at first glance; zooming in reveals that one is actually holding an object rather than truly crossing his hands. These cases demand particularly fine-gra mne-ggained visual discrimination and action-semantic grounding, which the model overlooked. (2) Unusually long relational chain. Failures occur in texts with very long spatial or relational chains. These require multi-step reasoning with frequent spatial shifts, where errors in intermediate links accumulate, as shown in Fig. 5b
5 Conclusion
We present SaFiRe, a cognitively inspired framework for referring image segmentation that addresses the limitations of existing seyword-matching approaches. Motivated by the challenge of referential ambiguity, our method simulates the human two-phase grounding process through alternating Saccade and Fixation operations. Built upon the Mamba state-space model, SaFiRe enables efficient global-to- local semantic alignment and achieves state-of-the-art performance both standard and ambiguous datasets, including the newly proposed aReíCOCO.
Limitation and Future Work. Despite its effectiveness, SaFiRe still faces challenges. in handling language diversity in real-world application, which cannot be fully represented by existing datasets. Future work will explore incorporating structured scene understanding and pragmatic reasoning, and extending the framework to dialog-based and video grounding tasks. Moreover, SaFiRe has the potential to be adapted to other tasks or modalities with minimal modification, such Open- Vocabulary Segmentation (OVS) 61+ by adjusting the task head, Audio- Visual Segmentation (AVS) (64-66] by substituting the backbone.
10

---

Acknowledgments
This work is supported by the National Key R&D Program of China (No. 20222D0160702), National Natural Science Foundation of China (No. 62306178) and STCSM (No. 22D222290055,, 111 plan (No. BPO719010).
References
[1] L. Ye, M. Rochan, Z. Liu, and Y. Wang, "Cross-modal self-attention network for referring image segmenta- tion," in Proceddings of the IEEE/CVF conference computer vision and pattern recognition, 2019, pp. 1050--10511.
[2] G. Luo, Y. Zhou, R. Ji, X. Sun, Su, C.-W. Lin, and Q. Tian, "Cascade grouped attention network for referring expression segmentation," Proceedings of the 28th ACM International Conference on Multimedia, 2020.
[3] Z. Hu, G. Feng. J. Sun, L. Zhang, and H. Lu, "Bi-directional relationship inferring network for refer- ring image segmentation." in Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2020, pp. 4424-4433,
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, "Swin transformer: Hierarchical vision transformer using shifted windows," Proceedings of the IEEE/CVE international conference on computer vision, 2021, pp. 10012 10022.
[5] M.-H. Guo, C.-Z. Lu, Q. Hou, Z. Liu, M.-M. Cheng, and S.-M. Hu, "Segnext: Rethinking convolutional attention design for semantic segmentation," Advances in Neural Information Processing Systems, vol. 35, pp. 1 1140-115 1156, 2022.
[6] Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang. Q. Ye, and Y. Liu, "Vmamba: Visual state space model," arXiv preprint Xivv:4411.10166 2024.
Z. Yang, Wang. Y. Tang, K. Chen, H. Zhao, and H. Torr, "LAVT: Language-Awwa Aware Vision Transformer for Referring Image Segmentation," " IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[8] S. Ouyang, H. Wang, S. Xie, Z. Niu, R. Tong, Y.W. Chen, and L. Lin, "SLVIT: Scale- Wise Language- Guided Vision Transformer for Referring Image Segmentation," in International Joint Conference on Artificial Intelligence (ICCAI), 2023.
Y. Yang, C. Ma, Yao, Zhong, Y. Zhang, and Y. Wang, "Remamber: Referring image segmentation with mamba twister," European Conference on Computer Vision. Springer, 2025, pp. 108-126.
[10] B. Chen, Z. Hu, Ji, Bai, and W. Zuo, "Position-aware contrastive alignment for referring image segmentation," arXiv preprint arXiv:2212.1344 2022.
[11] J. Tang, G. Zheng, C. Shi, and S. Yang, "Contrastive grouping with transformer for referring image segmentation." in CVPR, 2023.
[12] N. A. Shah, V. VS, and V. M. Patel, "Lqmformer: Language-aware query mask transformer for referring image segmentation," " in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024, pp. 12 12903-12913.
[13] S. Kazemzadeh, Ordonez, M. Matten, Berg, "ReferliGame; Referring objects in photographs of natural scenes." " in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014.
[14] C. Liu, H. Ding, and X. Jiang, "Gres: Generalized referring expression segmentation," in Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023, pp. 23 592 23 601.
[15] H. Ding, C. Liu, S. Wang, and X. Jiang, "VLT: Vision- LanguaLa Transformer and Query Generation for Referring Segmentation." " IEEE Trunsactions on Pattern Analysis and Machine Intelligence (TPAMI). vol. 45, 2022.
[16] Y. X. Chng. H. Zheng, Y. Han, X. Qiu, and G. Huang. "Mask grounding for referring image segmentation." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp.
26 573 583.
11

---

[17] A. M. Treisman and G. Gelade, "A feature-integration theory of attention," Cognitive Psychology, vol. 12, no. I. pp. 97-136, 1980.
[18] H. H. Clark and W. G. Chase, "On the process of comparing sentences against pictures," Cognitive Psychology, vol. 3, no. 3. pp- 472-517, 1972.
[19] X. Lai, Z. Tian, Y. Chen, Y. Li, Y. Yuan, S. Liu, and J. Jia, "Lisa: Reasoning segmentation via large language model," arXiv preprint arXiv:2308. 00692, 2023.
[20] H. Rasheed, M. Maaz. S. Shaji. A. Shaker, S. Khan, H. Cholakkal, R. M. Anwer, E. Xing, M.-H. Yang, and
F. S. Khan, "Glamm: Pixel grounding large multimodal model," arXiv preprint arXiv:231 1.03556, 2023.
[21] Z. Xia, D. Han, Y. Han, X. Pan. S. Song, and G. Huang, "Gsva: Generalized segmentation via multimodal large language models," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3858-3869.
[22] A. Gu, K. Goel, and Ré. "Efficiently modeling long sequences with structured state spaces." arXiv preprint arXiv:2111.0053 1.00996 2021.
[23] A. Gu and T. Dao, "Mamha: Linear-time sequence modeling with selective state spaces," arXiv preprint arXiv:2312.00752. 2023.
[24] L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang. "Vision mamba: Efficient visual representation learning with bidirectional state space model," arXiv preprint arXiv:2401.0944 2024.
[25] H. Guo, Li, T. Dai, 7. Ouyang, X. Ren, and S.-T. Xia, "Mambair: A simple baseline for image restoration with state-space model," in European Conference on Computer Vision. Springer, 2025, pp. 222-241.
[26] J. Park, J. Park, Z. Xiong. N. Lee, J. Cho, S. Oymak, K. Lec, and D. Papailiopoulos, "Can mamba learn how to learn? a comparative study on in in-context learning tasks," arXiv preprint (xXii:2402.04244 2024.
[27] S. Peng, X. Zhu, H. Deng, L.-J. Deng, and Z. Lei, "Fusionmamba: Efficient remote sensing image fusion with state space model." IEEE Trunsactions on Geoscience and Remote Sensing, 2024.
[28] X. Xie, Y. Cui, C.-L. Ieong, T. Tan, X. Zhang, X. Zheng, and Z. Yu, "Fusionmamba: Dynamic feature enhancement for multimodal image fusion with mamba," arXiv preprint 24X1:2240.009498, 2024.
[29] W. Dong, H. Zhu, S. Lin, X. Luo, Y. Shen, X. Liu, J. Zhang, G. Guo, and B. Zhang, "Fusion-mamba for cross-modality object detection," arXiv preprint arXiv:2404.09146 2024.
[30] H. Li, Q. Hu, Y. Yao, K. Yang, and P. Chen, "Cfmw: Cross-modality fusion mamba for multispectral object detection under adverse weather conditions." arXiv preprint ar0iv:2404. 16302, 2024.
[31] Z. Li, H. Pan, K. Zhang, Y. Wang, and F. Yu, "Mambadfuse: A mamba-based dual- -halep model for multi- modality image fusion," arXiv preprint arXiv:2404.08406 2024.
[32] W. Li, H. Zhou, J. Yu. Z. Song, and W. Yang. "Coupled mamba: Enhanced mudti-modal fusion with coupled state space model," arXiv preprint arXiv:2405.1800 2024.
[33] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint 1810:1810.048000 2018.
[34] W. Peebles and S. Xie, "Scalable diffusion models with transformers," in Proceedings of the IEEE/CVF international conference computer vision, 2023, pp. 4195-4205.
[35] F. Milletari, N. Navab, and S.-A. Ahmadi, "V-net: Fully convolutional neural networks for volumetric medical image segmentation." in 2016 fourth international conference on 3D vision (3DV). Teee, 2016, pp. 565-571.
[36] T.-Y. Ross and G. Dolkár, "Focal loss for dense object detection," " in proceedings the IEEE conference on computer vision and pattern recognition, 2017, pp. 2980-2988
[37] V. K. Nagaraja, L. Morariu, and Davis, "Modeling context between objects for referring expression understanding," in Computer ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part /V 14. Springer, 2016, pp. 792-807.
[38] T.-Y. Lin, M. Maire, S. Belongie, Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft coco: Common objects in context," in Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 2014, pp. 750-755.
12

---

[39] J. Bai. S. Bai, S. Yang. S. Wang, S. Tan, P. Wang, J. Lin. C. Zhou, and J. Zhou, "Qwen-vl: A versa- tile Lasgun- model for understanding, localization, text reading, and beyond," arXiv preprint arXiv:2308. 12966, vol. 1. no. 2, p. 3, 2023.
[40] 7. Wang, Y. Lu, Q. Li, X. Tao, Y. Guo, M. Gong, and T. Liu, "CRIS: CLIv-Driven Referring Image Segmentation." in IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2022.
[41] H. Ding, S. Zhang, Q. Wu, S. Yu, J. Hu, I.. Cao, and R. Ji, "Bilateral Knowledge Interaction Network for Referring Image Segmentation," IEEE Transactions on Multimedia (TMM), vol. 26, 2023.
[42] Z. Yang, J. Wang. Y. Tang, K. Chen, H. Zhao, and P. H. S. Torr, "Semantics- Aware Dynamic Localization and Refinement for Referring Image Segmentation," in AAAI Conference on Artificial Intelligence (AAAI), 2023.
[43] Y. Hu, Q. Wang, W. Shao, E. Xie, Z. Li, J. Han, and P. Luo, "Beyond One-to-One: Rethinking the Referring Image Segmentation," in Proceedings of International Conference on Computer Vision (ICCV), 2023.
[44] S. Kim, M. Kang, D. Kim, Park, and Kwak, "Extending clip's 's image-text alignment to referring image segmentation." arXiv preprint arXiv:2306.08499 2024.
[45] W. Chen, L. Chen, and Y. Wu, "An efficient and effective transformer decoder-based framework for multi-task visual grounding," " in European Conference on Computer Vision. Springer, 2024, pp. 125-141.
[46] C. Shang. Z. Song, H. Qiu, L. Wang, F. Meng, and H. Li, "Prompt-driven referring image segmentation with instance contrasting," " in Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, 2024, pp. 4124-4134.
[47] L. Xiao, X. Yang. F. Peng. Y. Wang, and C. Xu, "Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling," Advances in Neural Information Processing Systems, vol. 37, pp. 139 8588834-15 885, 2024.
[48] M. Dai, J. Li. J. Zhuang, X. Zhang, and W. Yang. "Multi-task visual grounding with coarse-to- tine consistency constraints." Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 3. 2025, pp. 2618-2626.
[49] J. Liu, H. Ding, Z. Cai, Y. Zhang, R. K. Satzoda, V. Mahadevan, and R. Manmatha, "Polyformer: Referring image segmentation as sequential polygon generation." " in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023, pp. 18 653-18311161 18 663.
[50] B. Yan, Y. Jiang, J. Wu, D. Wang, Luo, Z. Yuan, and H. Lu, "Universal instance perception as object discovery and retrieval," " Proceedings of the IEEE/CVF Conference on Computer Vision and Pateern Recognition, 2023, pp. 15 15322-15336
[51] Xu, Xu, Y. Yang, X. Li, Xie, Y.J. Huang, and Y. Li, "u-llava: Unifying multi-modal tasks via large language model," arXiv preprint arXiv:2311.0534, 2023.
[52] Z. Zhang, Y. Ma, E. Zhang, and X. Bai, "Psalm: Pixelwise segmentation with large multi- model," in European Conference on Computer Vision. Springer, 2025, pp. 74-91.
[53] Z. Ren, Z. Huang, Y. Wei, Y. Zhao, D. Fu, J. Feng, and X. Jin, "Pixellm: Pixel reasoning with large multimodal model," arXiv preprint arXiv23312.02236 2023.
[54] S. Shao, Z. Li, T. Zhang, C. Peng, G. Yu, X. Zhang, J. Li, and J. Sun, "Objects365: A large-scale, high-quality dataset for object detection," in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 8430 8439.
[55] B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba, "Semantic understanding of scenes through the ade20k dataset," International Journal of Computer Vision, vol. 127, pp. 302-321, 2019.
[56] H. Cacsar, J. Uijlings, and V. Ferrari, "Coco-stuff: Thing and stuff classes in context," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1209-1218.
[57] V. Ramanathan, A. Kalia, V. Petrovic, Y. Wen. B. Zheng, B. Guo, R. Wang, A. Marquez, R. Kovvuri,
A. Kadian et al., "Paco: Parts and attributes of common objects," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 7141-7151.
[58] X. Chen, R. Mottaghi, X. Liu, S. Fidler, R. Urtasun, and A. Yuille, "Detect what you can: Detecting and representing objects using holistic models and body parts," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1971-1978
13

---

[59] M. Everingham. L. Van Gool. C. K. C. L Williams. J. Winn, and A. Zisserman, "The PASCAL Visual Object Classes Challenge 2010 (VOC2010) Results," http://ww..pacaal network.
[60] S. Yang. T. Qu. X. Lai. Z. Tian, B. Peng. S. Liu, and J. Jia, "An improved baseline for reasoning segmentation with large language model," CoRR, 2023.
[61] B. Li, K. Q. Weinberger, S. Belongie, V. Koltun, and R. Ranftl, (Lanuuage-drive driven semantic segmentation." arXiv preprint (tt::2201.035466 2022.
[62] G. Ghiasi, X. Gu, Y. Cui, and T.-Y. Lin, "Scaling open-vocabulary image segmentation with image-level labels," in European conference on computer vision. Springer, 2022, pp. 540-557.
[63] C. Ma, Y. Yang. Y. Wang. Y. Zhang, and W. Xie, vision-language models." arXiv preprint arXiv:2210.15136 2022.
semantic segmentation with frozen
[64] J. Zhou, J. Wang. J. Zhang, W. Sun, J. Zhang. S. Birchfield, D. Guo, L. Kong, M. Wang. and Y. Zhong, "Audio-visss" segmentation." in European Conference on Computer Vision. Springer, 2022, pp. 386-403,
[65] J. Liu, Y. Wang, C. Ju, C. Ma, Y. Zhang, and W. Xie, "Annotation-fiee audio-visual segmentation." in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 5604- 5614.
[66] J. Liu, C. Ju, C. Ma, Y. Wang. Y. Wang, and Y. Zhang, "Audio-aware query-enhanced transformer for audio-visual segmentation," arXiv preprint arXiv:2307. 13236, 2023.
[67| J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431-3440.
[68] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmen- tation," in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015. pp. 234-241.
[69] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, "Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs," IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4. pp. 834 848, 2017.
[70] K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
[71] C. Ma. Q. Xu, X. Wang. B. Jin, X. Zhang, Y. Wang. and Y. Zhang, "Boundary-awar supervoxel-level iteratively refined interactive 3d image segmentation with multi-agent agent reinforvement learning," IEEE Transactions on Medical Imaging, vol. 40, no. 10, pp. 2563-2574, 2020.
[72] Z. Ye. C. Ju, C. Ma, and X. Zhang, "Unsupervised domain adaption via similarity- prototypes for cross- modalityy segmentation," in MICCAI Workshop Domain Adaptation and Representation Transfer. Springer, 2021, pp. 133-143.
[73] S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng. T. Xiang, P. H. Torr et al., "Rethinking semantic segmentation from sequenec-to sequencc-to-sqquence -o-suueeccceeeeee perspective with transformers." in Proceedings of the IEEE/CVF conference on computer vision pattern recognition, 2021, pp. 6881-6890.
[74] R. Ranftl, Bochkovskiy, and V. Koltun, "Vision transformers for dense prediction," in Proceedings of the IEEE/CVF international conference on computer vision. 2021. pp. 179-12 188.
[75] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, "Segformer: Simple and efficient design for semantic segmentation with transformers," Advances in neural information processing systems, vol. 34, pp. 12077 090, 2021.
[76] B. Cheng, A. Schwing, and A. Kirillov, "Per-pixel classification not all you need for semantic segmenta- tion," Advances in neural information processing systems, vol. 34, pp. 17 864-177778 875, 2021.
[77] A. Kirillov. Mintun, Ravi, H. Mao, Rolland, L. Gustafson, T. Xiao. Whitehead, A. C. Berg. W.Y Lo et al., "Segment anything." in Proceedings of the IEEE/CVF international conference on computer vision. 2023, pp. 4026.4026.
[78] W. Liu, C. Ma. Y. Yang. W. Xie, and Y. Zhang, "Transforming the interactive segmentation for medical imaging." in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2022, pp. 704-713.
14

---

[79] D. Baranchuk, I. Rubachev, A. Voynov, V. Khrulkov, and A. Babenko, "Label-efficient semantic segmenta- tion with diffusion models." arXiv preprint 12x::2212.031166 2021.
[80] Wolleb, Sandkühler, F. Bieder, P. Valmaggia, and C. Cattin, "Diffusion models for implicit image segmentation ensembles," in International conference on medical imaging with deep learning. PMLR, 2022, pp. 1336-1348.
[81] T. Amit, T. Shaharbany, F. Nachmani, and I.. Wolf, "Segdiff: Image segmentation with diffusion proba- bilistic models," arXiv preprint arXiv:2112.00390, 2021.
[82] J. Wu, R. Fu, H. Fang, Y. Zhang. Yang. H. Xiong, H. Liu, and Y. Xu, "Medsegdiff: Medical image segmentation with diffusion probabilistie model," in Medical Imaging with Deep Learning. PMLR, 2024, pp.
[83] T. Zhang, F. Zhang, J. Yao, Y. Zhang, and Y. Wang. "G4seg: Generation for inexact segmentation refinement with diffusion models," arXiv preprint arXiv:2506.0/555 2025.
[84] C. Ma. Yang. C. Ju, Y. Shi, Zhang. and Y. Wang, "Freesegdiff: Annotation- free saliency segmentation with diffusion models," in ICASSP 2025-2025 IEEE International Conference on Acoustics, Speeck and Signal Processing (ICASSP). IEEE, 2025, pp. 1-5,
[85] C. Ma, Y. Yang, C. Ju, F. Zhang, J. Liu, Y. Wang. Y. Zhang, and Y. Wang. "Diffusionseg: Adapting diffusion towards unsupervised object discovery," arXiv preprint arXiv:2303.0981/3 2023.
[86] T. Zhang, C. Ma, and Wang. "Tracking the rareness diseases: Improving long-tail tail medical detection with a calibrated diffusion model," Electronics, vol. 13, no. 23, p. 4693, 2024.
[87] C, Ma, Y. Yang, C. Ju, F. Zhang, Y. Zhang, and Y. Wang, "Attrseg: oocn-vocyb semantic segmentation via attribute decomposition aggregation." Advances in neural information processing systems, vol. 36, pp. 10225-102270 270, 2023.
decompostioonaagrrgaatinn
[88] Xu. S. De Mello, Liu, W. Byeon. Breuel, J. Kautz, and X. Wang. "Groupvit: Semantic segmentation emerges from text supervision." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 18 134-18 144.
[89] F. Zhang. T. Zhou, B. Li, H. He, C. Ma, T. Zhang, J. Yao, Y. Zhang, and Y. Wang. "Uncovering proto- typical knowledge for weakly opea-vocabulary semantic segmentation," Advances in Neural Information Processing Systems, vol. 36, pp. 73 652-73 663, 2023.
[90] Y. Yang. C. Ma. C. Ju, F. Zhang. J. Yao. Y. Zhang, and Y. Wang. "Multi-modal prototypes for open-world semantic segmentation." International Journal of Computer Vision, vol. 132, pp. 6004-6020, 2024.
[91] L. Liu, M. Zhang, J. Yin, T. Liu, Ji, Y. Piao, and H. Lu, "Defmamba: Deformable visual state space model," in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 8838-8847.
[92] J. Wang, D. Paliotta, A. May, A. Rush, and Dao, "The mamba the Ilama: Distilling and accelerating hybrid models," Advances in Neural Information Processing Systems, vol. 37, pp. 62 432-6224444 457, 2024.
[93] L. Wang. B. Zhu, Y. Chen, Y. Zhang, M. Tang, and J. Wang, "Mug: Pseudo labeling augmented audio-visual mamba network for audio-visual video parsing," arXiv preprint arXiv:2507.013388 2025.
[94] Y. Yang, C. Ma. Z. Mao, J. Yao, Y. Zhang, and Y. Wang, "Moma: Modulating mamba for adapting image foundation models to video recognition," arXiv preprint (rXXi:2506.2328. 2025.
[95] K. Li, X. Li, Y. Wang, Y. He, Y. Wang, L. Wang, and Y. Qiao, "Videomamba: State space model for efficient video understanding." arXiv preprint 24X.0.2403.06977 2024.
[96] M. H. Erol, A. Senocak, J. Feng, and J. S. Chung, "Audio mamba: Bidirectional state space model for audio representation learning." arXiv preprint arXiv:2406.03:: 033444 2024.
[97] Ma, Li, and Wang, "U-mamba: Enhancing long-range dependency for biomedical image segmenta- tion," arXiv preprint arXiv:2401.0472 2024.
[98] J. Liu, M. Liu, Z. Wang, P. An. X. Li, K. Zhou, S. Yang. R. Zhang, Y. Guo, and S. Zhang. "Robomamba: Efficient vision- lanuua-annuag-actttoo model for robotic reasoning and manipulation." " Advances in Neural Information Processing Systems, vol. 37, pp- 40 085-40 110, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes, our paper discussed the limitations of the work in the Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

This paper is application-orieee oriented and does not contain any theoretical results, assumptions, or formal proofs. The focus is on model design and benchmark evaluation rather than theoretical development.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides comprehensive details necessary for reproducing the main experimental results. In Section 3, we describe the model architecture and in Section 4.3, we elaborate on the training setup.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
17

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification:
We will release the code and dataset publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips.cc/ppblic. wwwpppppc/gguueaeecceeeoeoooooee det.ggddde//oeeummiiieeeee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies experimental details in Section (Method) and Section 4.3 (Implementation). Furthermore, Section 4.5 (Ablation Study) details the configurations and modifications used in component-level experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Statistical significance analysis is generally not reported in referring image segmentation literature. It is widely considered unnecessary and is not part of the standard evaluation protocol in this field.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we provide sufficient information in Section 4.3 (Implementation),
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://pp//.ppppp...ooo////////a
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and confirm that our research fully conforms to its principles.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Yes, discuss broader impacts in the Appendix B
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not involve the release of models datasets with high risk misuse. The released components are task-specific and do not pose safety or dual-use concerns.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used in our work- pre-trained models (e.g.. Swin Transformer-B BB and BERT-B) and datasets-are properly cited in the paper. We explicitly follow their respective licenses: Swin Transformer (MIT License) and BERT (Apache 2.0 license). For all datasets (RefCOCO family), we ensured compliance with their original license terms (Apache 2.0 license).
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
20

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We will publicly release assets (code, checkpoints, and dataset) on publication.
Full documentation will be provided alongside the assets to ensure reproducibility and ease of use.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing or human subject research in its core contribution.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
21

---

Justification:
The work involves no new human subjects or data collection requiring IRB review.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: A large language model (Qwen) was used solely during the dataset construction phase to assist in generating textual annotations for aRefCOCO. The LLM does not play any role the core methodology, model design, or training.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

A Limitations
While our method achieves strong performance across various benchmarks, several limitations must be acknowledged. First, the model's ability to process extremely complex ambiguous expressions remains limited. Although we evaluated primarily on datasets like RefCOCO family and aRefCOCO, these datasets may not fully represent the diversity of real-world scenarios. Additionally, for dataset construction, we rely on a large language model to generate textual descriptions. While this approach ensures broad range of expressions, it may inadvertently introduce bias, even after meticulous filtering.
B Broader Impacts
The work presented here has the potential for both positive and negative socictal impacts. On the positive side, method's ability understand complex referring expressions and improve image segmentation could significantly benefit fields such as medical imaging, robotics, and accessibility technologies particularly for individuals with disabilities. On the negative side, the model may inherit biases from pre-trained encoders (e.g., Swin or BERT), potentially leading to stereotypical associations in referring expressions (e.g.. linking gender social roles to certain objects), thereby amplifying societal biases in segmentation outcomes.
C More Related Work
From Traditional Segmentation to Language-Based Segmentation. Traditional segmentation has long focused on delineating object regions from visual cues alone, employing various methods such as CNN 72), Vision Transformers 73 and Diffusion Models 86). Moving beyond purely visual cues, language-based segmentation introduces linguistic grounding. enabling segmentation guided by natural language. Within this field, referring image segmentation (RIS) aims to localize and segment objects described by textual expressions, yet many methods still treat a shallow keyword- matching task, leaving ambiguous expressions unresolved. open-vocabulary segmentation (OVS), in contrast, emphasizes category generalization and broad semantic coverage through large-scale vision-language pretraining 66 661 87-90) More recently, reasoning segmentation, introduced by
LISA [9 extends beyond RIS by requiring complex reasoning or world knowledge, leveraging large language models to perform such inference. Our work, however, remains within the RIS scope: rather than relying on external knowledge, we focus on resolving the referential ambiguities that an ideal RIS model should handle but current approaches fail to address.
Mamba and Its Broader Applications. Recently, Mamba has been actively explored across unimodal and multimodal learning [91493), owing to its ability to model long-range dependencies with linear-time efficiency. It has since been adopted general-purpose backbone in diverse downstream tasks. In the visual domain, Mamba has shown promise in video understanding and spatiotemporal modeling (94 955, where its state-space formulation enables efficient temporal integration while preserving spatial fidelity. Beyond vision, it has been applied to audio and speech modeling [96]. biomedical image analysis (97], and robotics with lision-language-actto demonstrating strong adaptability across modalities. These developments highlight Mamba's growing influence as a unified sequence modeling paradigm. In this work, we further leverage its long-context sequential modeling capacity and sean-then-uppe of human perception resolve referential ambiguities in referring image segmentation.
models (98).
dynamics-mirrring mirroring the saccade-fixatee process
D More Details of aRerCOCO Dataset
D.1
Generation, Filtering and Validation
D.1.1 Generation Prompts
In order to ensure the richness and diversity of object descriptions while adhering to the fundamental requirement of single-object localization in RIS tasks, we used a prompt template for Qwen-VL VL-Max as shown in Appendix D.I.I
23

---

Generation Prompts
### Role
You are professional assistant, proficient in object recognition and description. Your task is provide clear and focused description of the main object inside the red box the image. You given the category information of the main object inside the red box as '(target)', The description should be based on the provided target information, and you should generate more detailed description with your visual analysis of the image. Please note that there is visible box in the actual image.
Target Category Information targt category information: '(target)'
### Skills
1. [Important!] Please only focus the main object, i.e., '(target)', and ignore the background irrelevant details.
2. [Important!]] Do not assume or include any information that is not present in the image content.
3. [Important!] !] Provide 5 independent English sentences, each of which can independently identify '[target)' without relying on other sentences. Each sentence should describe unique identifying feature.
[Important!] Use the subject '(target)' from the targt category information. on, and ensure you are describing a single object '[target}'! Others should able to locate only one object based on your aentences.
5. [Important]] Use singular language to ensure the description focuses only on main object. Do not plural words like 'they' the subject, but you allowed use pronouns the subject.
6. [Important!] you need describe based on objects around '[target}', please keep the subject as '[target}'.
D.1.2 Filtering
We first pre-screen all descriptions with GPT-4o for the two patterns:" patterns:"object--iiii "ojjcctddisttratinn" (i.e., con- taining multiple nouns) and "category-implici" (i.e., where the subject pronoun). Items that match either pattern are retained. Then, to verify the remaining pairs meet the basic rules we set for annotation, GPT-4o involved an evaluator. The prompt is as shown in Appendix D.1.2
Filtering Prompts
Role:
You are professional visual-language evaluator. Your task is to verify whether each given description sentence is faithful, grounded, and rule-abiding description of the target object **inside the red box** in the provided image.
Information:
The target object belongs to the category: (target).
The description was generated under the following constraints:
1. It must describe only the main object (' ('(target)'), ignoring the background or other irrelevant elements.
not include objects that of box are allowed.
in the whole image, but objects
3. Each sentence must independently identify the target.
4. It can contains objects beyond the target.
5. It must use singular phrasing, and refer to the object using either its singular pronouns.
category label ('(target)")
You will given sentences. Your task to verify, for each sentence, whether it satisfies **all** the above requirements based solely on the image content.
Sentences to Verify: (Sentences)
24

---

Instructions:
1. For each sentence, respond with **"yes*** ***no"**, if "no", follow it by **brief reason**.
2. any rule is violated or the content is ungrounded, answer "no" and specify which rule(s) are broken.
3. Do not use external knovledge or assumptions. Base your judgment strictly on what is visible in the image.
4. Provide
be its 5. Do not add commentary outside the ansuers. Return only the ansuers.
per sentence, in order. Each
line.
6. Ensure the number of answers equals 5.
D.1.3 Validation
Manual review was conducted by least two annotators with RIS domain expertise. Each pair was independently validated to confirm (1) whether description uniquely refers to a single object in the image, and (2) whether the expression obeys referential ambiguity criteria while remaining image-groundn coaese--rundabble. borderline cases, annotators reached consensus through discussion, and a third reviewer was consulted when needed.
D.2 Comparison with Original Descriptions
conduct quantitative comparisons to highlight the distinct features aRefCOCO compared original descriptions, visualized in Fig. 6 Specifically, the comparisons focus the following aspects:
Description Length. As shown in Fig. 6a i). compared to the original descriptions in RefCOCO and RefCOCOg, aReiCOCO contains higher proportion of longer sentences and significantly reduces the frequency of short, phrase-like like descriptions. This shift towards more complex sentence structures underscores the value aRefCOCO as an extension the RefCOCO series, making it particularly relevant for real-world RIS tasks that involve longer and more intricate sentence constructions.
Relational Words. We calculated the growth rates of the most frequent words in the original and aRefCOCO descriptions, focusing on those strongly correlated with the relationships between targets and distractors. The results in Fig. 6al ii) show significant increase in these words, particularly prepositions like "and", "to", and "of which grew by 450.7%. 288.6%, and 249.8% respectively. These growths contribute more structurally and semantically complex sentences, enhancing spatial and relational expressions and enriching the semantic content in aRcíCOCO descriptions.
Text Embedding Distribution. To further examine the semantic differences between the original and aRefCOCO descriptions, we visualize their text embeddings in a reduced 3D space using PCA, as shown in Fig. [6b] The figure illustrates clear shift in embedding distributions, where the aRefCOCO (red) embeddings are consistently displaced from their original (blue) counterparts. This systematic divergence indicates that the rewritten descriptions introduce substantial semantic variations rather than minor lexical edits.
E More Visualization Results
In this section, we provide more visualization results of our SaFiRe, ReMamber 9 and PSALM 52 on aRefCOCO for further comparison, as shown in Fig.
25

---

(1) Seniicne Length Distriutioon
Embedding Districutinn Companioon with Chanpes
Ponsting (64
000- 8.9
Sentice Lingh (onely
Growth Reiational
Couod tax 2M
SMRAL
1417%
543%
Relvional Werd
(a) Sentence-level statistics comparison
(b) 3D text embedding distribution comparison
Figure 6: Comparison of Sentence-Leve Statistics and Text Embedding Distributions Between the Original Descriptions and aRefCOCO.
ReMamber
Rektamer
PSAI
is part the whole pizza, taken out from the plate. showing stringy cheese
This Individual stands in front of fruit and vegetable stand, hoiding white plastic
ReMamber
PSALM
Ours
BeMamber
PSALM
Ours
This carntt Is centrally Incated in the image, with Its tip pointing disrmwaedd.
ths fur white, contranting with the red bedidy beer the child's hand.
girln ReMamber
photo.
man.
PSALM
Ours
GT
BeKamber
PSALM
Ourss
in perbad parkking lot, closs to the foregrmundd Ics shin
dark gray, large slow, standing in the foreground:
stancing among
image.
seemingly waiting something.
Figure More Results on aRefCOCO (Part 1).
26

---

Oure
stands next to an adut. ready ski
Only part his shoulder and back are visibie.
CT
ReMamiber
PCA. M
Ours
GT
Bektamber
PGAL M
Ours
The handbag is medium size. rectanguar in shape. and features prominent pink
Equipped chest and lng guards and gloves, the catcher is ready catch the
ReNamber
mevers hlack orange glaves, muuhinng plate. BerMmmerr
right with porieets,
PSALM
Behlamber
PSALM
Ours
chocstate-coveees the
food.
Figure 7: More Results on aRefCOCO (Part 2, continued).
27


Reviews:
[{"review_id": "7cIJ4drGSj", "reviewer": "Reviewer_8mtA", "summary": "This paper proposes SaFiRe, a novel framework for Referring Image Segmentation (RIS) that addresses referential ambiguity in natural language expressions. Inspired by human two-phase cognitive processes, SaFiRe integrates Saccade and Fixation operations using Mamba’s state-space modeling. The Saccade operation establishes coarse cross-modal alignment, while Fixation refines it via region-wise text reiteration. The authors also introduce aRefCOCO, a benchmark for ambiguous expressions, and demonstrate SaFiRe’s superiority over SOTA methods on both standard and aRefCOCO datasets. Key contributions include modeling cognitive mechanisms, efficient multi-cycle refinement with linear complexity, and a new benchmark for ambiguous RIS.", "questions": "Q1: Could the authors explicitly state the primary scientific concern that SaFiRe aims to address—beyond merely introducing a new framework? Given the experimental limitations (marginal gains over Magnet, and absence of recent SOTA benchmarks like EEVG/C3VG), it is unclear whether the core contribution lies in cognitive modeling or practical performance advancement. \n\nQ2:Could the authors clarify whether the first SFLayer uses the final-layer features from Swin Transformer and BERT, and whether you use LAVT’s cross-modal fusion in the backbone stage?\n\nNote: A thorough response that resolves these concerns and demonstrates the solidity of experimental validations could potentially influence the evaluation score positively.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.The paper introduces SaFiRe, a novel framework that simulates human two-phase cognitive processes: global understanding via Saccade and detailed refinement via Fixation, to address referential ambiguity in RIS. By leveraging Mamba’s state-space modeling, the framework achieves efficient multi-cycle refinement with linear complexity, enabling robust alignment between visual regions and ambiguous language expressions. This design achieves significant performance gains on standard benchmarks and superior accuracy on ambiguous expressions.\n\n2.The authors present aRefCOCO, a test-only benchmark specifically designed to evaluate RIS models under object-distracting and category-implicit scenarios. With an average of 12.6 words per expression, 3.1 same-category distractors per image, and frequent pronoun usage, aRefCOCO pushes the field to tackle real-world referential ambiguity. \n\nWeaknesses:\n1.Unfair, Marginal and Incomplete Comparative Evaluation\nThe experimental validation lacks rigor due to suboptimal benchmarking choices. SaFiRe uses Swin Transformer as the visual backbone rather than Mamba, making direct comparisons with ReMamber (a pure Mamba-based model) inherently unfair, as the latter’s architecture is tailored for state-space modeling . When compared to Magnet (a Swin-BERT-based method more aligned with SaFiRe’s backbone), performance gains are marginal, failing to demonstrate substantial superiority. Additionally, the evaluation omits recent SOTA methods(e.g., EEVG[1], C3VG[2], OneRef[3], and PromptRIS[4]). This absence hinders the paper’s ability to validate SaFiRe’s competitiveness in the evolving RIS landscape, casting doubt on the robustness of its claimed advancements.\n\n2.Inadequate Transparency in aRefCOCO Dataset Generation\nThe reliability of aRefCOCO, the proposed benchmark for ambiguous expressions, is undermined by insufficient evidence of its annotation process. While the authors mention that a significant portion of text-image pairs were generated using Qwen-VL-Max, they omit critical details such as the prompting strategies, filtering criteria, and validation mechanisms . The absence of concrete examples (e.g., sample prompts, annotation guidelines) hinders reproducibility and raises concerns about potential biases introduced by LLM-generated texts. Without transparent documentation, it is difficult to assess whether the dataset truly captures real-world referential ambiguity or inadvertently reflects the limitations of the underlying language model .\n\n[1] An efficient and effective transformer decoder-based framework for multi-task visual grounding. In ECCV, 2024.\n[2]Multi-task visual grounding with coarse-tofine consistency constraints. In AAAI, 2025.\n[3]Oneref: Unified one-tower expression grounding and segmentation with mask referring modeling. In NeurIPS, 2024.\n[4]Prompt-driven referring image segmentation with instance contrasting. In CVPR,2024.", "was_revised": true, "final_justification": "I raise the score in this stage.", "rebuttal": "Thank you for the detailed review. We would like to first thank you for highlighting the novelty and performance gains of our Saccade–Fixation framework—benefits that R7DT and jD3V likewise praised, with even Wg1a calling the experimental results “promising.” \n\nBelow, we address each of your comments point‑by‑point and clarify how our revisions and additional experiments resolve the concerns you raised.\n\n>**8mtA-W1:** Concerns regarding comparative evaluation: (1) \"Unfair\" comparison with ReMamber due to backbones difference; (2) \"Marginal\" gains over Magnet; (3) Missing recent SOTAs \n### (1) Comparing with ReMamber\nWe include ReMamber not for a direct comparison of performance, but rather **as a reference baseline**, since it is the only published Mamba-based RIS method. We will revise our tables to clearly indicate that ReMamber is a full-Mamba-based architecture, to avoid any misunderstanding regarding the fairness of the comparison. Meanwhile, as other methods, ReMamber still approaches RIS through keyword-matching modeling and cannot effectively address the referential ambiguity problem that we aims to solve.\n### (2) Comparing with MagNet\nSaFiRe demonstrates consistent gains across all datasets of RefCOCO series. Our advantage on RefCOCO is kind of marginal because this dataset is relatively simple. However, as the dataset becomes more challenging, our model's advantage becomes evident, with strong improvements on the most difficult **RefCOCOg** dataset (+1.3 oIoU on average). \n\nAdditionally, we here provide more comparisons with MagNet, including mixed‑dataset training (which better reflects the linguistic variety than single-dataset) and inference speed. Results are averaged over each split, with the numbers in parentheses indicating the **performance gap relative to ours**. SaFiRe surpasses MagNet in both accuracy and speed. \n\n|Method|Backbone|Metric|RefCOCO|RefCOCO+|RefCOCOg|\n|-|-|-|-|-|-|\n|MagNet|SwinB+BERT|oIoU|75.7(-5.4)|67.9(-6.1)|68.5(-7.8)|\n|**SaFiRe (Ours)**|SwinB+BERT|**oIoU**|**81.1**|**74.0**|**76.3**|\n\n|Models|FPS (images/s)|Speedup|\n|-|-|-|\n|MagNet|3.55|1x|\n|**Ours**|**8.26**|**2.32x**|\n\n### (3) Comparing with recent SOTAs\nFirst, we want to clarify that the main evaluation metric reported in [1]-[4] is **mIoU**, which differs from the widely-used **oIoU** in this field. mIoU is typically numerically ***higher*** than oIoU (see results in [2],[4]). The distinction between the two is shown in LAVT(CVPR2022).\nHere, we provide the comparison using mIoU and oIoU. Note that their main results are under mixed-dataset setting (similar with Tab.2 in our main paper). Results are averaged over each split, with the numbers in parentheses indicating the **performance gap relative to ours**. Compared with these methods, SaFiRe still achieves the best accuracy.\n\n|Method|Backbone|Metric|RefCOCO|RefCOCO+|RefCOCOg|\n|-|-|-|-|-|-|\n|EEVG [1]|Swin-B+BERT|mIoU|77.5(-4.5)|70.5(-6.4)|71.7(-6.0)|\n|PromptRIS [4]|CLIP+SAM|mIoU|78.0(-4.0)|70.7(-6.2)|69.8(-7.9)|\n|OneRef-B (ft)[3]|BEiT3-B|mIoU|79.6(-2.4)|74.1(-2.8)|74.5(-3.2)|\n|C3VG [2]|BEiT3-B|mIoU|81.1(-0.9)|76.3(-0.6)|76.7(-1.0)|\n|**SaFiRe (Ours)**|Swin-B+BERT|**mIoU**|**82.0**|**76.9**|**77.7**|\n\n|Method|Backbone|Metric|RefCOCO|RefCOCO+|RefCOCOg|\n|-|-|-|-|-|-|\n|PromptRIS [4]|CLIP+SAM|oIoU|76.3(-4.8)|66.5(-7.5)|66.0(-10.3)|\n|C3VG [2]|BEiT3-B|oIoU|80.6(-0.5)|73.8(-0.2)|75.4(-0.9)|\n|**SaFiRe (Ours)**|Swin-B+BERT|**oIoU**|**81.1**|**74.0**|**76.3**|\n\n(ft denotes a further fine-tuning is performed on the specific dataset.)\n\nSecond, since C3VG and EEVG provide codes and weights, we further test them on aRefCOCO; SaFiRe keeps a clear lead on all metrics.\n\n|Method|oIoU|mIoU|Pr@50|Pr@70|Pr@90|\n|-|-|-|-|-|-|\n|EEVG [1]|50.7|59.4|66.9|62.4|29.4|\n|C3VG [2]|62.9|69.1|78.4|72.8|32.8|\n|**SaFiRe (Ours)**|**65.1**|**71.4**|**78.9**|**73.6**|**45.2**|\n\nThird, given C3VG’s good accuracy, we also compare inference speed: SaFiRe runs 3.3× faster, confirming its advantage in application.\n\n|Models|FPS (images/s)|Speedup|\n|-|-|-|\n|C3VG [2]|2.53|1x|\n|**Ours**|**8.26**|**3.27x**|\n\n>**8mtA-W2:** Inadequate Transparency in aRefCOCO Dataset Generation. The reliability of aRefCOCO, including VLM prompts,  filtering and validation steps.\n- **8mtA-W2-A:** Here we provide details of this process. Codes and datasets will be released upon publication.\n### (1) Annotation Prompting\nIn order to ensure the richness and diversity of object descriptions while adhering to the fundamental requirement of single-object localization in RIS tasks, we used a prompt template for Qwen-VL-Max as below:\n\n    \"### Role\\n\"\n    \"You are a professional assistant, proficient in object recognition and description. Your task is to provide a clear and focused description of the main object inside the red box in the image. You are given the category information of the main object inside the red box as '{target}'. The description should be based on the provided target information, and you should generate a more detailed description with your visual analysis of the image. Please note that there is no visible box in the actual image.\\n\\n\"\n\n    \"### Target Category Information\\n\"\n    f\"targt category information: '{target}'\\n\\n\"\n\n    \"### Skills\\n\"\n    \"1. [Important!] Please only focus on the main object, i.e., '{target}', and ignore the background or irrelevant details.\\n\"\n    \"2. [Important!] Do not assume or include any information that is not present in the image content.\\n\"\n    \"3. [Important!] Provide 5 independent English sentences, each of which can independently identify '{target}' without relying on other sentences. Each sentence should describe a unique identifying feature.\\n\" \n    \"4. [Important!] Use the subject '{target}' from the targt category information, and ensure you are describing a single object '{target}'! Others should be able to locate only one object based on your sentences.\\n\"\n    \"5. [Important!] Use singular language to ensure the description focuses only on one main object. Do not use plural words like 'they' as the subject, but you are allowed to use pronouns as the subject.\\n\"\n    \"6. [Important!] If you need to describe based on objects around '{target}', please keep the subject as '{target}'.\\n\\n\"\n\n\n### (2) Filtering\nWe first pre-screen all descriptions with GPT 4o for the two patterns:\"object-distracting\" (i.e., containing multiple nouns) and \"category-implicit\" (i.e., where the subject is a pronoun). Items that match either pattern are retained. \nThen, to verify the remaining pairs meet the basic rules we set for annotation, GPT 4o is involved as an evaluator. The prompt is as follows:\n\n    Role: \n    You are a professional visual-language evaluator. Your task is to verify whether each given description sentence is a faithful, grounded, and rule-abiding description of the target object **inside the red box** in the provided image.\n    \n    Information:\n    The target object belongs to the category: {target}.\n\n    The description was generated under the following constraints:\n    1. It must describe only the main object ('{target}'), ignoring the background or other irrelevant elements.\n    2. It must not include objects that are not in the whole image, but objects out of box are allowed.\n    3. Each sentence must independently identify the target.\n    4. It can contains objects beyond the target.\n    5. It must use singular phrasing, and refer to the object using either its category label ('{target}') or singular pronouns.\n\n    You will be given 5 sentences. Your task is to verify, for each sentence, whether it satisfies **all** the above requirements based solely on the image content.\n\n    Sentences to Verify:\n    {Sentences}\n\n    Instructions:\n    1. For each sentence, respond with **\"yes\"** or **\"no\"**. if \"no\", follow it by a **brief reason**.\n    2. If any rule is violated or the content is ungrounded, answer \"no\" and specify which rule(s) are broken.\n    3. Do not use external knowledge or assumptions. Base your judgment strictly on what is visible in the image.\n    4. Provide one answer per sentence, in order. Each answer must be on its own line.\n    5. Do not add commentary outside the answers. Return only the answers.\n    6. Ensure the number of answers equals 5.\n\n### (3) Validation\nManual review was conducted by at least two annotators with RIS domain expertise. Each pair was independently validated to confirm (1) whether the description uniquely refers to a single object in the image, and (2) whether the expression obeys our referential ambiguity criteria while remaining image-groundable. In borderline cases, annotators reached consensus through discussion, and a third reviewer was consulted when needed.\n\n> **8mtA-Q1:** Given the experimental limitations, state the primary scientific concern that SaFiRe aims to address \n\n**8mtA-Q1-A:** First, we hope the experiments and analysis in the above response (to 8mtA-W1) can **resolve the reviewer's concern about experimental limitations**. Our method surpass all existing RIS models, demonstrating its superiority.\n\nSecond, we want to clarify that our goal is to **resolve referential ambiguity**, a fundamental challenge in real-world RIS scenarios. Previous methods struggle in such scenarios due to their **reliance on keyword-matching modeling**. Instead, we **jump out of such paradigm** and propose our Saccade-Fixation mechanism to address the issue. To further demonstrate this point, a more challenging benchmark aRefCOCO is proposed besides traditional benchmarks.\n\n>**8mtA-Q2:** clarify whether the first SFLayer uses the final-layer features from Swin and BERT, and whether use LAVT’s cross-modal fusion in the backbone stage\n\n**8mtA-Q2-A:** Yes, the first SFLayer uses only the final-layer features from both the Swin Transformer and BERT. We also adopt LAVT’s cross-modal fusion in the backbone stage as CGFormer (CVPR23), SADLR (AAAI23), and ReLA (CVPR23) do. We will clarify these details in the revision."}, {"review_id": "hqzIW0sLXh", "reviewer": "Reviewer_jD3V", "summary": "This paper proposes a referring image segmentation framework that tackles two challenges: object-distracting expressions and category-implicit expressions. Built upon Mamba, the model first forms a global understanding and then refines it through region-wise inspection. A new benchmark is introduced to evaluate RIS models under ambiguous settings. Experiments show the effectiveness of the proposed method.", "questions": "1. How is the inference speed compared to ReMamber [9]?\n2. What are the failure cases of the proposed method?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n1. The paper is overall well-written and easy to follow.\n2. The identification of object-distracting and category-implicit expression types is well-motivated and grounded in practical RIS limitations.\n3. The introduction of the new benchmark evaluates RIS models under ambiguous referring expressions, which better reflects real-world challenges.\n4. The experiments show the superiority of the proposed method in accuracy and inference speed.\n\n[Weaknesses]\n1. While the Saccade-Fixation design is novel, the rest of the model relies heavily on standard components (Swin-B, BERT). The innovation is primarily in the fusion mechanism rather than in the overall pipeline, which may limit its extensibility to other tasks or modalities without further architectural adjustments.\n\n2. Although the introduced benchmark aRefCOCO increases ambiguity, it is still generated by a VLM, which could inadvertently introduce biases, even with meticulous filtering. The paper lacks evaluations on real-world, human-annotated ambiguous datasets or interactive grounding scenarios, limiting its external validity.\n\n3. The paper could benefit from more analysis on failure cases of the proposed method. This would help clarify its limitations, especially in handling complex descriptions.", "was_revised": true, "final_justification": "The rebuttal addresses most of my concerns. I will maintain my initial score.", "rebuttal": "Thank you for the careful review. We would like to first thank you for valuing the paper’s clarity and our method’s novelty，accuracy and speed—strengths also stressed by R7DT and supported by the promising results noted by Wg1a, while 8mtA further commended the strong gains on ambiguous benchmarks.\n\nBelow, we address each of your comments point‑by‑point and clarify how our revisions and additional experiments resolve the concerns you raised.\n\n>**jD3V-Q1:** How is the inference speed compared to ReMamber?\n\n**jD3V-Q1-A:** We have added a inference speed comparison (by fps) with ReMamber as follows. Our model achieves  **1.46×** speedup over ReMamber. That's because its Twister design involves two operations with relatively large-scale computation. As a result, although we use Swin transformer (vs ReMamber's Mamba-B) as visual backbone, our model outperfom it in inference speed.\n\n| Models   | FPS (image/s) | Speed  |\n| -------- | -------------- | ---------------- |\n| CGFormer | 2.78           | 0.34x            |\n| PSALM    | 2.19           | 0.27x            |\n| ReMamber | 5.64           | 0.69x            |\n| **Ours** | **8.26**       | **1x**               |\n\n>**jD3V-Q2 && jD3V-W3:** What are the failure cases of the proposed method?: The paper could benefit from more analysis on failure cases of the proposed method. This would help clarify its limitations, especially in handling complex descriptions\n\n**jD3V-Q2-A && jD3V-W3-A:** Thank you for your question. We will add detailed visualization of failure cases in revised version. Here, we summarize below the main patterns observed in SaFiRe’s failure cases:\n- **Fine-grained attribute ambiguity.** Failures often occur when multiple candidates share broadly similar attributes but only one matches the intended semantics of the description. In such cases, even subtle differences in attributes (e.g., hand positioning) can carry decisive referential meaning, yet the model may overlook them due to visual similarity.\n*Example: “He crosses his hands while talking to the other person.”* In the image, one man rests one hand over the other, while another interlaces his fingers. Although both are hand-related gestures and might visually resemble “crossed hands,” only the latter accurately captures the intended meaning. The model, however, selects the former, revealing its difficulty in grounding fine-grained action attributes to textual descriptions.\n\n- **Long spatial/relational chain ambiguity.** Another common failure mode arises in texts involving long, compositional chains of spatial or relational references. These expressions require the model to perform multi-step understanding with frequent spatial shifts, jumping between different objects across the scene. A typical symptom is that the grounding process breaks down at one or more intermediate links, leading to cumulative error.\n*Example: “the mug on the tray beside the plate nearest the chef.”* In the image, there are multiple mugs, trays, and plates spread across a crowded table, and the chef is positioned off to one side. There is no obvious shortcut to the target—only by following the full relational chain can the correct object be identified. The model, however, misidentifies an object linked to an incorrect plate early in the chain, revealing its difficulty in maintaining spatial consistency and grounding excessive relational dependencies to textual descriptions.\n\nWe will add the above analysis into revision to clarify the potential future directions.\n\n>**jD3V-W1:** The Saccade-Fixation design is novel; the rest of the model relies heavily on standard components (Swin-B, BERT). The innovation is primarily in the fusion mechanism rather than in the overall pipeline, which may limit its extensibility to other tasks or modalities.\n\n\n**jD3V-W1-A:** \nFirst, we sincerely thank you for recognizing the novelty of our Saccade–Fixation design. We would like to clarify that our choice of Swin-B and BERT as the backbone is primarily to ensure a fair comparison with mainstream methods. Meanwhile, it is a convention to focus on fusion mechanism designing in this field, as many other methods share the same backbones in Table 1 in our paper. \n\nSecond, the Saccade–Fixation mechanism is a general-purpose fusion method; however, this work focuses on the RIS task, and we leave broader exploration to future work. For example, SaFiRe have the potential to be adapted to other task or modalities like Open Vocabulary Segmentation (OVS) by changing the task head with minimal changes, or Audio Visual Segmentation (AVS) by changing the backbones. We will mention such potential way of exploration in the conclusion as future work.\n \n>**jD3V-W2:** Although the introduced benchmark aRefCOCO increases ambiguity, it is still generated by a VLM, which could inadvertently introduce biases, even with meticulous filtering. The paper lacks evaluations on real-world, human-annotated ambiguous datasets or interactive grounding scenarios, limiting its external validity.\n\n**jD3V-W2-A:** We acknowledge this bias concern and have already claimed it in our limitations section (Appendix B). Since this study mainly focuses on the techinical innovation in designing a model to handle the referential ambiguity, we mainly verify our method in the common benchmark and our initial small set, aRefCOCO. For broader evaluations on real-world, and human annotated datasets as well as interactive grounding scenarios, we will conduct the continuous verification and extension in our future explorations."}, {"review_id": "XrrY7y8MbE", "reviewer": "Reviewer_Wg1a", "summary": "The paper presents a novel framework, called SaFiRe, to address the challenges in referring image segmentation: object-distracting expressions and category-implicit expressions. It is based on a Mamba’s scan-then-update function. A new benchmark called aRefCOCO is also introduced for evaluating on ambiguous expressions. Experimental results are promising when compared to existing RIS methods.", "questions": "Please refer to the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n+ Experiment results are promising.\n+ The utilization of state-space sequence modeling for RIS is reasonable.\n\nWeaknesses:\n- The motivation of the paper is kind of vague and the explanation of SaFire’s design is questionable. Even though the authors claim that their `Saccade` and `Fixation` come from cognitive science, it can not convince me that such operation aligns with their original aspects.\n- Also, the novelty of the proposed framework is questionable, as it borrows largely similar designs from VMamba, like the scanning and folding techniques in other SSM works.\n- It is unclear whether the samples selected for the aRefCOCO are valid or not when including VLM (i.e., Qwen-VL-Max) for curation due to the absence of further discussion of it. This would also cause a potential cherry-picking by selecting challenging samples from RefCOCO where the proposed method outperforms and the baselines do not. A good practice is to provide more samples in the Supp. Materials for evaluation.", "was_revised": true, "final_justification": "I appreciate the detailed response provided by the authors. The rebuttal addresses most of my concerns. I will maintain my initial score.", "rebuttal": "Thank you for your time on our submission. We would like to first thank you for acknowledging the promising experimental results and the suitability of state‑space modeling for RIS—results similarly lauded by 8mtA and jD3V, with R7DT additionally emphasizing the methodological novelty behind them.\n\nBelow, we address each of your comments point‑by‑point and clarify how our revisions and additional experiments resolve the concerns you raised.\n\n>**Wg1a-W1:** The motivation of the paper is kind of vague and the explanation of SaFire’s design is questionable. Even though the authors claim that their Saccade and Fixation come from cognitive science, it can not convince me that such operation aligns with their original aspects.\n\n**Wg1a-W1-A**: \nFirst, we would like to clarify that our main goal is to resolve **referential ambiguity**, a fundamental challenge in real-world RIS scenarios. Previous methods struggle in such scenarios due to their reliance on keyword-matching modeling. Instead, we jump out of such paradigm and propose our Saccade-Fixation mechanism to address the issue.\n\nSecond, we are inspired, rather than conducting a bio-faithful simulation to the original concepts. Our Saccade–Fixation mechanism is a computational **abstraction**: Saccade performs global scanning for coarse understanding , while Fixation executes focused, token-level binding and language alignment. \n\nAdditionally, our motivation has also been recognized by Reviewer jD3V as \"**well-motivated and grounded in practical RIS limitations**\" and by Reviewer R7DT as \"**well-conceived**.\"\n\nFurther, as already cited in our paper, two well-established findings ([17], [18]) support the scan–then–inspect pattern that inspires our approach. Below, we provide a detailed table that includes direct citations, explanations, and a mapping of each theory to its concrete implementation in SaFiRe, showing how we are inspired.\n\n| Classical Finding                                                                                                   | Key Quotation                                                                                                                                                                                                                      | Implementation in SaFiRe                                                                                                                                                                   |\n|---------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| **Feature-Integration Theory** (Treisman & Gelade, 1980): Features are registered in parallel, but *objects* emerge only when focal attention serially “fixes” each location. | “Features are registered early, automatically, and in parallel… stimulus locations are processed **serially with focal attention**. Any features present in the same central **fixation** are combined to form a single object.”   | ***Saccade***: Serial sweep over patches to supply coarse candidates; ***Fixation***: binds local visual tokens to the text, providing the “glue” Treisman describes. |\n| **Sentence–Picture Verification Model** (Clark & Chase, 1972): Comparing language to images proceeds through “encode → locate → verify” sub-operations whose latencies add up. | “Sentences and pictures are compared in an **algorithmic series of mental operations, each of which contributes additively to the response latency**; encoding, comparing and responding are serially ordered stages.”             | ***Saccade***: Supplies the encoded, coarse alignment (encode + locate); ***Fixation***: Iteratively verifies and refines those candidates, mirroring the verification stage.                    |\n\nWe appreciate the reviewer's comments and will enrich the introduction part to better clarify our motivation in our technical design.\n\n>**Wg1a-W2:** The novelty of the proposed framework is questionable, as it borrows largely similar designs from VMamba, like the scanning and folding techniques in other SSM works.\n\n**Wg1a-W2-A**: We believe there are some misunderstandings. \n\nVMamba is designed for **single-modal vision tasks**. We use the SS2D block serves as a **fundamental processing unit** in our framework, similar to how attention mechanism functions in other architectures. Our novelty lies in the tailored design of Saccade-Fixation mechanism for tackling **multimodal RIS tasks**: First, in Saccade operation, the visual features are **modulated** by the overall meaning of the text and then are scanned to grasps a quick glimpse of image-text correspondence; Second, the Fixation operation follows a **carefully designed \"group-scan-recover\"** pattern to achive detailed examination of cross-modal alignment. We have conducted comprehensive ablation studies (see Tables 4 and 5) and visual comparisons (see Fig. 3) to validate the effectiveness of our approach.\n\nAdditionally, we clarify that our goal is to better address the RIS task in referial ambiguity, rather than merely designing a new version of Mamba architecture. \n\n>**Wg1a-W3:** It is unclear whether the samples selected for the aRefCOCO are valid or not when including VLM (i.e., Qwen-VL-Max) for curation due to the absence of further discussion of it. This would also cause a potential cherry-picking by selecting challenging samples from RefCOCO where the proposed method outperforms and the baselines do not.\n\n **Wg1a-W3-A**: Sorry for any confusion. The original samples from RefCOCO are randomly selected and re-annotated using Qwen via its API. Our SaFiRe model is **NOT** involved during the entire dataset construct process. For more details about the dataset construction, please refer to **8mtA-W2-A** to Reviewer 8mtA. We will release the code and dataset upon publication."}, {"review_id": "7JOCeXVdh6", "reviewer": "Reviewer_R7DT", "summary": "This paper critiques existing approaches in Referring Image Segmentation (RIS) for their reliance on simple keyword matching, which often fails in ambiguous situations. To address this limitation, the paper introduces a framework called SaFiRe, inspired by human cognitive processes consisting of a global understanding phase (Saccade) followed by a fine-grained refinement phase (Fixation). The architecture utilizes the Mamba state-space model to effectively manage long-range dependencies, aligning with the cognitive scan-then-inspect paradigm. Furthermore, to evaluate model performance under referential ambiguity, the paper presents a new benchmark, aRefCOCO, specifically designed to include challenging cases such as object-distracting and category-implicit expressions. The proposed method consistently outperforms existing approaches across both standard and ambiguous benchmarks.", "questions": "1. Novelty Clarification: How does SaFiRe fundamentally differ from “Reasoning Segmentation” approaches such as LISA? Is the distinction only architectural, or is there a deeper conceptual or functional divergence?\n\n2. Failure Case Insights: While ambiguous expressions are addressed, what patterns emerge in failure cases (e.g., pronoun chains, conflicting attributes)? Are there categories of ambiguity where SaFiRe still struggles?\n\n3. Reasoning Model Benchmarking: Have the authors considered evaluating SaFiRe against approaches that share similar conceptual (i.e., reasoning-centric baselines like LISA or LISA++)? A clearer empirical comparison could help position this work more accurately within the reasoning segmentation landscape.", "limitations": "YES.\nThe authors have adequately outlined both the methodological and ethical limitations of their work. A suggestion to enhance this section would be to include failure case visualizations, which would complement the limitations by providing concrete examples along with appropriate explanations. This addition could help illustrate the specific challenges encountered and further clarify the contexts in which the proposed method may struggle.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper makes an intriguing contribution to the field, with current strengths and potential suggestions outlined below: \nStrengths: \n1. Methodological Novelty: The concept of mapping the two-phase human cognition-inspired “Saccade” for global coarse grounding and “Fixation” for local refinement provides an interesting and well-conceived framework for addressing RIS challenges.\n2. Architectural Efficiency: Leveraging the SFLayer in existing segmentation pipelines through Mamba’s linear complexity enables scalable refinement, which is reflected in improved inference speed during evaluations.\n3. New Benchmark: The identification of oversimplified referring expressions in existing RIS benchmarks and the creation of a dataset focused on evaluating model performance under ambiguous expressions represent a significant advancement in addressing real-world linguistic challenges.\n4. Evaluations and Experimentations: A clear evaluations across various methods on common benchmarks, along with clear ablations conducted in the paper, enhance understanding of the proposed method's advantages and the components contributing to its results.\nWeakness:\n1. Problem Novelty: The problem addressed in this paper closely resembles that of one of the cited papers (LISA), with insufficient articulation of the differences between the two. Clearly distinguishing between these works is crucial to avoid confusion regarding their focus on “reasoning segmentation” under ambiguous language prompts.\n2. Underexplored comparisons: Given the objective of handling ambiguity in text prompts for segmentation, it is essential to explicitly discuss and evaluate reasoning models like LISA and LISA++ to clarify SaFiRe’s position within the wider community.\n3. Benchmark: The availability of aRefCOCO as a test-only dataset could limit its utility within the community. Providing a clear train/val split would enhance its usefulness for future benchmarking efforts.\n4. Quantification of Speedup: While the proposed method's speedup is specified, quantifying it in relation to computational requirements (GFLOPS) would be beneficial for the community.", "was_revised": true, "final_justification": "The rebuttal addresses most of my concerns.", "rebuttal": "Thank you for the thoughtful review. We would like to first thank you for identifying the methodological novelty and efficiency of our Saccade–Fixation design—an assessment that echoes 8mtA’s praise for its robust alignment and jD3V’s recognition of its accuracy and speed, while Wg1a also noted the results to be promising.\n\nBelow, we address each of your comments point‑by‑point and clarify how our revisions and additional experiments resolve the concerns you raised.\n\n>**R7DT-W1 && R7DT-Q1:** What are the fundamental differences between SaFiRe and reasoning segmentation approaches like LISA? \n\n**R7DT-W1-A && R7DT-Q1-A:** \nWe first emphasize that SaFiRe and LISA tackle **fundamentally different tasks**. \nLISA brings up a novel task beyond RIS, while our SaFiRe focuses on bringing current RIS task into a more practical usage. We compare **Existing RIS**, our persuade on **Ideal RIS** and **ReasonSeg** as the following table.\n\n||Existing RIS|Ideal RIS|ReasonSeg|\n|-|-|-|-|\n|Segment target based on text| v|v|v|\n|Simple expression|v|v|v|\n|**Ambiguous expression**|-|**v**|-|\n|Complex reasoning|-|-|v|\n|World knowledge|-|-|v|\n\nIn few words: **(1)** LISA focus on what other RIS models **can NOT** do, while **(2)** we focus on what other RIS models **should do but fail**.\n\n### (1) LISA focus on what other RIS models can NOT do.\n\nThe ReasonSeg task proposed in LISA aims to response queries requiring **complex reasoning or world knowledge**. However, **complex reasoning** is an ability emerged from LLMs' large corpus pertraining; while **world knowledge** also requires large diversity of training data and large parameter size for memorize. LISA utilizes LLaVA (7B) to achieve this. Obviously, these abilities are out of the ability of a standard RIS architecture (typically with Swin+Bert as backbone).\n\n### (2) We focus on what other models should do but fail.\n\nWe focus on the referential ambiguity problem in RIS task: dealing with **object-distracting** or **category-implicit** expression. The cases in our aRefCOCO are still **in RIS’s scope** and an ideal RIS model should complete these cases perfectly. However, **existing RIS models tends to fail** due to their inherent limitations: they models RIS as a \"key word matching problem\". Our work shows that by **jumping out of \"key word matching\" paradigm**, we’re able to solve these hard cases.\n\nAs for LISA, through it brings novel idea and solution about reasoning ability, they still fails on our **\"should do but fail\"** cases. Since they utilize SAM-style architecture for mask producing, which is still inside “key word matching” paradigm.\n\n### (3) Evaluating LISA on our aRefCOCO benchmark.\n\nHere we provide the evaluation results of LISA and LISA++ on aRefCOCO. Notably, while LISA++ is designed to outperform LISA on ReasonSeg tasks, it actually performs worse than LISA on aRefCOCO. \nThis indicates that, pursuing stronger reasoning ability may harm the model's performance on ambiguous expression, further highlights the necessity of introducing such a benchmark.\n\n| Method | oIoU | mIoU | Pr@50 | Pr@70 | Pr@90 |\n|--------|------|------|-------|-------|-------|\n| LISA(7B) | 49.0 | 53.8 | 58.7 | 49.2 | 20.6 |\n| LISA++(7B) | 45.3 | 50.4 | 51.5 | 43.5 | 19.7 |\n| **SaFiRe (Ours)** | **65.1** | **71.4** | **78.9** | **73.6** | **45.2** |\n\nWe appreciate the reviewer's comments and will add above analysis to the revision for better clarity.\n\n\n>**R7DT-Q3:** Benchmarking SaFiRe on the Reasoning Segmentation task\n\n**R7DT-Q3-A:**  As explained above, SaFiRe is not designed for reasoning segmentation, we believe it is unfair and unreasonable to evaluate SaFiRe on the ReasonSeg benchmark, as it does not have access to world knowledge.\n\n\n>**R7DT-Q2:** Failure Case Insights: While ambiguous expressions are addressed, what patterns emerge in failure cases (e.g., pronoun chains, conflicting attributes)? Are there categories of ambiguity where SaFiRe still struggles?\n\n\n**R7DT-Q2-A:** Thank you for your question. We will add detailed visualization of failure cases in revised version. Here, we summarize below the main patterns observed in SaFiRe’s failure cases:\n- **Fine-grained attribute ambiguity.** Failures often occur when multiple candidates share broadly similar attributes but only one matches the intended semantics of the description. In such cases, even subtle differences in attributes (e.g., hand positioning) can carry decisive referential meaning, yet the model may overlook them due to visual similarity.\n*Example: “He crosses his hands while talking to the other person.”* In the image, one man rests one hand over the other, while another interlaces his fingers. Although both are hand-related gestures and might visually resemble “crossed hands,” only the latter accurately captures the intended meaning. The model, however, selects the former, revealing its difficulty in grounding fine-grained action attributes to textual descriptions.\n\n- **Long spatial/relational chain ambiguity.** Another common failure mode arises in texts involving long, compositional chains of spatial or relational references. These expressions require the model to perform multi-step understanding with frequent spatial shifts, jumping between different objects across the scene. A typical symptom is that the grounding process breaks down at one or more intermediate links, leading to cumulative error.\n*Example: “the mug on the tray beside the plate nearest the chef.”* In the image, there are multiple mugs, trays, and plates spread across a crowded table, and the chef is positioned off to one side. There is no obvious shortcut to the target—only by following the full relational chain can the correct object be identified. The model, however, misidentifies an object linked to an incorrect plate early in the chain, revealing its difficulty in maintaining spatial consistency and grounding excessive relational dependencies to textual descriptions.\n\nWe will add the above analysis into revision to clarify the potential future directions.\n\n>**R7DT-W3:** Benchmark: The availability of aRefCOCO as a test-only dataset could limit its utility within the community. Providing a clear train/val split would enhance its usefulness for future benchmarking efforts.\n\n**R7DT-W3-A:** Thank you for your suggestion. The primary goal of our paper is to provide a robust and challenging benchmark for evaluating model performance under referential ambiguity, rather than to construct a large-scale training dataset. We fully agree that establishing a training split for aRefCOCO could further benefit the community and enhance future research. We will consider construct train/val splits in the future.\n\n>**R7DT-W4:** Quantification of Speedup: While the proposed method's speedup is specified, quantifying it in relation to computational requirements (GFLOPS) would be beneficial for the community.\n\n**R7DT-W4-A:** Thank you for your valuable suggestion. We have added GFLOPs results, including both LISA and LISA++ in our comparison. Furthermore, we present the speed comparison in terms of FPS (image-text pair/s).  As shown in the table below, SaFiRe achieves a much lower computational cost and faster inference speed compared to LISA and LISA++, which both require near 5000 GFLOPs (SAM- and LLM-driven) and are much slower. At the same time, SaFiRe delivers significantly better segmentation accuracy on the RIS task.\n\n| Model           | GFLOPs | FPS   |\n|-----------------|--------|-------|\n| CGFormer        | 631    | 2.78  |\n| PSALM           | 455    | 2.19  |\n| LISA-7B         | 4880   | 2.67  |\n| LISA++(7B)      | 4943   | 1.29  |\n| **SaFiRe (Ours)** | **384**    | **8.26** |"}]

Author rebuttal:

Meta-review: The paper introduces SaFiRe, a framework for Referring Image Segmentation (RIS) inspired by human cognitive processes, combining a global “Saccade” stage with a fine-grained “Fixation” stage, supported by Mamba’s scan-update mechanism. It also proposes aRefCOCO, a new benchmark designed to test RIS under ambiguous expressions, including object-distracting and category-implicit scenarios. Experiments on standard datasets and aRefCOCO demonstrate clear improvements over existing methods.

According to reviewers, strengths include the novel cognitive-inspired design, effective use of Mamba for efficient iterative refinement, and the valuable benchmark contribution which targets a genuine gap in RIS evaluation. The paper is generally well written, with ablation studies supporting its claims.

However, several weaknesses remain. Comparisons are incomplete: recent SOTA methods (e.g., EEVG, C3VG, LISA, PromptRIS) are missing, which makes competitiveness less clear. The novelty of SaFiRe’s mechanism is debated, with some components seen as adaptations of existing designs. The generation of aRefCOCO relies partly on VLM outputs without sufficient transparency, raising concerns about reproducibility and dataset validity. Also, the gains over Magnet are sometimes marginal, and distinctions from related reasoning segmentation frameworks (e.g., LISA) are not fully articulated.

During rebuttal, the authors clarified aspects of model design and dataset construction, and addressed fairness of comparisons, though concerns about broader benchmarking and dataset documentation were only partially resolved. On balance, I think the methodological idea and benchmark are timely and solid despite limitations. The contributions warrant acceptance, but revisions should strengthen all issues aforementioned.
Decision: Accept (poster)