AMLRIS: Alignment-aware Masked Learning for Referring Image Segmentation
Authors: Tongfei Chen
Shuo Yang
Yuguang Yang
Linlin Yang
Runtang Guo
Changbai Li
He Long
Chunyu Xie
Dawei Leng
Baochang Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Reference Image Segmentation， Masked Learning， VLM
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Referring Image Segmentation (RIS) aims to segment the object in an image uniquely referred to by a natural language expression. However, RIS training often contains hard-to-align and instance-specific visual signals; optimizing on such pixels injects misleading gradients and drives the model in the wrong direction. By explicitly estimating pixel-level vision–language alignment, the learner can suppress low-alignment regions, concentrate on reliable cues, and acquire more generalizable alignment features.
In this paper, we propose Alignment-Aware Masked Learning (AML), a simple yet effective training strategy that quantifies region–referent alignment (PMME) and filters out unreliable pixels during optimization (AFM). Specifically, each sample first computes a similarity map between visual and textual features, and then masks out pixels falling below an adaptive similarity threshold, thereby excluding poorly aligned regions from the training process. AML does not require architectural changes and incurs no inference overhead, directing attention to the areas aligned with the textual description. Experiments on the RefCOCO (vanilla/+/g) datasets show that AML achieves state-of-the-art results across all 8 splits, and beyond improving RIS performance, AML also enhances the model’s robustness to diverse descriptions and scenarios.

Published as a conference paper at ICLR 2026
AMLRIS: ALIGNMENT-AWARE MASKED LEARNING FOR REFERRING IMAGE SEGMENTATION
Tongfei Chen', Shuo Yang"; Yuguang Yang Linlin Yang"; Runtang Guo?, Changbai Li,, He Long', Chunyu Xie Dawei Leng"! Baochang Zhang
"School of Artificial Intelligence, Beihang University, China
2360 AI Research, Qihoo 360, China
School of Electronic Information Engineering, Beihang University, China
*State Key Laboratory of Media Convergence and Communication, China Communication University of China, China
'Hangzhou International Innovation Institute, Beihang University, China
IInttitute of Unmanned System, Beihang University, China
"School of Automation Science and Electrical Engincering, Beihang University, China
ABSTRACT
Referring Image Segmentation (RIS) aims to segment the object in an image uniquely referred by natural language expression. However, RIS training of- ten contains hard-to-align and instance-specific visual signals; optimizing on such pixels injects misleading gradients and drives the model in the wrong direction. By explicitly estimating pixel-level dision-languee suppress low-alignment regions, concentrate on reliable cues, and acquire more generalizable alignment features. In this paper, we propose Alignment-Aware Masked Learning (AML), a simple yet effective training strategy that quantifies eggion-refereet alignment (PMME) and filters out unreliable pixels during op- timization (AFM). Specifically, each sample first computes a similarity map be- tween visual and textual features, and then masks pixels falling below adap- tive similarity threshold, thereby excluding poorly aligned regions from the train- ing process. AML does require architectural changes and incurs inference overhead, directing attention to the areas aligned with the textual description. Ex- periments on the RefCOCO (vanilla/+/g) datasets show that AML achieves state- th-the-and art results across all 8 splits, and beyond improving RIS performance, AML also enhances the model's robustness to diverse descriptions and scenarios. Code is available at
alignment, the learner can
INTRODUCTION
Referring Image Segmentation (RIS) (Hu et al., 2016 Cheng et al. 2014) aims to segment the ob- ject in an image that uniquely identified by a natural language expression. Unlike conventional segmentation tasks, RIS requires precise crosl reasoning under sparse supervision each training sample typically includes only one annotated object. However, resolving such expressions often depends understanding the visual context surrounding the referent, including other objects, spatial relations, and appearance-based contrasts. For instance, identifying "the giraffe closest to people" (Fig. (1) requires distinguishing multiple similar objects based proximity, while resolv- ing "lower broccoli" (Fig. [T)) requires fine- grained attention to localized attributes and utilize this information to determine the full target entity. This contextual grounding crucial, yet difficult to learn from limited pixel-level supervision.
"Equal contribution. (troruttiioonnnennnnn.. lyng, guangbuaa| buaa. edu.en "Corresoondigg author. yang@cuc. edu. en, x echunyuu 360.cn Project leader.

---

Published as a conference paper at ICLR 2026
(a) Description: "givaffe closest people"
(c) Benchmark Comparison over & Splits (d) Cross-Dataset Robustness (? Scenarios)
Ruroco
Refonce-val (mlol)
(b) Description: "lower broccoli"
Ball OK
OGFNeRNT PGRD
CARDS
Ground Truth
CARIS
AMI RIS
Figure Overall results of AMLRIS, including qualitative examples, benchmark comparisons (oloU), and cross-dataset robustness evaluation, where the model is trained only on RefCOCO+ and evaluated on RefCOCOg and RefCOCO under seven perturbation scenarios.
2022 Zhang et al. 2022; Ding et al. 2022] Liu et al. 2023a] Zhao et al., [023] Xie et al. 2023 Zhou et al. 2024 Yang al. 2025 Xie al. 2025) have introduced architectural modules to improve cross- modal alignment. LAVT (ang et al. 2022) incorporates pixel-word attention in the encoder to fuse visual and linguistic cues early. CARIS (Liu et al. 2023c) enhances decoding with bidirectional cross-attention and contextual prompts, while DETRIS (Huang et al. (205)) promotes hierarchical alignment through densely connected language adapters, allowing language signals to influence visual representations across multiple semantic levels. These designs aim compensate for weak supervision by strengthening vision-languagn interaction. However, without reliable supervision beyond the referred object, modules trained under dense loss may overfit to unrelated regions, allowing misaligned gradients to dominate training.
To address the supervision bottleneck, prior works (Wang al. 2022a Yang
To address the above limitations, propose Alignment-Award Masked Learning (AML), simple yet effective training strategy that improves RIS by selectively filtering out unreliable pixels during optimization. The core idea is decouple learning from poorly aligned regions, allowing the model to concentrate gradient updates on more trustworthy visual-textual correspondences. Therefore, in the first forward, we introduce PatchMax Matching Evaluation (PMME) to compute fine-grained similarity map, where each visual patch is matched its most similar language token to quantify patch-text alignment. Note that in many RIS architectures, the vision and language backbones are not jointly pretrained and often output features with mismatched dimensionalities, To enable meaningful similarity computation, we introduce Johnson-Lindenn Lindenstrauss random projec- tion that maps both modalities into common embedding space. This projection preserves pairwise distances and angular structures with high probability, ensuring that the cross-modal geometry re- mains intact while aligning feature dimensions (we have provided thorough theorem for this in Sec. 3.2).
Subsequently, based the similarity map, we construct an Alignment-Aware Filtering Mask (AFM), masking out pixels whose similarity falls below a pre-set threshold. The model is then optimized on the remaining well-aligned regions in the second forward. AML improves optimiza- tion stability by filtering out weak learning signals early in training. As noted by [Zeng et al. (2023), this reduces the search space and prevents overfitting to misaligned regions. In RIS, where fine-grained alignment is key to distinguishing similar objects, AML guides attention toward the most relevant regions-for example, focusing on discriminating the correct individual in expressions like "giraffe closest to people" (Fig. IT].
Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate that AML consis- tently improves performance across multiple backbones and achieves state-of-the-art results. More- over, we show that AML enhances model robustness in occluded or noisy conditions, confirming the value of alignment-awant supervision under limited annotations. Our contributions can be sum- marized as follows:
We propose an Alignment-Awen Aware Masked Learning (AML) framework, which selectively filters out poorly aligned pixels based patch-level cross-modal similarity map.
2

---

Published as a conference paper at ICLR 2026
We introduce PatchMax Matching Evaluation (PMME) to quantify cross modal feature alignment and Alignment-aware Filtering Masking (AFM) to enable fine-
fine-grained grained region selection.
Extensive experiments on RefCoco, RefCOCO+, and RefCOCOg demonstrate that AMLRIS achieves state-of-the-art performance across all 8 splits with consistent improvements over previ- ous SOTA methods (e) and exhibits superior cross-dataset robustness, surpassing the base- line under diverse perturbation scenarios (Fig. Tt).
RELATED WORK
Referring Image Segmentation. Referring Image Segmentation (RIS) aims to segment a target object in image based a referring natural language expression. Most methods (Wang & Ye, 2024 Wu et al. [025] Shi et al., 2018, Chen et al.] (019) follow a two-stage paradigm: visual and linguistic features first extracted separately and then fused for segmentation. Depending on the fusion stage, RIS methods can be categorized into late fusion and early fusion approaches. Late fu- sion methods perform cross-modal alignment after feature extraction, typically within the decoder.
VLT (Ding et al. 2022). CARIS (Liu et al. 2023c), and ReSTR (Kim et al.) (2022) adopt cross- attention to connect visual patches and language tokens, while CRIS (Wang et | 20226) leverages CLIP for pixel-level adaptation. However, these methods often suffer from limited vision- vision-language interaction and fail establish trustworthy image region-context correspondences. Early fusion methods inject language features into the visual encoder to enable language-aware representation during encoding. LAVT (Yang et al., 2022) introduces pixel-word attention, CGFormer (Tang et al.) 2023) uses gated cross-modal fusion, and DETRIS (Huang al. 2025) distributes lightweight lan- guage adapters throughout the visual backbone. These designs (Li et al.
2025 Yang et al. 2022 Huang et al. 2025 Ding al. 2022: Liu et al. [023c] Kim al. [2022] Wang et al. [022b Yang
et al. 2022; lang et al. 2023: Huang et al. [2055] Long et al.] 2025 Zhou et 2025 Yang et al.
2024a) strengthen lision-language coupling and enhance referential grounding, especially in com-
plex scenes.
However, existing methods attempt to model all spatial and semantic relationships through increas- ingly intricate fusion mechanisms, implicitly assuming that all regions are equally informative. In contrast, our approach takes the opposite view: rather than modeling every relation, we first clim- inate regions that are poorly aligned with the expression. By masking out these ambiguous areas, AML allows the model to focus more on trustworthy regions and learn referential cues from clearer, more consistent visual-textual correspondences.
Masking- Masking-based Data Augmentation. More recently, data-centric approaches (Cubuk et al. 2018
2020 Touvron et al. 202 Lee et al., 2024 Chng et al.) 2024 Ha et al. 2024) have emerged to supplement the limited training signal. However, designing effective data augmentation for RIS poses unique challenges. Standard transformations used in vision tasks (Shorten & Khoshgoftaar
2019
Alomar et al., 2023 Perez & Wang 2017; Yu et al. 2024 Lyu al.) 2024), such as hori- zontal flipping or color jittering, can easily violate the referential integrity of the expression. For instance, flipping the image invalidates spatial expressions like "on the left", while color jitter may distort expressions such as "the woman in red". Unlike classification, RIS requires that any im- age transformation preserve the semantic link between expression and referent. To address this, MaskRIS {Lee et al. 2024) introduces distortion- aware augmentation strategies that avoid opera- tions likely to break semantic consistency. Other efforts (Chng al. (024)) leverage masking- based training signals: Mask Grounding applies token-level masking on the text side and requires the model to recover masked words using visual cues, thereby enhancing alignment between image re- gions and key linguistic units. NeMo (Ha et al.) 2024) adopts a vision-side strategy, embedding hard negative distractors into mosaic images to increase discrimination difficulty and force the model to resolve fine-grained ambiguity.
Although these methods enhance supervision, they still apply full-pixel loss, allowing gradients from irrelevant or misaligned regions to dominate training. In contrast, AML directly suppresses noisy gradients by masking out poorly aligned pixels based on patch-token similarity, guiding learning toward reliable regions and improving referent grounding under sparse supervision.
3

---

Published as a conference paper at ICLR 2026
Original Image
Image Features
Image Encoder
Seg
Masked Image
Similarity
Feature Fusion
AFM
PMME
Mask Decoder
neg/pos Prediction
Text Features
First Forward
"bowl darker sandwich side"
Text Encoder
Second Forward
Figure 2: Overview of Alignment-aware Masked Learning (AML) framework.
METHOD
3. PRELIMINARY
RIS Formulation. Given an image E mxxwax and referring expression T {Wi, W2,.... w of length L, the goal RIS predict a binary mask M E 10, where each pixel indicates whether it belongs to the referred object.
Baseline Model. We build upon CARIS (Liu et 2023c). a recent context-aware baseline for RIS. CARIS adopts dual-branch architecture with a B image encoder & and a BERT-based text encoder e extract multi-scale visual features (Vihe
1K k= at K scales and token embeddings with D,-dimensional Teme E IR NRx,,, respectively. Then, a mask decoder is used produce the positive region prediction Mpas and negative region prediction Mneg. And the probabilities of the position (i, )) for the referred object is:
exp(M (3.1))
p(i.)
(1)
exp(M (b)) + exp M (t.1)
The loss function calculated as follows:
H W
Gaeg
y (63) log plis) + (1 -1 (655) log(1 p(is))]
(2)
HW
where ((+))) = 1 for positive pixels and ( = for negative pixels.
3.2 PATCHMAX MATCHING EVALUATION
PMME with Random Projection Alignment. To diagnose the modality gap between vision and language representations in referring image segmentation (RIS), we introduce PatchMax Matching Evaluation (PMME), which quantifies the semantic alignment between visual patches and language tokens.
key challenge lies in the dimensional mismatch between visual and language features, which makes direct similarity computation infeasible. To address this, project both modalities into a common embedding space via modality-specifie random linear mappings (Achlioptas) 2003)) Such projections approximately preserve pairwise distances and inner products with high probabil- ity, allowing reliable similarity computation the embedding space (see Theoreml). Specifically, given the deepest layer of visual features VK {um " E H and textual features ene
Tene Ju E IRN: Nt,,, perform flattening operation followed by an Ey-normalization process:
6. Un/ /am////mllll. ūn, un, /u////unl|2
(3)
4

---

Published as a conference paper at ICLR 2026
N, D2
Ni D2
I1 x W
x W
T-Linear Projection
suffuuer
R(P)
Text Fratunn
Dealigned I-iistturs
Similurtt Scares
Poor-alianad Plachs
Random Seleciion
MAN W
bhooking
My
Hp
I-Lincar Projection
upsamppe
Maak Imags
Tmagee Features
Dallgned I.Peatures
PanchMa Sinilesity
Piud Sinilarity
Misk Blocks
Figure 3: Architecture for PMME and AFM modules.
H and T = [ūn J Then we project them into and thus we have the normalized V {ūm J Jm=l D. Do-dimensional space using i.i.d. gaussian matrices W; E BDDxD and W, E D,xD.
V' Vw, T' =T=W,T
(4)
[ ((O,11/D.) [Welpg (=(00,1/.)
(5)
where [Wilpg- [Welog R are the elements for W; and Wt, respectively, Then we perform dot product and SoftMax for our raw alignment matrix:
Shar SoftMax(V""TT) (H
where SoftMax applied across each row. This yields probability distribution over all language tokens for each visual patch, quantifying their alignment confidence. To derive fine-grained patch- level alignment signal, we select for each visual patch its most strongly aligned token:
5(155 max 1,
((i.j.) norm
(H)))
where ((1.5) [0, 1] indicates the peak alignment confidence of patch (i,j). The resulting metric E xH, serves as a fine-grained alignment heatmap, which is later used to filter unreliable regions during training. Furthermore, we rigorously demonstrate the validity of this computation.
Theoreml Cross-Modal Inner Product Preservation via Gaussian Random Mapping. Let Z = Amn [8m, " D.+. } the joint embedding space. A block-diagonal projection can be
defined as:
W = /2 diag WW, W.) E ((D++D))x2D.
(8)
Then, for any distortion (0, 1) and failure probability (0, 1),
D2 2 (oog(H,,,
then with probability least 1 σ,, the projection W preserves all pairwise distances in Z:
(WGum, Wūū,)) (im, ū.)) s, V m,n. n.
(10)
This theorem ensures that all pairwise dot products between the joint embeddings (Vm, u,,) approximately preserved under the random block-diagonal projection W. The guarantee holds uni- formly with high probability long as the projection dimension D, satisfies Eq. 9 The thorough proof can be found in Appendix F
3.3 ALIGNMENT- AWWAR FILTERING MASKING (AFM)
To assess fine- fineggrained risual-languee alignment, first upsample the patch-level similarity matrix S E HW, to the original image resolution using bilinear interpolation, yielding a dense pixel- wise similarity map Spisel HW
Spizel BillnearUpsample(())
(11)

---

Published as a conference paper at ICLR 2026
This upsampling step enhances spatial consistency by propagating alignment scores to neighboring pixels. We then identify weakly aligned pixels those with similarity below a threshold 7 7-and collect their positions into candidate set:
weak
{(m, n) E [] x W]
slmm.) pixel < TT.
(12)
To prevent over-filtering and encourage generalization, we randomly retain proportion 1 - p of these weak pixels:
sseeectdd DropOut (Pweak:
(13)
Next, we aggregate the selected pixel-level mask into patch-level binary blocks. The image is par- titioned into non-overlapping 3hth 8 blocks, where 3a and 8 refer to the height and width of each image patch, each corresponding to a downsampled mask element:
(p-9)
max [[(m, n)) E Pselocted); (m.,))EBr.o)
(14)
where Blp-9) [p (p + 1) x Lg (q 1) denotes the pixel range of the (p, q)-th
block, and [[-] is the indicator function. The max operator ensures that block is masked if any its constituent pixels are poorly aligned, following conservative "any-triggers-all" policy to suppress noisy gradients.
Finally, apply this binary mask to the imput image, zeroing out the selected blocks:
i(B(x-)) (BB())) (1 plock
(15)
where @ denotes dlment-wwise multiplication. This masking suppresses gradients from unreliable regions, allowing the model to focus well-aligned areas during training, ultimately improving the overall alignment quality (see Appendix E..
3.4
AML TRAINING FRAMEWORK
To integrate alignment-ave aware augmentation while maintaining compatibility with the baseline opti- mization, as shown in Fig. 2
we adopt two-stage training scheme with shared model parameters
across stages.
In the first stage, the original image and referring expression T are passed through the vision and language encoders to compute the similarity map S and derive the binary mask Motect via our alignment-aware filtering. The masked image I is constructed by zeroing out poorly aligned re- gions. This stage is forward-only, incurs gradient computation, and adds slight overhead (a 4.9% memory and 17.2% training time overhead compared with CARIS, detailed in the Appendix G.2).
In the second stage, i and T are fed into the baseline model to perform standard segmentation and compute loss. Only this stage updates the model parameters. Since both stages share weights and the masking stage is lightweight, the total training cost remains comparable to the baseline.
This framework enables the model to benefit from alignment-inforned masking while preserving training efficiency and objective consistency. Notably, during inference, the masking stage skipped, and the model operates on the original input. Despite never being trained on unmasked images, the model exhibits improved generalization and robustness, outperforming both the baseline and random masking variants (see Figure
4 EXPERIMENT
4.1 EXPERIMENTAL SETUP
Benchmark Metric. We evaluate our method on three widely adopted Referring Image Segmen- tation (RIS) benchmarks: RefCOCO, RefCOCO+, and RefCOCOg. These datasets provide diverse challenges in terms of expression complexity and object density (See Appendix A.1 for detailed introduction). And adopt overall intersection-vve--nniooo (oloU), mean Intersection- Union (mloU), and Precision at loU thresholds (P@X X e {0.5, 0.7, 0.9}) to evaluate the predicted mask (see Appendix

---

Published as a conference paper at ICLR 2026
Algorithm 1 Alignment-Awart Masked Learning (AML) for RIS
1: Input:
training set Gaussian matrices W. E RDixDs W, E RDxxDs; threshold т; dropout ratio block size (Bh, B.).
image encoder o,; text encoder Φ5; RIS model fo: random
2: Output: updated parameters 0.
3: for each mini-batch (I, Y) do
4: / Stage I: PMME-based alignment-awen -aware masking (forward-only) %/
5:
VK - (I( I/ deepest visual feature, flattened RR x.D.
6:
Tene + (T) token features R. N.D:
7:
V - fg-normalize(V enc
π T f-g-normalize(T coc)
8:
V + VWT. T TW, /1 project Da-dim space
9:
Snorm < SoftMax(V"T")))
row-wise over tokens
10: For each patch (i, compute ((6.)) maxIckSN. (.....)
Daom
11: Spizel + BilinearUpsample((
1/ Spicel E R
12:
Pweak + [mm, n) pixel
s(man) <T}
13:
Pselected < DropOut (Pweak: p)
14: Initialize block mask M(P.9) + 0 for all blocks.
Partition the image into non-overlapping blocks of size Bh x B".
16:
for each block (p, q) do
17:
Define B(p.9) = [p/h, (p + 1) x [q (q + 1)
18:
there exists (m, n) E B(r-a) such that (m, E Peeeettdd then
M(P.2)
19:
20: end if
end for
22: I +
for each block (p. q) do
24:
if H.Mi = 1 then [(B(r-a) <- 0 // zero out poorly aligned blocks end if end for
25:
26:
27:
28:
1 Stage H: RIS training on masked image #/
29:
(Mpous Mneg) < fI(I.T)
30:
Compute p(i.)) as in Eq. (1).
Compute Cyey as Eq. (2) using Y.
32:
Update 0 +- 0 V V
33:
end for
Training Details. We integrate AML into several representative RIS frameworks, including DE- TRIS (Huang et al. (025), CARIS (Liu et al. 2023c) and ReLA (Liu et al. 2023a), Without special notation, the baseline model is CARIS. We set T = 0.4, p 0.25, HI WW x2x322 D. = 2048. More training details Appendix C
4.2 COMPARATIVE EXPERIMENT ON RIS DATASETS
CARIS with AML achieves new SOTA performance on the RIS task. As shown in Table our method consistently outperforms existing approaches across RefCOCo, RefCOCO+, RefCOCOg, and the combined setting, in terms of both mloU and oloU. On RefCoCo, CARIS+AML improves over the baseline by +1.12% +0.50% and +0.43% mloU, and +0.80% +0.50% and +0.42% oloU on val/testA/testtB, respectively. On RefCOCO+, we observe consistent gains: +2.00% +1.10%, and + +1.92% mloU, and +1,83% +1.33% and +1.54% oloU. On RefCOCOg, CARIS+AML surpasses the baseline by +1. 1.22% (test) in mloU, and +1.78% in oloU. These consistent improvements validate the effectiveness of AML in enhancing semantic precision and spatial consistency. especially under complex linguistic expressions. Moreover, under the combined training setting, approach further improves upon the baseline by +0.53% (RefCOCO val), +0.96% (RefCOCO+ val), and +1 +1.40% (RefCOCOg val) in oloU, while maintaining superior performance across all test splits.
7

---

Published as a conference paper at ICLR 2026
Deseripiio:: "wediuu slzed yase"
()) Desciptioe: "womsee sieeees
Baseliz
Basclie
(a) Comparison of prediction maps: Baseline, Random Mask (RM), and AML, Brighter regions indicate higher scores. cation is predicted as target when pos >> neg.
(h) Comparative performances with dif- ferent mask stralegies using occluded im- ages on RefCOCO+.
Figure 4: Qualitative and quantitative comparison with the Random Mask strategy.
Table I:
Comparison with SOTA methods. * denotes the reproduced results across all experiments. Bold and underline numbers indicate the best and the second best performance. The last column is average score across available splits. Gray texts denote large language model based methods.
Method
RefCOCO
RefCOCO+
RefCOCOg
Avg
val
lestA
testB
val
testA
testB
test
Standard: Training on the training split of each dataset.
mIoU
CRIS [Wang et al.
2022a)
70.47
73.18
66.10
62.27
68.08
53.68
59.87
60.36
64.3
ETRIS Xu et al. 2023
71.06
74.11
66.66
62.23
68.51
52.79
60.28
60.42
64.5
BarLeRla Wang el al. 2024)
72.40
75.90
68.30
65.00
70.80
56.90
LAVT Yang
2022
74.46
76.89
70.94
65.81
70.97
59.23
63.34
63.62
68.0
DETRIS Huang al.
202.5
75.64
77.39
73.11
68.47
73.03
60.00
CGFormer Tang ct al. 202.1
76.93
78.70
73.32
68.56
73.76
61.72
67.57
67.83
71.1
PGBD (Wu ct al. 2025
76.45
78.57
73.14
68.82
73.84
61.56
68.42
67.99
71.1
CARIS Liu el 2023c
76.77
79.03
74.56
69.33
74.51
62.69
68.87
68.51
71.8
MagNet (
Chnn
2025
77.43
79.43
74.11
70.10
74.50
63.59
68.53
69.15
72.1
Ours
77.89
79.53
74.99
71.33
75.61
64.61
69.24
69.73
72.9
oloU
LAVT Yang et al.
2022
72.73
75.82
68.79
62.14
68.38
55.10
61.24
62.09
65.8
DETRIS (Huang el
2025
74.05
76.68
71.33
65.39
70.82
$5.82
NeMo Ha al 2024
74.48
76.32
71.51
62.86
69.92
55.56
64.40
64.80
ReMamber Yane et al
2024b
74.54
76.74
70.89
65.00
70.78
57.53
63.90
64.00
67.9
CGForer Tang et al.
2023)
74.75
77.30
70.64
64.54
71.00
57.14
64.68
65.09
68.1
PGBD Wu al. 2025
74.43
76.89
70.91
65.27
71.26
56.98
65.21
66.31
68.4
CARIS L.u et al.
2023c
74.65
77.83
71.70
65.54
71.86
57.97
65.15
65.00
68.7
MagNet CChng ct al.
2024
75.24
78.24
71.05
66.16
71.32
58.
65.36
66.03
Ours
75.45
78.33
72.12
67.37
73.19
59.51
65.67
66.78
69.8
Combined: Training the combination three datasets.
oloU
PixelLM-7B Ren et al. 2024
73.0
76.5
68.2
66.3
71.7
58.3
69.3
70.5
69.2
GSVA-7B Xia et
2024
76.4
77.4
72.8
64.5
67.7
58.6
71.1
72.0
70.1
PerceptionGPTT-7 Ren ct
2024
75.1
78.6
71.7
68.5
73.9
61.3
70.3
71.7
71.4
PolyFormer (Liu et al. 20236
74.82
76.64
71.06
67.64
72.89
59.33
67.76
69.05
69.2
CARIS [Liu et al. 2023c
76.67
78.24
71.32
67.64
72.77
60.87
67.44
69.17
MagNet CChng et al. Ours
2024
76.55
78.27
72.15
68.10 10
73.64
61.81
67.79
69.29
70.9
77.20
79.53
73.61
68.60
73.76
61.55
68.84
70.01
71.9
Effectiveness Across Multiple Baselines. To demonstrate the generality and early-stage efficiency (effectiveness in the initial few epochs) of AMI., we apply it to multiple representative RIS baselines. As shown in Table 2 AML consistently improves oloU across all RefCOCO splits, boosting CARIS by +1.83% +1.33% +1.54%, and DETRIS by +0.80% +0.38% +0.98% on val, testA, and testB, respectively.
Effectiveness the Early Training Stage. In multi-object grounding experiments on the GRES dataset, integrating AML into ReLA yields consistent carly-stage benefits (at 4999 iterations, Appendix G.33, AML+ReLA reaches 30.86 cloU and 21.08 gloU, compared to 28.49 and 17.33 for vanilla ReLA (Table 3). Similarly, under the same ten-epoch protocol MagNet (Chng al.
8

---

Published as a conference paper at ICLR 2026
Table AML with more baseline models on Re- fCOCO+ (oloU).
Table 3: Early-stage comparisons across datasets and metrics.
Imuge Eneoder
Text Encoder
Method
(Metric)
Method
val
lestA
testB
Single-object
RefCOCO (oloU)
RefCOCO+ (oloU)
Fully-supervieed (aloU)
CARIS
Swin B
BERT
65.54
71.86
57.97
MagNet
68.52
57.26
+AML
Swin-B
BERT
67.37
73.19
59.51
Ours
71.83 (+3.31)
63.79 (+5.53)
Purameter Effciient (oloU)
Multi-object
GRES (cloU)
GRES (gloU)
DETRIS DINOv2.B
CLIP
65.39
70.82
55.82
ReLA
DINOv2-B
CLIP
Ours
30.86 (+2.37)
21.08 (+3.75)
Table 4: Threshold (r) ablation on val splits.
Table 6: PMME phase ablation on RefCOCO+
RefCOCO
RefCOCO+
ReFCOCOg
P@0.5
P@007
O (CARIS)
74.65
63.54
65.15
CARIS
78.85
71.76
33.48
63.54
75.00
65.78
76.98
69.67
33.66
0.3
75.81
66.82
65.13
Prod PMME
79.57
73.02
35.40
66.41
0.4
75.45
67.37
65.67
Early PMME
80.82
73.85
36.35
67.37
Table 5: Projection dimension (Da) ablation.
Table 7: PMME projection strategies (epoch 1).
Dim (Da)
oloU
mloU
Pro:50
Pr@90
Error
Method
Pe005
P@0.77
P600.
oloU
70.83
80.15
0.2955
CARIS
8.66
2.70
0.25
20.24
2048
67.37
71.33
80.82
36.35
0,2088
Learnable
10.36
6.02
0.92
16.92
4096
67.08
71.21
80.85
36,22
0.1478
Random 17.35
9.40
1.33
24.85
2024), our method(AML+ achieves 71.83% RefCOCO and 43.79% on RefCOCO+, sur- passing MagNet by +3.31% and +5.53% (Table 3j. These improvements stem from discarding poorly aligned regions the early training stage, which guides the optimization toward reliable correspondences (see Appendix G.4 and consistently boosts performance across both multi-object and single-object settings.
4.3 ROBUSTNESS UNDER CROSS-DATASET AND VISUAL PERTURBATIONS
Setup. To evaluate scene robustness, we train models on RefCOCO+ and test them on RefCOCO and RefCOCOg under multiple visual perturbations. The perturbation types cover illumination shifts, occlusion, and other scene-level variations, as detailed in Appendix G.5
Results. As shown in Figure Dd. AMI. consistently improves resilience across settings. On Ref- COCO, the average mloU increases from 58.98% to 62.48% (+3.50). On RefCOCOg, the average mloU rises from 51.36% to 53.69% (+2.34). These results show that AML can handle varied tex- tual descriptions across datasets while resisting diverse visual disturbances, leading to more reliable performance under real-world scene shifts.
4.4 ABLATION & ANALYSIS
Ablation on т. As shown in Table 4 fixing p 0.25 (see Appendix G.6), small thresholds (r 0.4)) yield consistent gains: on RefCOCO, mloU increases from 74.65% (r=0) to 75.00% (r=0.2) and 75.45% (r=0.4); on RefCOCO+, from 65.54% to 66.99%. and 67.37%. RefCOCOg also improves over baseline, peaking at 65.78% with r=0.2 (65.67% at 0-4.4). These results suggest that small effectively filters poorly aligned regions, and that near-zero alignment can hinder optimization. Unless otherwise noted, we adopt 7=0.4 average-best choice across datasets.
Ablation on D.. As shown in Table 5 we compare D. on RefCOCO+: moving from 1024- 2048 yields consistent gains (oloU 66.82% + 67.37%, mloU 70.83% - 71.33%. Pr@50 80.15% + 80.82%. Pr@90 36.27% 3). Pushing to 4096 brings only marginal changes. Meanwhile, the JL distortion bound decreases monotonically (0.2955 + 0.2088 + 0.1478), indicating better geometry preservation with larger D.,. Balancing accuracy and scalability, adopt D,, 2048

---

Published as a conference paper at ICLR 2026
as the default, offering favorable trade-off with controlled bound (c 0.2088) while nearing saturation on the main metrics.
Ablation Study Alignment Feature Stage in PMME. We design three variants within the AML framework, as shown in Table 6 Late PMME (computes similarity after fusion operation, where features are highly entangled) performs the worst, likely due to the absence of semantic anchors when relying solely on entangled fused features. Pred PMME (uses the first-stage predicted mask as a coarse visual prior) ranks second, outperforming Late PMME in P@0.9 and oloU, suggesting that even coarse localization guidance mitigates semantic misalignment. Early PMME (computes similarity using visual and textual features before cross- modsll fusion) achieves the best performance across all metrics, improving P@0.9 and oloU by +2.87% and +1.83% over the baseline, respec- tively. This validates computing alignment using clean, unimodal features before fusion preserves more accurate patch-token correspondences and enhances segmentation quality.
Ablation Study on Projection Strategie in PMME. We introduce a learnable projection variant of PMME for ablation, using trainable linear layers jointly optimized with the main network. Apart from the projection module, all other network components and training settings remain identical to ensure fair comparison. Implementation details are provided in Section strategies at the first training epoch to examine their influence on early-stage semantic alignment. As shown in Table 7 random projection outperforms both the CARIS baseline and the learnable variant by +7.9% and +4.6% in oloU, respectively. This advantage is attributed to the stability of fixed mappings, which effectively suppress gradient noise and preserve moosl geometry, leading to faster and more reliable optimization.
We assess the two
Robustness to Occluded Inputs. To evaluate robustness under occlusion, we masked 0%, 30%, and 50% of image areas during inference. Compared to the control variant (CARIS+random mask), as shown in Figure 4b) AML consistently exhibits the smallest performance drops. On testA, AML reduced the oloU drop from 9.4% to 3.7%, demonstrating superior robustness against degraded inputs compared to CARIS.
Comparison of the Visualized Prediction Maps. Following the robustness experimental setup with three compared methods, we visualize their negative and positive activation maps (bright re- gions indicate high response scores) in Figure 4a For the same referring image, the baseline pro- duces blurred prediction boundaries with activations leaking to semantically similar adjacent objects, while the random masking variant shows improved edge sharpness but still fails to precisely localize the correct mask position. In contrast, our method generates crisply defined boundaries and effee- tively filters out competing objects through negative activation suppression, demonstrating superior performance in both boundary precision and semantic discrimination. The observed characteris- ties indicate our method's superior semantic boundary modeling, which directly translates more accurate referent localization and instance segmentation, as evidenced by the P@0.9 gains.
Analysis of Training Overhead and Fairness. Detailed in G.2 CARIS+AML increases training time by 17.2% per epoch but keeps total optimization steps identical. Under equal wall-clock time, CARIŠ+AML. consistently outperforms CARIS (e.g., 66.7/70.8 vs. 65.5/69.3) and matches the 50- epoch CARIS performance in only 30 epochs, proving gains stem from alignment-aware masking rather than additional optimization.
5 CONCLUSION & LIMITATION
We propose Alignment-Awaan Masked Learning (AML), a lightweight training strategy that im- proves referring image segmentation by filtering out poorly aligned pixels based on patch-level similarity. Extensive experiments on the RefCOCO family benchmarks demonstrate that AMI. con- sistently improves performance across different backbones and evaluation splits, without introduc- ing any inference-time overhead or architectural modification. Its plug-and-pla, nature makes AML broadly applicable existing RIS frameworks.
While AML proves broadly effective, it introduces a minor training overhead and shows sensitivity to few hyperparameters. Future work will explore adaptive tuning and lighter variants to stabilize performance. We also plan to extend AML to broader tasks like video understanding and model families like niignment-pretraine foundation models.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This research was supported by the National Natural Science Foundation of China (Grant No. 62550184).
ETHICS STATEMENT
This submission adheres the ICLR Code of Ethics and Code of Conduct. Our work does involve human subjects, crowdsourced annotators, animal experiments, or sensitive personal data (e.g., face face images linked to identity, medical financial records); thus IRB approval is required. All datasets used are publicly available for research under their respective licenses, which we follow without redistributing restricted content.
REPRODUCIBILITY STATEMENT
We provide complete algorithmic and architectural specifications, including loss definitions and hyperparameters, to ensure end-to-end reproducibility. After the review process, we will release the full source code, training/testing scripts, and model checkpoints enable transparent verification.
REFERENCES
Dimitris Achlioptas. Database-frient friendly random projections: Sansson-lnndenstrauss with binary coins. Journal of computer and System Sciences, 66(4))671-687, 2003.
Khaled Alomar, Halil Ibrahim Aysel, and Xiaohao Cai. Data augmentation in classification and segmentation: A survey and new strategies. Journal of Imaging, 9(2):46, 2023.
Ding-Jie Chen, Songhao Jia, Yi-Chen Lo, Hwann-Tzong Chen, and Tyng-Luh Liu. See-through- text grouping for referring image segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7454-7463, 2019.
Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer universal image segmentation. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 1290-1299, 2022.
Ming-Ming Cheng, Shuai Zheng, Wen- Yan Lin, Vibhav Vineet, Paul Sturgess, Nigel Crook, Niloy J Mitra, and Philip Torr. Imagespirit: Verbal guided image parsing. ACM Transactions on Graphics (ToG), 34(1):1-11, 2014.
Yong Xien Chng. Henry Zheng, Yizeng Han, Xuchong Qiu, and Gao Huang. Mask grounding for referring image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16573-26584 2024.
Ekin Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv: 2018.
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 702-703, 2020.
Henghui Ding, Chang Liu. Suchen Wang, and Xudong Jiang. VIt: Vision- Vision-language transformer and query generation for referring segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7900-7911 2022.
Seongsu Ha, Chaeyun Kim, Donghwa Kim, Junho Lee, Sangho Lee, and Joonseok Lee. Finding nemo: Negative-mined mosaic augmentation for referring image segmentation. In European Conference on Computer Vision, pp. 121-137. Springer, 2024.
Ronghang Hu, Marcus Rohrbach, and Trevor Darrell. Segmentation from natural language expres- sions. In European conference computer vision, pp. 108-124. Springer, 2016.
11

---

Published as a conference paper at ICLR 2026
Jiaqi Huang, Zunnan Xu, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, and Xiu Li. Densely connected aarameter-efficient tuning for referring image segmentation. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 3653-3661, 2025.
Namyup Kim, Dongwon Kim, Cuiling Lan, Wenjun Zeng. and Suha Kwak. Restr: Convolution-free referring image segmentation using transformers. In CVPR, pp. 18145-18154, 2022.
Minhyun Lee, Seungho Lee, Song Park, Dongyoon Han, Byeongho Heo, and Hyunjung Shim. Maskris: Semantic distortion-awar data augmentation for referring image segmentation. arXiv preprint drXiv:2411.1906) 2024.
Yingjie Li, Weiqi Jin, Su Qiu, and Qiyang Sun. Multimodal crompt-guided bidirectional fusion for referring remote sensing image segmentation. Remote Sensing, 17(10):1683, 2025.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on Computer Vision, pp. 740-755. Springer, 2014.
Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Generalized referring expression segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 23592-23601, 2023a.
Jiang Liu, Hui Ding, Zhaowei Cai, Yuting Zhang, Ravi Kumar Satzoda, Vijay Mahadevan, and R Manmatha. Polyformer: Referring image segmentation as sequential polygon generation. In CVPR, pp. 18653- 2023b.
Sun-Ao Liu, Yiheng Zhang. Zhaofan Qiu, Hongtao Xie, Yongdong Zhang, and Ting Yao. Caris: Context-aware referring image segmentation. In Proceedings of the 31st ACM International Con- ference on Multimedia, pp. 779-788, 2023c.
Qiuyu Long, Nan Zhao, and Haifeng Liu. Dsnet: Predicting drug-side effect frequencies via dual- graph ensemble and similarity learning. Knowledge-Baud -Based Systems, pp. 113537, 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. 1rXiv:1111.05101, 2017.
arXiv preprint
Haoxin Lyu, Tianxiong Zhong, and Sanyuan Zhao. Gtms: gradient-driven tree-guided mask-free referring image segmentation method. In European Conference Computer Vision, pp. 288-304. Springer, 2024.
Luis Perez and Jason Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04622 2017.
Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26374-26383, 2024.
Hengcan Shi, Hongliang Li, Fanman Meng, and Qingbo Wu. Key-word-aware network for referring expression image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 38-54, 2018
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
Jiajin Tang, Ge Zheng, Cheng Shi, and Sibei Yang. Contrastive grouping with transformer for referring image segmentation. In Proceedings of the IEEF/CVE conference on computer vision and pattern recognition, pp. 23570-23580, 2023.
Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pp. 10347-10357. PMLR, 2021.
Roman Vershynin. High-dimensional probability, 2009.
12

---

Published as a conference paper at ICLR 2026
Yaoming Wang, Jin Li, Xiaopeng Zhang, Bowen Shi, Chenglin Li, Wenrui Dai, Hongkai Xiong, and Qi Tian. Barleria: An efficient tuning framework for referring image segmentation. In The Twelfth International Conference on Learning Representations, 2024.
Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. Cris: Clip-driven referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11886-11695, 2022a.
Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. CRIS: Clip-driven referring image segmentation. In CVPR, 2022b.
Zhenhua Wang and Linwei Ye. Referring image segmentation with two-stage multi-modal interac- tion. In 2024 IEEE International Conference on Image Processing (ICIP), pp. 2549.2549. IEEE, 2024.
Junxian Wu, Yujia Zhang, Michael Kampffmeyer, and Xiaoguang Zhao. Prompt-guided bidirec- tional deep fusion network for referring image segmentation. Neurocomputing, 616: 128899, 2025.
Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Gen- eralized segmentation via multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3858-3866 2024.
Chunyu Xie, Heng Cai, Jincheng Li, Fanjing Kong, Xiaoyu Wu, Jianfei Song, Henrique Morimitsu, Lin Yao, Dexin Wang, Xiangzheng Zhang, et al. Cemb: large- scale chinese cross-modal bench- mark. In Proceedings of the 31st ACM International Conference on Multimedia, pp. 2219942227, 2023.
Chunyu Xie, Bin Wang, Fanjing Kong, Jincheng Li, Dawei Liang, Gengshen Zhang. Dawei Leng. and Yuhui Yin. Fg-elip: Fine-grained visual and textual alignment. arXiv preprint arXiv:2505.05071, 2025.
Zunnan Xu, Zhihong Chen, Yong Zhang, Yibing Song, Xiang Wan, and Guanbin Li. Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 77503-17512, 2023.
Yuguang Yang, Runtang Guo, Sheng Wu, Yimi Wang, Linlin Yang, Bo Fan, Jilong Zhong, Juan Zhang, and Baochang Zhang. Decomcam: Advancing beyond saliency maps through decompo- sition and integration. Neurocomputing, 610:127826, 2024a.
Yuguang Yang, Tongfei Chen, Haoyu Huang, Linlin Yang, Chunyu Xie, Dawei Leng, Xianbin Cao, and Baochang Zhang. Prompt as knowledge bank: Boost vision-language model via structural representation for zero-shot medical detection. In The Thirteenth International Conference on Learning Representations, 2025.
Yuhuan Yang, Chaofan Ma, Jiangchao Yao, Zhun Zhong, Ya Zhang, and Yanfeng Wang. Remamber: Referring image segmentation with mamba twister. In European Conference on Computer Vision, pp. 108-126. Springer, 2024b.
Zhao Yang, Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao and Philip HS Tor. Lavt: Language-aware vision transformer for referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18155-18165, 2022.
Zhao Yang. Jiaqi Wang, Yansong Tang, Kai Chen, Hengshuang Zhao, and Philip HS Torr. Semantics-aware dynamic localization and refinement for referring image segmentation. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 3222-3230, 2023.
Sconghoon Yu, Paul Hongsuck Seo, and Jeany Son. Pseudo-ris: Distinctive pseudo-supervision generation for referring image segmentation. In European Conference on Computer Vision, pp. 18-36. Springer, 2024.
13

---

Published as a conference paper at ICLR 2026
Zicheng Zhang, Yi Zhu, Jianzhuang Liu, Xiaodan Liang, and Wei Ke. Coupalign: Coupling word- pixel with sentence-men mask alignments for referring image segmentation. Advances Neural In- formation Processing Systems, 35:14729-1444 14742, 2022.
Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, and Jiwen Lu. Unleashing text- to-image diffusion models for visual perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9729-5739, 2023.
Xiawu Zheng, Chenyi Yang, Shaokun Zhang, Yan Wang. Baochang Zhang, Yongjian Wu, Yunsheng Wu, Ling Shao, and Rongrong Ji. Ddpnas: Efficient neural architecture search via dynamic distribution pruning. International Journal of Computer Vision, 131(5):1244-124,, 2023.
Sheng Zhou, Junbin Xiao, Xun Yang, Peipei Song, Dan Guo, Angela Yao, Meng Wang. and Tat- Seng Chua. Scene-text grounding for text-based video question answering. arXiv preprint 2rXiv:2409.14319, 2024.
Sheng Zhou, Junbin Xiao, Xun Yang, Peipei Song, Dan Guo, Angela Yao, Meng Wang, and Tat- Seng Chua. Scene-text grounding for text-based video question answering. IEEE Transactions on Multimedia, 2025.
14

---

Published as a conference paper at ICLR 2026
A DATASET
A.1 DETAILS
RefCOCO is widely adopted benchmark dataset sourced from the MSCOCO (Lin et al. 2014) dataset via a two player game and contains 19,99, images annotated with 142,210 referring ex- pressions for 50,000 objects. The dataset is partitioned into four subsets: 120,624 training samples, 10,834 validation samples, 5,657 samples for test A, and 5,095 samples for test B. The referring expressions have an average length of 3.6 words, and each image features at least two objects on average.
RefCOCO+ dataset consists of 141,566 referring expressions related to 49,856 objects 19,992 images. The dataset also divided into four subsets: 120,624 training samples, 10.758 validation samples, 5,726 samples for test A, and 4,889 samples for test B. Distinctively, RefCOCO+ is de- signed to be more challenging than RefCOCO excluding certain types of absolute location words in the referring expressions.
RefCOCOg contains 85,474 referring expressions for 54,822 objects across 26,711 images, collected non-interactively. Notably, its descriptions are significantly more complex (8.43 words/cxpression) than RefCOCo (3.61) and RefCOCO+ (3.53). Additionally, RefCOCO and Ref- COCO+ exhibit higher intra-category object density (3.9 vs. 1.6 objects/image).
To evaluate generalization in multi-object and zero-object settings, and its gRefCOCO dataset. gReíCOCO contains 278,232 expressions, including 80,022 multi- target and 32,202 no-target expressions, referring 60,287 distinct instances across 19,994 images; masks and boxes are provided for all targets, and a portion of single-target expressions are inherited from RefCOCO. Evaluation follows GRES with cloU and gloU (and separate no-target accuracy when relevant), using the same UNC partition as RefCOCO.
the GRES benchmark
A.2 COMPARISON
Although RefCOCo, RefCOCO+, and RefCOCOg share common MSCOCO image base, they differ substantially in annotation style and linguistic complexity. RefCOCO features short, spatially grounded expressions (avg. 3.6 words) collected interactively. RefCOCO+ suppresses absolute location terms, encouraging models to rely uppearance-basee cues. RefCOCOg contains longer, non-interactive descriptions (avg. 8.43 words), posing greater challenges semantic parsing and low object density contexts. These differences introduce meaningful distribution shifts across both language and vision, making this setup a robust test for generalization.
B EVALUATION METRICS
Following the common practice (Yang et al.) [0222] 2023 Zhang et al. 2022), we adopt the metrics (mloU), and P@ X. Specif- ically, oloU measures the total intersection area over the total union area across all test samples, while mloU averages the per-sample IoU values. P@ X computes the percentage of samples with IoU exceeding threshold X, where X E {0.5, 0.7, 0.9}. These metrics provide a comprehensive assessment of both global segmentation quality and alignment precision.
of overall intersection-over-uniie (oloU), mean intersection-over-uiiis
C TRAINING DETAILS
To ensure fair comparison, we integrate AML into several representative RIS frameworks, in- cluding DETRIS (Huang al., 2025) CARIS (Liu al. 2023c) and ReLA (Liu al. 2023a). Unless otherwise specified, our main implementation is built upon CARIS (Liu et al. (023c). re- cent state-of-the-art method. We follow its original training protocol: the model is optimized using AdamW (Loshchilov & Hutter, 2017) with weight decay of 0.01. The initial learning rate is set to le-5 for the image/text encoders and le-4 for the remaining components. A poly learning rate decay strategy is applied with power parameter of 0.9, gradually annealing the learning rate to zero. All models are trained for 50 epochs with a batch size of 16. The input resolution fixed to 448 x 448 for both training and testing. For the proposed AMI. framework, adopt the following
15

---

Published as a conference paper at ICLR 2026
woman
eldest zebra
broccoli whove floret
side the fisk
red sleeves
68
giruffe closest
people
Image
Ground Truth
Baseline
Baseline - RRM
Baseline + AML
Figure 5: More comparison of prediction maps: Baseline Raandom Mask (RM), and AML.
default hyperparameters: = 0.4, p 0.25, Hp Wp 32 x 32, D. 2048. All experiments are conducted on a single A800 GPU.
D MORE VISUALIZATION OF PREDICTION MAPS
The more comparison of prediction maps is shown as Figure which consistently demonstrates our superior performance in both boundary precision and semantic discrimination.
E DETAILS FOR ANALYSIS OF CROSS-MODAL SIMILARITY
For the computation of S, our method directly uses the similarity scores from PMME, while for the baseline model, to ensure fair comparison, we replicate the identical projection layer and append to features after encoding. Since the projection layer in PMME remains fixed after initialization, this approach introduces unfairness in training adaptation. Our statistical analysis is conducted on 100 randomly batches sampled instances from RefCOCO+ val dataset, yielding total of 313600 simi- larity scores. As shown in Figure our method yields a higher mean similarity and a lower standard deviation, indicating more concentrated and semantically coherent feature activations. Additionally, the proportion visual regions with strong similarity also improves substantially. These results demonstrate that discrepancy-awant masking effectively suppresses misaligned or ambiguous regions, allowing the encoder to focus on high-confidence, text-relevant areas.
Glibal Mtsrttn
Globa
(a) Contribution of similarity scores on CARIS
(b) Contribution of similarity scores on AML
Figure 6: Contribution of similarity scores.
16

---

Published as a conference paper at ICLR 2026
F PROOF FOR PMME'S RANDOM PROJECTION ALIGNMENT
F.1 THEOREMI: CROSS-MODAL INNER PRODUCT PRESERVATION VIA GAUSSIAN RANDOM MAPPING
Let E (0, 1) be the distortion parameter and E (0, the target failure probability. Consider two sets of normalized feature vectors:
fumm E IR m== [u, E ERO
J m=
where
||0ml122 |/nn/2 = 1.
Define block- biock-diagonal random projection:
W
V2 diag(W, Wi) E R(D+tD.xx2s
where W, DDDxDD and W, DDxDD are random matrices with entries (Wiku, (Wi)a ((0, b.. f.dd Define the projected embeddings:
W Um ER Ps, w/a = =WW, un E P..
the projection dimension satisfies
DR (log(MN/o)
8²
then with probability least o, the following holds:
(e w/x)) (Um, Un) S.
In other words, the inner products between all visual-language vector pairs are approximately pre- served under the random block diigonaa projection.
F.2 PROOF FOR THEOREM
Step 1: Construct the concatenated embedding space. Define the joint embedding set:
Um
E ERD+DD D.
=
2mn
Un
mE[M], nE[N]
Let W
/2 diag(W., Wi) E ((D.+ where W, and We are independent Gaussian random matrices with i..dd entries from (O 1/D).
Step 2: Distance preservation over all pairwise combinations. We wish to ensure that for all
pairs (m, n), (m',n')
e] 2mm Zm'n' 2|2
W Smn
Zmmaa (1 + e)]l=mn - Zm/n' /|?
with high probability. According to Theorem2 below, for fixed pair of vectors, by standard results (e.g., chi-squared tail bound, Hanson- Wright inequality). we have:
-
Pr [[] W='ll ||= ='113 2 e||= z'13]] < 2ex)( for 2, E Z.
There are at most (MNN < (MNY² probability d, set:
distinct pairs. Using a union bound and requiring total failure
2
2 eexp(-cDD 5 (MN)²*
which gives:
1 log(2(MN))/00
D>
62
Using 1/8 (standard for Gaussian), we simplify:
Slog(MM (M/))
z
e2
17

---

Published as a conference paper at ICLR 2026
Hence, with probability least 1 1, the Euclidean distance between all projected pairs is preserved within (1 + E) distortion.
Step 3: From distance preservation to inner product preservation. Assume all Ums un unit vectors. Then:
|| 1zmmll = |/1mml + ||u = 2.
For any (m, n), (m', n²), define 2 = Zmn and z' Zm/nn.. We use the identity:
(2,2') =1 3 112 =112.
According to Theorem2 (see Appendix F.3). we know that:
(1 - =) z'112 < W W = (1 e][(= - =/112.
Then:
1
(W2, W2') (z, 2
- II W2'11 ||2 211
- 2
. || 2/112
Given || 2'11 2, we have
(W2, W2') - (z, 2/) S 2 2/2 21122 2E. f|
Thus the geometric distortion between samples before and after projection is bounded by 2s, which the foundation of our approximate modality-lifting guarantee.
F.3 THEOREM2: BL.OCK-DIAGONA DISTANCE PRESERVATION
Let E E (0, 1) be the distortion parameter, 5 E (0. 1) the target failure probability, and suppose we are given two modality-specific datasets consisting of n paired samples:
{uu u;)) U, E R2,
U, E R
Define the concatenated vector x;
peto and consider a random block-diagona projec-
tion:
W 1
diag(W,,, W.).
where W,, e RxP and W. e Rd have i..dd entries sampled from from.((O, ((0, 1/d).
If the projection dimension d satisfies
8 log(nn (8)
d
2²
then, with probability at least - 8, the following inequality holds uniformly for all n(x-1) 1) distinct sample pairs ji:
(1 c)]l,; œ. 11 < Wx, Wxy 1 < +
e) x. 2 ViSi<jEn.
Interpretation. This theorem ensures that all pairwise Euclidean distances between the joint em- beddings (u, 21)) are approximately preserved under the random block-diagonal projection W. The guarantee holds uniformly with high probability as long as the projection dimension scales loga- rithmically with the number of pairs n² and inverse failure rate 1/5.
18

---

Published as a conference paper at ICLR 2026
F.3.1 LEMMA 1: GAUSSIAN PROJECTION YIELDS SCALED X² DISTRIBUTION
We begin by analyzing the distribution of |[well when W R is random Gaussian matrix with i..d. entries:
0.. )
Wie ~
=k= le.......dd I,....P
and w E IPP is any fixed vector.
Let Tkk IRP denote k-th row W. Then the k-th component of the projected vector Ww is:
E = r w
WkeW
By properties of Gaussian linear combinations, since the Wsr are independent and w is fixed, each & is normally distributed:
0, 3 wi (0) = N N/., 1|012
& ~ ~~W
1=1
Hence, the squared norm of the projected vector is:
w
Now, observe that we can reparametrize each term as:
Ex 1),
( Ex
|10 |///l//v x²-
1 1 Vd
Summing over all & I...,, d, we obtain:
Ex
)
xa.
R=1 luulle/va
Therefore, we conclude:
||
||Wl||
xa.
This shows that the squared norm of the projected vector follows scaled chi-squared distribution, which concentrates tightly around its expectation as d grows.
F.3.2 LEMMA 2: TAIL. BOUND FOR x3 VIA BERNSTEIN'S INEQUALITY
We now establish a concentration bound for the x2 random variable appearing in Step E3.1 The key result is the following:
Result.
Let = X?, where X, ~ NO 1) ii.d Then, for any e (0, 1).
d
Pr [1xa
ed] S
exxp
(A.2)
This is the tail bound used in the JL lemma analysis, and we now prove it by applying Bernstein's inequality.
19

---

Published as a conference paper at ICLR 2026
Step 1: Define the zero-mean variable. Let
d S xa
(x? 1)..
i=1
Note that [[S] 0.
We are interested in bounding Pr[[S] 2 ed].
Step 2:
Verify Bernstein's inequality conditions. random variable:
Each X? is a centered sub-cxponential
X 1 E SubExp(K), with K |X] 1/ oo,
as shown in Vershynin (2018, Lemma 2.7.7). The variance is:
Var(X E[x;) (E(X?))* =3 1=
Therefore, the sum S satisfies:
Var(X] - 1) = 2d.
Step 3: Apply Bernstein's inequality. Vershynin's 's Hihg-Diman Dimnn-DDnnttinall Probability
This inequality be found in the Theorem 2.8.1 in
Vershynin 2009). Let æd, then Bernstein gives:
t2
))
(ed)² ed
Pr(S 2 t
exp
min
exp
min
4 2K
8d
2K
Since e E 10,1) and d is large, the first term dominates:
de
Pr[S 2 ed S exp
8
By symmetry, the lower tail satisfies the same bound:
de²
Pr[S < =dd s exp
Step 4: Combine upper and lower bounds. We conclude:
ds
Pr [Ix d 2 ed] Pr[[S| 2 ed] 2 eexp
8
which completes the proof.
Implication for Projection Norm. Recall from Step F.3.
that:
|| w [
2
x2.
therefore:
d
Pr [[]l w||
||w|||]
E||w||
Pr
[Ixà d)
2
ed]
2 exp
(A.3)
8
This tail inequality will be used in the next step uniformly control all pairwise distortions.
20

---

Published as a conference paper at ICLR 2026
F.3.3 PROOF OF THEOREM 2
Step Formulation. We now prove the distance-preservation guarantee for random block-diagonal projections over all pairs of concatenated visual-language embeddings. Recall we are given n pairs (u,, U;), where U E IR" (visual) and U; IR (language). Define the concatenated embedding:
U1
12
Osj biz)
2; ---- Ij
Let the random projection matrix be:
diag(W... W.), Wy ~ N(0,1///// W.~ N(0,
V2
Step 2: Concentration for each pairwise difference. From Lemma F33.2 we know: [|ll ass /| || ||? 2 =||aı; ||] S o/2,
and similarly for ||,bb,)) These bounds hold uniformly over all aij big for fixed (i,j), under the dimensionality condition:
8 8log(2n²/o
d
2²
Step 3: Joint success probability. Let event A, be: |W E ((-=s)laaylf* (1+5) 11²]. and B,y be the corresponding event W, side. Then:
Tr[Ag]21 .. Pr[B,]21-
Since Wa and Wi are independent, the joint event satisfics:
Pr[Ay n B,yl [ 2 21-0.
This gives a sharper bound than simply using union bound.
Step 4: Combine into distance bound. Under event A. 0 Bij. we have:
1 e [(1-=)]la,,l' 112 (1++) |a: bl e [(1-s)] 2 (1++) [13]
So the projected squared distance satisfies:
1
[[ 2 = 2 (1 |[Wbblll?) e [(1 - e)] x,113, (1++)]la, -x,1l² 2,]13].
Hence, with probability 2 1 - a. the block diagonal random projection preserves the pairwise dis- tance between 2; and 2, within (1 E).
Step 5: Extend to all (*) pairs. Let |P|
(M) < n². Apply union bound over all such pairs:
(i.j), violation occurs] $ |P| o S n²
Set this to be < 8. Then:
8 logg(n2200
>
n2
Step 6: Final conclusion.
If d > slog(2n' /6)
then with probability at least 1 - 8, we have:
(1 - |) - 2,11² | v;
Way 1| < (1
e)|,>, -x,1
V Vi
21

---

Published as a conference paper at ICLR 2026
G ADDITIONAL EXPERIMENTAL RESULTS
G.1 ABLATION: PROJECTION DIMENSION AND JL ERROR BOUND
To quantify the impact of the random projection dimension on cross-modal alignment and seg- mentation performance, we conducted a comparison on ReiCOCO+ with projection dimensions Da {1024, 2048, 40963. All other settings remained consistent.
Table Performance of different projection dimensions D, and corresponding JL error bounds.
Dim (Da)
oloU
mloU
Pr@50
Pr@90
JL Error Bound (c)
1024
66.82
70.83
80.15
36.27
0.2955
2048
67.37
71.33
80.82
36.35
0.2088
4096
67.08
71.21
80.85
36.22
0478
As shown in Table 8 when the dimension Da increases from 1024 to 2048, we observe a noticeable improvement in all metrics: oloU increases from 66.82 67.37, mIoU improves from 70.83 to 71.33, Pr@ 50 increases from 80.15 to 80.82, and Pr@90 rises from 36.27 to 36.35. However, further increasing the dimension to 4096 results in marginal gains, with oloU reaching 67.08, mloU at 71.21, Pr 50 at 80.85, and Pr@ at 36.22.
Meanwhile, the theoretical error bound decreases monotonically with increasing D, (0.2955 0.2088 - 0. 1478), which is consistent with the empirical trend. This confirms that higher dimen- sions better preserve mrosll geometric structures. Based on the trade-off between accuracy and scalability, we choose D, 2048 as the default setting. This configuration achieves a good balance between controlled theoretical distortion bound (e = 0.2088) and diverse, semantically consistent region selection, while nearing saturation the main segmentation metrics. Thus, this validates the rationale and cost-effectivenees of using this projection dimension in PMME.
G.2 TRAINING OVERHEAD OF AML
We conducted ablation study quantify the additional training time and memory overhead intro- duced by the AML approach, specifically due to the two forward passes per iteration. We compared the models with and without AML under identical batch sizes. All other network components and training settings remained the same to ensure a fair comparison.
As shown in Table y we observe that the introduction of AML results in only minimal increase in memory consumption (+4.9%) and training time per epoch (+17.2%). On DETRIS, the overhead is also limited to 5.3% in memory and 19.6% time. These results suggest that the additional overhead introduced by AML is lightweight, especially when compared to the significantly larger overheads that would arise from incorporating additional data during training. These findings validate that the overhead introduced by AML is relatively small, supporting its efficiency while maintaining strong model performance.
Table 9: Comparison of training overhead.
Method
Batch Size
Memory(MB)
Time/Epoch((inn)
CARIS
16
36,168
30.16
CARIS+AML
16
36,902(44.998
35.35(+17.2%
DETRIS
12,206
27.03
DETRIS+AML.
16
12,860 (+5.3 %
32.34 (+19.6%)
We further analyze the training overhead and fairness of CARIS+AML compared with CARIS. While CARIS+AML introduces 17.2% increase in training time due to an additional forward pass in the first stage, the total number of optimization steps remains unchanged, since parameter updates occur only in the second stage and each epoch sees the same number of batches as CARIS. As shown in Table 10, under the same wall-clock time (8/26/42 epochs for CARIS+AML vs. 10/30/50 for CARIS), AML consistently outperforms the baseline (e.g., 66.7/70.8 65.5/69.3 the 50-epoch
22

---

Published as a conference paper at ICLR 2026
time budget), and already reaches the 50-epoch CARIS performance using roughly the 30-epoch time budget (65.3/69.5 vs. 65.5/69.3), confirming that the improvements come from the alignment- aware masking strategy rather than from additional optimization steps.
Table 10: Analysis of training overhead and faimess CARIS. Memory and time are measured with batch size 16. Performance is reported as mloU oloU on RefCOCO+
Method
Memory (MB)
Time/Epoch (min)
10 epochs
30 epochs
50 epochs
CARIS
35,168
30.16
60.2 / 62.3
64.7 68.3
65.5 69.3
+AML/ same time
61.8 64.2
65.3 69.5
66.7 70.8
36,902 (+4.9%
35.35 (+17.2%%
+AML/ same steps
63.8 / 65.4
66.1 / 70.1
67.4
G.3 GENERALIZATION CAPABILITY FOR MULTI-TARGET AND COMPOSITIONAL EXPRESSIONS
The AML strategy is not limited to single-target referring expression localization; it is equally appli- cable to multi-target and compositional relational expressions. Specifically, the PatchMax Matching Estimation (PMME) in AML computes fine-grained similarities between all patches in the image and all tokens in the referring expression. Any patch that exhibits strong alignment with any to- cen-whether it is referring term (e.g.. "dog") or contextual cues (e.g., "on the left", "bench") is retained. This token-level matching mechanism preserves both the target regions and contextual regions necessary for disambiguation, which is particularly crucial for relational expressions.
For instance, consider the expression "the giraffe closest to that man". Correct localization depends on spatial relationships rather than appearance alone. AML is capable of preserving semantically critical regions (such as "man"), even without explicit multi-mask supervision.
To further evaluate AML's generalization capability in multi-target and zero-target scenarios, integrate AML with ReLA and conduct experiments on the GRES dataset. The results are as follows:
Table 11: Performance comparison on GRES dataset.
Method
cloU@5k
gloU@5k
cloU @ 150k
gloU @ 150k
ReLA
28.49
17.33
62.50
63.25
ReLA + AML
30.86
21.08
62.61
63.52
Compared to ReLA, AML brings significant improvements in the early training stage: at 5k itera- tions, cloU and gloU improve by +2.37% and +3.75%. respectively. However, at 150k iterations, the improvement is marginal 19%). This indicates that AML helps filter out misaligned regions in the early stage, but in multi-target or zero-target scenarios, its potential is not fully realized due to insufficient consistency in boundary definition. Future work will explore soft or adaptive masking strategies better handle these complex scenarios.
G.4 ALIGNMENT AND MASKING STABILITY ANALYSIS
Table 12: Evolution of cross- modal alignment metrics across training epochs 1-10. Arrows indicate desired optimization direction: J for minimization, for maximization.
Metric/ / Epoch
2
3
4
5
6
7
9
10
0.20 J
0.0041
0.0010
0.0008
0.0007
0.0008
0.0008
0.0007
0.0008
0.0007
0.0008
0.50
0.8021
0.8464
0.8620
0.8656
0.8650
0.8700
0.8720
0.8769
0.8836
0.8863
2
+
0.4192
0.4543
0.4667
0.4779
0.5022
0.5079
0.5076
0.5178
0.5196
0.5252
mean(sim) (sim) +
0.7542
0.7821
0.7917
0.7963
0.8031
0.8061
0.8142
0.8117
0.8182
0.8211
std(sim)
0.2393
0.2230
0.2171
0.2161
0.2097
0.2075
0.2051
0.2044
0.2053
0.2041
We investigate the stability dynamics of our AML training strategy, which employs curriculum- based alignment approach rather than enforcing uniform alignment during early training phases. The Alignment-awan Filtering Mask (AFM) progressively focuses regions with high optimization
23

---

Published as a conference paper at ICLR 2026
Mask-on-grouue ground-truth coverage rates across different threshold configurations during epochs 10. Lower values indicate better masking precision.
Table 13:
Epoch
1
4
7
9
10
0.20
0.0021
0.0007
0.0004
0.0005
0,0005
0.0005
0.0005
0.0005
0.0004
0.0005
0.0004
0.30
0.0097
0.0057
0.0058
0.0053
0.0052
0.0051
0.0049
0.0044
0.0046
0.0043
0.0043
0.40
0.0259
0.0178
0.0157
0.0139
0.0135
0.0129
0.0130
0.0125
0.0121
0.0122
0.0120
0.60
0.0726
0.0628
0.0571
0.0559
0.0558
0.0543
0.0537
0.0526
0.0523
0.0519
0.0520
Table 14: Training loss comparison between CARIS bascline and AML across 50 epochs.
Early Training Phase (Epochs 1-17)
Epoch
5
11
13
15
17
CARIS
0.3114
0.2060
0.1636
0.1413
0.1237
0.1109
0.1002
0.0919
0.0846
AML
0.2903
0.1803
0.1434
0.1209
0.1060
0.0928
0.0816
0.0742
0.0669
Improvement
6.8%
12.5%
12.3%
14.4%
14.3%
16.3%
18.6%
19.3%
20.9%
Mid Training Phase (Epochs 18
18-34)
Epoch
18
20
22
24
26
28
30
32
34
CARIS
0.0819
0.0759
0.0702
0.0643
0.0613
0,0565
0.0532
0.0501
0.0468
AML
0.0650
0.0580
0.0531
0.0489
0.0462
0.0424
0.0405
0.0385
0.0362
Improvement
20.6%
23.6%
24.4%
23.9%
24.6%
25.0%
23.9%
23.2%
22.6%
Late Training Phase (Epochs 35-50)
Epoch
35
37
39
41
43
45
47
49
50
CARIS
0.0458
0.0441
0.0415
0.0404
0.0382
0.0368
0.0352
0.0348
0.0342
AML 0.0348
0.0336
0.0321
0.0304
0.0297
0.0289
0.0281
0.0273
0.0271
Improvement
24.0%
23.8%
22.7%
24.8%
22.3%
21.5%
20.2%
21.6%
20.8%
feasibility and internal consistency through threshold and masking ratio control, establishing stable alignment baselines before extending to ambiguous regions. This curriculum strategy mitigates instability issues inherent in unpaired vision-language backbones during initial training stages.
Experimental Setup. We evaluate alignment and masking stability using patch-text similarity mea- sured by PMME and task--on-ground-mee coverage rates. Experiments are conducted on fixed evaluation set of 1,552 image-text pairs with 14 x 14 patch granularity. We track alignment met- rics across epochs 1-10, including weak alignment ratio (sim < 0.20), moderate alignment ratio (sim 2 0.50). high-confidence alignment ratio (sim 2 0.90), mean similarity, and standard devi- ation. Mask accuracy is assessed by computing coverage rates of ground-truth alignment patches filtered by threshold
{0.20, 0.30, 0.40, 0.60} during epochs 0-10. Convergence analysis com- pares 50-epoch training loss trajectories between CARIS baseline and AML-enhanced models under identical configurations.
Alignment Dynamies. Table presents the evolution of cross-modal alignment metries, revealing systematic improvement patterns. Weak aligmment regions (sim < 0.20) exhibit dramatic reduction from 0.41% 0.08% by epoch 10, with the most significant drop occurring between epochs 1-2 (0.41% - 0.10%). This rapid pruning of low-quality correspondences stabilizes thereafter, main- taining consistently low levels (< 0.1%). Moderate alignment ratios demonstrate steady growth from 80.21% % to 88.63%, while high-confidence alignments increase substantially from 41.92% to 52.52%, representing a 25.3% relative improvement. The similarity distribution undergoes concur- rent upward shift (mean: 0.7542 0.8211, +8.9%) and concentration (std: 0.2393 0.2041, -14.7%), manifesting the characteristic "upward shift + convergence" pattern that validates AML's selective filtering mechanism.
Masking Precision. Table B analyzes mask-on-Gi coverage across different threshold configura- tions, revealing threshold- filtering precision. Conservative thresholds demonstrate excep-
24

---

Published as a conference paper at ICLR 2026
tional accuracy: T 0.20 maintains coverage below 0.1% throughout training (0.21% 0.04%), while r = 0.30 achieves 55.7% reduction (0.97% - 0.43%). The T = 0.40 setting, despite higher initial coverage (2.59%), converges acceptable levels (1.20%) representing 53.7% improvement. These results indicate effective preservation of critical positive signals. However, aggressive thresh- old 0.60 exhibits concerning behavior with coverage remaining elevated above 5%, suggest- ing potential over-masking that could impair learning signals despite 28.4% reduction from initial 7. 7.26%.
Convergence Characteristics. Table 14 demonstrates superior optimization dynamics of AML across the complete 50-epoch training trajectory. Initial acceleration pronounced, with first-epoch losses of 0.2903 (AML) versus 0.3114 (CARIS), representing 6.8% improvement. The advantage persists throughout training phases: early stage (epochs 1-17) shows consistent 7-15% loss reduc- tion, mid-stage (epochs 18-34) maintains 15-23% advantage, and late-stage convergence (epochs 35-50) achieves 21-24% improvement. Terminal performance demonstrates 20.8% reduction in final loss (0.0271 vs. 0.0342), indicating significant steady-state error minimization alongside ac- celerated convergence. Notably, the loss gap widens progressively, suggesting AML's enhanced capacity for fine-grained optimization in later training phases.
The comprehensive quantitative analysis across alignment distribution evolution, masking precision metries, and convergence dynamics provides robust evidence for AML's stabilization effects during early training phases. The systematic improvements alignment quality (25.3% increase in high- confidence regions), masking accuracy (sub-3% coverage for conservative thresholds), and con- vergence efficiency (20.8% final loss reduction) substantiate the effectiveness of curriculum-based alignment strategies in addressing fundamental challenges of alignment and masking instability un- der unpaired backbone architectures.
G.5 GENERALIZATION AND ROBUSTNESS ANALYSIS
To comprehensively evaluate the model's real-world applicability and robustness, we designed series of rigorous tests. This is particularly crucial for the Referring Image Segmentation (RIS) task, as the model must parse diverse and complex natural language queries. For instance, the model might be required locate object based its spatial position (e.g.. "the car in the bottom right corner", "the left skier in the foreground"", its attributes and actions (e.g., "a fat woman playing with a frisbee"), or its relationship with surrounding objects (e.g.. "the banana separated from the bunch") ").
A truly robust model must not only accurately interpret these expressions under ideal image condi- tions but also maintain stable performance when the visual input is severely degraded. Therefore, introduce seven simulated visual transformations systematically evaluate the model's abil- ity to tackle this challenge. Visualizations of these transformations are shown in Figure 7 These transformations and their objectives are follows:
Haze: By overlaying a semi-transparent white mask with 50% opacity, we simulate foggy or hazy weather conditions. This transformation reduces overall image contrast and clarity, testing the model's ability to identify objects in low-visibility scenarios.
Highlight: The overall brightness is increased by fixed factor of 1.5x to simulate scenes under intense light or overexposure. This can lead to a loss of texture and detail in bright areas of the image.
The overall brightness is decreased by fixed factor of 0.45x to simulate scenes with insufficient light, at night, or underexposure. This makes details in dark areas of the image difficult to discern.
Lowlight:
Contrast:
The intensity difference between light and dark areas is enhanced by fixed factor of 1.8x. By increasing the image contrast, we challenge the model's ability to discern object edges and textures.
Occlusion Box: An opaque gray box with side length equal to 20% of the image's 's shorter dimen- sion is placed random location on the image to simulate real-world scenarios where the target object is partially occluded. This forces the model to infer and segment the complete outline of an object from its visible parts.
25

---

Published as a conference paper at ICLR 2026
blue man
a füt woman playing friez bee
Left skier in foreground
Cur the bottom right corner
bananu separated from bunch
original
haze
hightlight
lowlight
contrast
0cc
patch mask
color jitter
Figure 7: Visualization of the seven visual transformations used to simulate diverse visual con- ditions. From left to right: Original, Haze, Highlight, Lowlight, Contrast, Occlusion Box, Patch Masking, and Color Jittering.
The image is randomly overlaid with three black rectangular blocks varying sizes and positions to simulate scattered multi-point information loss. This represents a more severe, structured loss of information than a single occlusion box, testing the model's ability to leverage global context for local content inference.
Patch Masking:
Color Jittering: The image's brightness, contrast, and saturation are randomly altered within the range (0.6, 1.4). This simulates color distortions caused by different camera sensors, varying light- ing conditions, or image post-processing, testing model's generalization color variations.
Through this series of comprehensive evaluations, we can more reliably measure the AMI strategy's combined strength in handling linguistic diversity and visual uncertainty. The performance results presented in Table 15
Cross-dataset robustness under various visual transformations. We train on RefCOCO+ and evaluate the validation splits of RefCOCO and RefCOCOg. Results are reported as mloU/oloUppr@500 Our method (AML)) consistently outperforms the baseline.
Table 15:
RefCOCO
RefCOCOg
Transformation
Baseline
Ours (AMI.)
Baseline
Ours (AMI.)
Haze
59.33/ 57.85 70.18
62.54 60.63 73.32
51.23 /1.23/48.66 48.69 58.66
53.38 / 50.60 60 60.99
Highlight
57.81 / 59.65 69.76
62.28 / 59.85 72.76
51.74 48.98 59.74
53.66 / 50.83 61.50
Lowlight
58.32 56.81 69.07
61.63 / 72..46
50.46 48.02 02/ 58.03
52.07 / 49.43 60.13
Contrast
60.05 / 58.15 70.29
63.05 60.56 /3.62
52.03 49,09 59.82
54.68 51.83/ 62.85
Occlusion Box
57.20 55.78 67.23
61.94 60.26
49.40 44.20 56.47
53.11 / 50.89 60.99
Putch Masking
60.22 / 58.51 70.74
63.18
61.17
52.71 50.02 60.92
$4.71 / 51.73 62.62
Color Jitter
59.93 / 58.27 77.558
62.71 60.58 73.33
51.94 49.25 59.84
$4.25 / 51.23 62.17
Average
58.98 / 57.86 69.69
62.48 / 60.41 / 73.16
51.36 / 48.75 59.07
53.69 / 50.93 61.61
Improvement
+3.50 / +2.55 +3.47
+2.34 / +2.18 +2.54
As shown in Table 15 AML consistently outperforms the baseline across visual transformations. On ReiCOCO, the average improvements are +3.45% for mloU, +2.53% for oloU, and +3.49% for @ 50. On RefCOCOg, the average improvements are +2.32% for mloU, +2.1 1% for oloU, and +2.61% % for 50 The consistent improvements under different expression styles and severe visual degradation conditions demonstrate that AML possesses strong transferability and robustness beyond the training distribution.
26

---

conducted the Published as a conference paper at ICLR 2026
Furthurmore, we conducted visual experiments the Flickr30k dataset by loading the model trained on RefCOCO+ and performing inference on single-object images from Flickr30k. The results, as shown in the Figure demonstrates that method maintains precise decision boundaries even data outside of RefCOCO.
baby crawling
white dog jumping
girl on train
woman singing
Image
Baseline
Baseline + AMI
Figure 8: Visual segmentation results on the Flickr30k dataset, where models trained on RefCOCO+.
G.6 DETAILED ABLATION STUDY ON HYPERPARAMETERS 7 AND
In this section, we present a comprehensive ablation study on two key hyperparameters of our AML strategy: the masking threshold and the dropout ratio p. This study aims validate the robustness of our method to these critical hyperparameters and to provide an empirical basis for the parameter settings adopted in our main experiments.
G.6.1 ANALYSIS OF DROPOUT RATIO p
The dropout ratio p defines the proportion of pixels, among those marked as "misaligned" by т, that are randomly set to zero. To investigate its impact, we tested various p values under two well- performing threshold settings (r 0.3 and 0.4). These experiments
RefCOCO+ testB set, with results summarized in Table [6]
The analysis of Table 16 further solidifies the effectiveness and robustness of method. First, all tested (T, p) combinations significantly outperform the baseline, with average performance gains ranging from +0.62 +2.52. This strongly demonstrates the general effectiveness of our AML strat- egy, as its benefits are not contingent on a single, both threshold configurations reveals that م 1/4 consistently yields optimal performance across different 7 values, achieving the highest average gains of +2.52 and +2.09 for 7 = 0.3 and 7 0.4, respectively. This dropout ratio achieves an effective balance: higher ratios (p 1/2) may over- suppress learning signals from moderately aligned regions, while lower ratios (p = 1/6) may be insufficient in filtering noise from misaligned areas. The relatively stable performance of = 0.25 across multiple threshold configurations suggests that retaining approximately 75% of misaligned pixels can provide appropriate regularization while preserving contextual information necessary for cross-modal reasoning. Moreover, this consistent superior performance across multiple threshold configurations provides direct and compelling empirical support for our choice to fix = 0.25 in the main paper and in the analysis of Subsection G.6.2
parameter setting. Second, examining
27

---

Published as a conference paper at ICLR 2026
Table 16:
Performance on RefCOCO+ testB with different dropout ratios p. We report individual
metrics, their average (Avg. Metric), and the average gain over the baseline.
Gain over Bascline
Setting
mloU
oloU
Pr@
Pr@70
Pr@90
Avg. Metric
Baseline
62.69
57.97
70.24
62.41
31.27
56.92
0.3, p 1/2
63.79
59.19
71.69
63.69
33.42
58.36
+1.44
0.3, p 1/4
64.82
59.74
72.65
65.23
34.77
59.44
+2.52
0.3, p 1/6
64.61
59.41
72.18
64.41
34.73
59.07
+2.15
0.4, p 1/2
64.15
58.95
72.08
63.98
34.77
58.59
+1.67
7 0.4, p 1/4
64.61
59.51
72.04
64.63
34.26
59.01
+2.09
0.4, p 1/6
63.44
58.75
70.81
62.88
31.83
57.54
+0.62
G.6.2 ANALYSIS OF MASKING TURESHOLLD
The masking threshold determines which visual regions are filtered based on their alignment with the text. A suitable value for r must strike a balance between preserving useful information and filtering out noisy regions. To independently evaluate the impact of т. we first fix the dropout ratio = 0.25, a value demonstrated to perform optimally(see Subsection G.6.I). We then test the performance across multiple benchmark datasets varies within the range [0.2, 0.4]. The results are presented Table 17
Table 17: Performance with different threshold values across multiple datasets. Best performance on each dataset is bolded.
Threshold
RefCOCO
RefCOCO+
RefCOCOg val
val
testA
val
testA
0.2
75.00
78.14
66.99
58.05
65.78
0.3
75.81
78.09
66.82
59.19
65.13
0.4
75.45
78.33
67.37
59.51
65.67
Several key conclusions can be drawn from the results in Table First, the necessity of non-zero threshold is evident. As mentioned in the main text, with r 0 (i.e., no filtering), the performance on RefCOCO and RefCOCO+ 74.65 and 65.54, respectively. All configurations with r 0 in Table outperform this baseline, demonstrating that filtering poorly aligned pixel regions is crucial for guiding the model's optimization.
Second, method exhibits remarkable performance stability and robustness for E [0.2, 0.4]. The performance does not fluctuate drastically with changes т, indicating that our approach provides benefits without requiring tedious fine-tuning, We also explored dynamic thresholding strategy, where т was adjusted per sample based on prediction confidence and referring expression length. However, this complex approach did not yield superior results compared to fixed 7 0.4.
Considering the robust performance across all datasets, particularly on the more challenging Ref- COCO+ and RefCOCOg, select 0.4 as default value for the main experiments, it offers an excellent balance between aggressive filtering and stable performance.
G.7 ABLATION: IMAGE-LEVEL VS. FEATURE- MASKING
To validate the effectiveness of our image-level alignment- masked (AML)) approach, we con- duct comprehensive ablation study comparing different masking strategies. Masking mechanisms can be implemented at two distinct levels: feature-level and image-level. Existing methods such as Mask2Former (Cheng et al. 2022) typically adopt feature-level masking, applying spatial suppres- sion during feature interaction processes. In contrast, our method performs masking operations at the image level, i.e., removing low-alignma alignmetmnnt regions before feature extraction. To comprehensively evaluate the effectiveness of these two strategies, we design corresponding comparative experiments with the following setup:
28

---

Published as a conference paper at ICLR 2026
Table 18: Comparison different masking strategies on RefCOCO+ dataset using oloU metric.
Method
val
testA
testB
Avg. 65.12 65.29 (+0.17)
Baseline
65.54
71.86
57.97
Feature Mask
66.34 (+0.80)
71.21 (-0.65)
58.32 (+0.35)
AML
67.37 (+1.83)
73.19 (+1.33)
59.51 (+1.54)
66.69 (+1.57)
Feature-level Masking, Following standard practices in segmentation literature (Cheng et al. 2022) we implement feature-level variant of AML where alignment masks are applied during feature interaction processes. Specifically, the similarity map S is computed on visual encoder features F, E I. NR xxxxxxx xD and low-alignment regions are suppressed through attention gating downstream modules. The feature-level gating mechanism is formulated as:
QQKT .Mussk V
Attention( Q, V) SoftMax
(16)
where the spatial mask Mmask E R H is defined as:
{ -00 otherwise
0
if S(t,,})
>
Minak [ =
(17)
Here, Mmusk is spatial mask used to suppress invalid positions before the softmax operation.
Image-level Masking (Ours). contrast, our image-level AMI. removes misaligned regions before feature extraction, ensuring that only vision-language consistent content enters the visual encoder. Results are shown in table
Results demonstrate that image-level masking significantly outperforms feature-level methods. First, our method achieves consistent improvements across evaluation splits, with an average gain of 1.57 oloU points, demonstrating the robustness of our approach. In contrast, while feature-level masking provides marginal improvements (average +0.17), it exhibits unstable performance across different splits,
showing performance degradation fundamental design distinctions between the two approaches: image-level masking prevents par- tial noise propagation through early intervention, while feature-level masking allows low-alignment regions to participate in early self-attention computations, leading to noise propagation through encoder layers before suppression. These results validate the effectiveness of early intervention strategies and provide important guidance for masking strategy selection in vision-languag tasks.
testA. This difference stems from the
H USE OF LARGE LANGUAGE MODELS
A large language model (LLM) was used only for stylistic polishing of prose (grammar, clarity) and for assisting table formatting (e.g., column alignment, caption phrasing). All content was authored, verified, and is fully owned by the authors, who take full responsibility for the submission.
29


Reviews:
[{"review_id": "EyDaOoG726", "reviewer": "Reviewer_pnob", "summary": "This work proposes a novel framework called AMLRIS, which aims to improve the performance of Referring Image Segmentation (RIS). The framework first introduces a Johnson-Lindenstrauss random projection to measure the similarity between image representations and token features. Then, the image pixels with low similarity are filtered out from the training process to stabilize the training and improve performance. Experimental results demonstrate that AMLRIS achieves superior performance compared to standalone training, and the ablation experiments show the effectiveness of the proposed modules.", "questions": "See Weakness above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "• The projection design provides a novel approach for measuring similarity between representations of different modalities. This method can be extended to more tasks.\n• Experiments demonstrate the effectiveness of the proposed structure, achieving competitive results across multiple downstream datasets.\n• The proposed structure does not significantly increase training overhead while maintaining inference time.\n• The proposed idea is interesting and generally well-motivated, and the experimental evaluation is relatively thorough", "weaknesses": "My primary concern is the method’s sensitivity to small or low-contrast objects. The AML\nframework relies on PMME to generate alignment-based masks by identifying high-confidence visual patches. This mechanism inherently depends on the relative distribution of features within the image. As a result, small objects or objects with low visual saliency may produce low peak alignment scores and be incorrectly masked out during training. Consequently, the model’s performance may degrade on images where the target occupies a very small region or is visually subtle.\n• In Figure 2, the masked pixels appear to be almost exclusively background regions. It is unclear\nwhether masking such areas truly helps the model focus on the target objects. In my view, it would be more important to mask regions corresponding to potentially confusing objects rather than background. This raises some concerns regarding the effectiveness of PMME in guiding the model’s attention.\n• The mechanisms underlying some of the core components of the model remain unclear. I would be willing to consider a higher score if the authors provide clear explanations addressing my concerns.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B1alG04U7D", "reviewer": "Reviewer_P4qH", "summary": "This paper presents an alignment-aware masked learning method for referring image segmentation. In particular, each sample is masked out by discarding pixels below an adaptive similarity threshold. The similarity map between visual and textual features is quantified by region-referent alignment. The framework is then trained after the first masking step. The above two steps are conducted in an interleaved fashion. In addition, the region-referent alignment is implemented via a PatchMax Matching Evaluation strategy on randomly projected visual and textual features. Experimental results validated the effectiveness of the proposed method.", "questions": "No.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "(1) The motivation is well presented of using the proposed alignment-aware masked learning approach for referring image segmentation.\n\n(2) The explanations and illustrations are mostly clear and intuitive of the PatchMax Matching Evaluation, the alignment-aware filtering mask and the training strategy.", "weaknesses": "(1) The approach of using a previous-step inference for mask prediction and guide the current learning may face convergence issue. In fact, the initial state of mask is largely incorrect and can result in unexpected learning curves. There is no discussion on this issue.\n\n(2) On the fairness of experimental comparison, since CARIS+AML uses 17.2% more training time than CARIS (according to Appendix G.2), the performance gain in Table 1 is also possibly coming from longer training. There is no ablation study and discussion on this issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M4ayErPZzX", "reviewer": "Reviewer_Vg5H", "summary": "- This work proposes Alignment-Aware Masked Learning (AML), a training strategy that quantifies region–referent alignment (PMME) and filters out unreliable pixels during optimization (AFM), which is validated to improve RIS performance.", "questions": "Refer to the Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- This work proposes Alignment-Aware Masked Learning (AML), a training strategy that quantifies region–referent alignment (PMME) and filters out unreliable pixels during optimization (AFM), which is validated to improve RIS performance.\n\n- The writing overall is good and it is easy for readers to understand the proposed framework.\n\n- The experiments analysis is detailed for readers  to realize the benefits of AML.", "weaknesses": "- Motivation clarification: the motivation is not well clarified from figure-1.  I suppose the author's motivation is that a number of regions (especially background regions) dominate the training loss.\n\n- Method contributions\n  - Based on the above motivation, I am more inclined to believe that this work is actually an implementation of curriculum learning in the RIS task. It is also be validated from the efficiency of early-training stage. In view of the originality, it decreases the contributions of this work.\n  - The work claims that the stage-I is forward-only. I am curious how the similarity of these raw visual-language features reflects the degree of alignment.\n  - The whole performance improvement is weak especially on the more strong models, which makes the work seem less significant at present community.\n\n- Some writings are confusing.\n  - The explanation about $B^h$ and  $B^w$ (line-267).\n  - For the `early-stage efficiency', to my knowledge, this is not a common term and it deserves a specific explanation.\n\n- Extra experiments for validation\n  - The author can verify the results of using different models at different stages (e.g., utilize ReLA for stage-1, CARIS for stage-2), which may bring some new observations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XCxnyjrK57", "reviewer": "Reviewer_X2d4", "summary": "The paper proposes Alignment-Aware Masked Learning (AML) for referring image segmentation (RIS). AML computes a patch–token alignment map via PMME (random projections per modality) and then applies an Alignment-Aware Filtering Mask (AFM) that suppresses low-alignment image regions during training only; inference is unchanged. Experiments show consistent gains on RefCOCO/+/g when AML is added to CARIS, smaller gains on DETRIS, early-stage experiments on ReLA, and robustness to synthetic perturbations.", "questions": "Refer to weaknesses.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.  **Plug-and-Play Method with Zero Inference Overhead** - Masking is applied exclusively during training. Consequently, the method requires no architectural changes and incurs no additional compute or memory costs during inference.\n\n2.  **Strong Performance** - AML consistently boosts CARIS performance across all eight RefCOCO/RefCOCO+/RefCOCOg splits. It also transfers effectively to other baselines, such as DETRIS, and shows promising results with ReLA.\n\n3.  **Robustness and Interpretability Analysis** - The paper shows improved performance under seven different visual perturbations (e.g., haze, low-light, occlusion), supported by clear qualitative maps that explain where AML helps.\n\n4.  **Thorough Ablation Studies** - The paper contains comprehensive ablations examining the threshold, projection dimension, projection strategy (random vs. learnable), and masking level (image vs. feature), which validate the design choices.", "weaknesses": "1. **Gaps in Mathematical Derivations.** The appendix asserts\n   $$\n   \\langle W_i v, W_t u\\rangle = \\tfrac{1}{2}\\langle \\tilde W z, \\tilde W z\\rangle,\\quad\n   \\tilde W = \\tfrac{1}{\\sqrt{2}}\\mathrm{diag}(W_i, W_t),\\quad\n   z = \\begin{bmatrix} v \\ u \\end{bmatrix}.\n   $$\n   But\n   $$\n   \\langle \\tilde W z, \\tilde W z\\rangle\n   = \\left\\langle \\tfrac{1}{\\sqrt{2}}\\begin{bmatrix} W_i v \\ W_t u \\end{bmatrix},\n   \\tfrac{1}{\\sqrt{2}}\\begin{bmatrix} W_i v \\ W_t u \\end{bmatrix}\\right\\rangle\n   = \\tfrac{1}{2}\\big(|W_i v|^2 + |W_t u|^2\\big),\n   $$\n   which is the *sum of squared norms* (no cross term), so it cannot equal $\\langle W_i v, W_t u\\rangle$ except in degenerate cases. Moreover, with *independent* Gaussian projections $W_i, W_t$ (entries $\\sim \\mathcal N(0, 1/D_a)$) and unit $v, u$,\n   $$\n   \\mathbb{E}[\\langle W_i v, W_t u\\rangle] = 0,\\qquad\n   \\mathrm{Var}[\\langle W_i v, W_t u\\rangle] = \\tfrac{1}{D_a},\n   $$\n   so the projected cross dot *concentrates at 0* as $D_a$ grows, making preservation of a nonzero cross-modal inner product *impossible* under the stated construction. This invalidates the paper’s stated Theorem 1 (cross-modal inner-product preservation under block-diagonal projection) and any corollaries that rely on it (e.g., PMME’s “geometry-preserving” property).\n\n2. **Evaluation Gaps**:\n    1. *Lack of Direct Baseline Comparison:* The evaluation is missing a direct comparison against a key alignment-based baseline, Mask Grounding, within the same experimental setting. Since AML is presented as a plug-and-play, alignment-aware strategy similar to Mask Grounding, its claimed universality should be validated by outperforming this baseline under identical, fully trained conditions. Table 2, however, only reports CARIS/DETRIS add-ons and omits Mask Grounding, leaving this critical parity untested. The table should have directly compared Mask Grounding on CARIS/DETRIS with AML on CARIS/DETRIS.\n    2. *Omission of MagNet mIoU Score:* Table 1 omits the mIoU score for MagNet, an important alignment baseline. \n\n3. **Training Efficiency:** Training and memory costs from implementing AML should be put in the main paper, not the appendix.\n\n4. **Generalization / Real-world robustness:** The paper's visualizations are limited to the heavily optimized RefCOCO datasets, making it difficult to assess performance on unconstrained \"in-the-wild\" images. Including qualitative results on such images, mirroring the style of Figure 4(a), is essential to demonstrate the model's real-world generalization\n\n5. **Reproducibility:** Since code is not given, adding some crucial pseudo code in the paper will be tremendously helpful for the broader audience to reproduce this work.\n\n6. **Many Writing & Formatting Issues**: \n    1. *References style:* inconsistent reference format; inconsistent capitalization (e.g., an author name in ALL CAPS on line 594); duplicate/inconsistent entries (e.g., CRIS 2022a/2022b). Standardize to ICLR format.\n    2. *Typos and formatting errors:* e.g. casing of softmax/Softmax is sometimes small (equation 16) and sometimes large (equation 6); no spacing in Theorem1 and Theorem2; PPME in line 1106.\n    3. *Unconventional figure naming:* Figure a, Figure b, Figure c etc.\n    4. *Readability of text in figures:* Some texts in Figure 1, Figure a and Figure b are too small.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)