Salient Object Ranking via Cyclical Perception-Viewing Interaction Modeling
Authors: Rongjin Guo
Ke Xu
Rynson W. H. Lau
Venue: ICLR 2026 Poster (2026)
Keywords: Saliency Ranking
Human Attention Shift Modeling
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Salient Object Ranking (SOR) aims to predict human attention shifts across different salient objects in a scene. Although a number of methods have been proposed for the task, they typically rely on modeling the bottom-up influences of image features on attention shifts. In this work, we observe that when free-viewing an image, humans instinctively browse the objects in such a way as to maximize contextual understanding of the image. This implies a cyclical interaction between content (or story) understanding of the image and attention shift over it. Based on this observation, we propose a novel SOR approach that models this explicit top-down cognitive pathway with two novel modules: a story prediction (SP) module and a guided ranking (GR) module. By formulating content understanding as the image caption generation task, the SP module learns to generate and complete the image captions conditioned on the salient object queries of the GR module, while the GR module learns to detect salient objects and their viewing orders guided by the SP module. Extensive experiments on SOR benchmarks demonstrate that our approach outperforms state-of-the-art SOR methods.

Published as a conference paper at ICLR 2026
SALIENT OBJECT RANKING VIA CYCLICAL PERCEPTION- VIEWING INTERACTION MODELING
Rongjin Guo, Ke Xu; Rynson W.H. Lau"
Department of Computer Science, City University of Hong Kong Rongjin. Guo cityu. edu.hk, skangwinggmaiil.cmm com, Rynson. Mauuciityy edu. edu.hk. hk
High
Low
Perception
understand the scene
1f
To shift one's attention
Viewing
People room
Women kitchen
women and a man are praparing food / Nitheen
(a) Cyclical Perception-Viewee Interaction.
TARTAS COLORED FOWORTOS
(b) Input
SeqRank
(d) DSGNN
PosESOR
(f) (initial) (g) (refined)
(h) GT
Figure/ 1: Illustration of eyclical interaction framework (a), where perception and viewing alternately influence each other, guiding attention shifts and deepening scene comprehension. Existing state-of-the-ar methods, such (c) (c). often fail semantically rich scenes due to their heavy reliance on low-level visual cues. Our initial prediction (f) exhibits similar limitations. However, through the proposed iterative refinement approach, our model progressively corrects these errors by modeling eyclical snceptioo-vvewwnng interactions, leading to accurate saliency rankings (g).
ABSTRACT
Salient Object Ranking (SOR) aims to predict human attention shift across different salient objects a scene. Although number of methods have been proposed for the task, they typically rely modeling the bottom-up influences of image features on attention shifts. In this work, we observe that when free-viewing an image, humans instinctively browse the objects in such way as maximize contextual understanding of the image. This implies a cyclical interaction between content (or story) understanding of the image and attention shift over it. Based on this observation, we propose novel SOR approach that models this explicit top-down cognitive pathway with two novel modules: a story prediction (SP) module and a guided ranking (GR) module. By formulating content understanding as the image caption generation task, the SP module learns to generate and complete the image captions conditioned on the salient object queries of the GR module, while the GR module learns to detect salient objects and their viewing orders guided by the SP module. Extensive experiments SOR benchmarks demonstrate that approach outperforms oftee--f-thee-e art SOR methods. Codes are available here.
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Salient Object Ranking (SOR) aims to model the human attention shift across salient objects in scene, by detecting and ranking a sequence of salient objects with their corresponding viewing orders. SOR can facilitate human visual behavior understanding (Lin et al., 2024) and various downstream computer vision tasks, such as scene understanding (li et al., 2023; Du et al., 2019; Zhang et al., 2014) and autonomous driving (Huang Wang, 2024).
Islam et al. (2018) first propose to rank the saliency degrees of objects by predicting a relative saliency map based on the consensus degrees of multiple viewers. Later, Siris et al. (2020) propose the SOR task to study how humans shift their attentions across salient objects, with a neural method to predict saliency ranks based on modeling the relation between objects and global contexts. Liu et al. (2021a) propose a graph-based network to learn relations among objects and local contexts. Tian et al. (2022a) propose to leverage both spatial and object-based attention mechanisms, which used in the human visual system, model the bidirectional object-context relations. Recently, Guan & Lau (2024b) propose to model the processing of visual information and attention shifts in the human visual system by incorporating both foveal and peripheral vision for SOR. Qiao et al. (2024) propose to model the impact of scene context on attention shifts by constructing a scene graph to reason the saliency ranks. Guan & Lau (2024a) demonstrate that human pose affects the observer's attention, and propose to incorporate human poses to infer SOR. All these existing methods primarily consider bottom-up factors (i.e., visual features and semantic information) that influence human attention (Ramos Gameiro et al., 2017).
In this work, we observe that when given an image for free viewing, our brain instinctively engages in scene perception to maximize contextual understanding (Murlidaran & Eckstein, 2024; Rayner & Pollatsek, 1992), with fixations concentrated on objects that are critical for comprehending the overall scene (Murlidaran & Eckstein, 2024). This process can be viewed as cyclical interaction between scene perception and eye movement, studied as active perception and predictive processing in cognitive science (Rao & Ballard, 1999; Peelen et al., 2024; Zacks et al., 2007; Berman & Colby, 2009). In other words, human attentions are continuously shifted across salient objects, driven by the evolving scene-level understanding. The observer first focuses objects that are essential for understanding the scene and potentially forms predictions about the story behind the scene. This perceptual process then guides the observer's attention shift, which turn shapes the predicted story as it moves through the scene. The attention continues shifting until the final salient object is reached, at which point the predicted story stabilizes and the observer gains a more complete understanding of the image content, as illustrated in Fig. 1(a).
Motivated by the above observation, we propose in this paper a novel object query-based method to model the cyclical interaction of human perception and attention shift for SOR. Our method has two novel modules, the story prediction (SP) module and the guided ranking (GR) module. By formulating the "contextual understanding" as the image caption generation process, the SP module predicts and refines the image caption conditioned on the current saliency ranking result from the GR module. Meanwhile, the GR module learns to refine the saliency ranks while incorporating the text modality from the SP module as guidance. The SP and GR modules perform synergistically to summarize the input image through caption generation while predicting the attention shift across salient objects. As shown in Fig. 1(b), our method can self-refine its predictions through the cyclical perception-viewing interaction, producing more faithful ranking results compared the state-of-the-ant methods. In summary, we make the following main contributions:
1. We propose novel approach that incorporates top-down cognitive process (scene understand- ing), which is inspired by psychological studies, for SOR. The key idea is to model an explicit cyclical interaction between content perception and human attention shift in free-viewing.
2. Our SOR approach has two novel modules: a story prediction (SP) module and a guided ranking (GR) module. The SP module simulates the brain's process of predicting the story behind the scene using a generative captioning model, conditioned on the latest predicted saliency ranks, while the GR module predicts the saliency ranks by iteratively refining object queries, guided by the latest predicted story.
3. Extensive experiments demonstrate the effectiveness of our model, and that our method outper- forms the ofte-offthheart SOR methods.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Salient Object Ranking (SOR). When viewing an image, humans typically shift their attentions across salient objects sequentially. Islam et al. (2018) make the first attempt to rank saliency degrees based on the consensus degrees of several observers, which ignores the visual/spatial relations of ob- jects in the scene. Following psychological and behavioral studies (Itti & Koch, 2000; Neisser, 2014), Siris et al. (2020) propose the SOR task to study human attention shift across objects in an image, and neural network to model the relations between objects and the global image context for SOR. Some SOR methods are subsequently proposed to enhance the SOR performance by incorporating object position coordinates (Fang et al., 2021a), modeling inter-object relations via neural graphs (Liu et al., 2021a), and incorporating both object-based and spatial attention mechanisms (Tian et al., 2022a).
Recently, Guan & Lau (2024b) propose to model sequential viewing and attention shifting by using foveal vision to focus on an object and peripheral vision to locate the next object. Qiao et al. (2024) propose hyper-grape graph-basphhee network, while Deng et al. (2024) propose a tri-tiered nested Graph Neural Network, to incorporate object-context relationships, for SOR. Wu et al. (2024) construct a graph for each scene, while explicitly using the shape and texture features of objects as graph edges for SOR prediction. Guan & Lau (2024a) propose to model human poses as cues to help enhance SOR performances. Most recently, Liu et al. (2025) exploit the implicit orders in the descriptions of Large Vision-Language Models (LVLMs) to guide SOR.
All the above works rely on bottom-up image features cues (i.e., poses, scene contexts, object attributes, and inter-object relations) for SOR predictions, which may not be reliable enough to faith- fully reproduce human attention shifts over salient objects. In this work, we propose incorporate the top-down cognitive process as guidance for SOR, by explicitly modeling the cyclical interaction between image content perception and human attention shifts.
Salient Object Detection (SOD). This task aims to identify the most visually conspicuous objects in an image, and has been extensively studied. Early SOD methods (Cheng et al., 2014; Klein Frintrop, 2011; Perazzi et al., 201 2; Achanta et al., 2009) primarily rely on low-level image features, such as contrast, edge, and structure responses, to construct saliency maps. Subsequently, a large number of deep learning based SOD methods (Liu et al., 202 Siris et al., 2021; Wang et al., 2023; Wei et al., 2020; Zhang et al., 2019; Veksler, 2023; Tian et al., 2023; Li et al., 2024) are proposed. Their models typically incorporate multi-scale feature fusion (Liu et al., 2021b; Wang et al., 2023), contextual semantic aggregation (Siris et al., 2021; Zhang et al., 2019), multi-tasking (Wang et al., 2018; Zhang et al., 2019; Wei al., 2020; He al., 201 2017b), and attention mechanisms (Liu et al., 2018; Zhang et al., 2018), to enhance spatial coherence and semantic awareness. However, as SOD methods neither differentiate salient instances of the same class nor do they estimate the attention shift across objects, they cannot be directly applied to the SOR task.
Salient Instance Detection (SID). It aims to identify each salient object at the instance level. Early SID methods (Fan et al., 2019; Wu et al., 2021) are predominantly based on the Mask R-CNN (He et al., 2017a) architecture. They first detect object instances through region proposal networks and then learn discriminative features to distinguish salient instances from non-salient ones using pixel- level supervision. To reduce the reliance on costly pixel-wise annotated masks for training, some recent works tend to propose weakly-supervise
(Tian et al., 2020; 2022b) or unsupervised (Tian et al., 2024) approaches. While SID methods can provide instance-wise segmentation of salient objects, they do not attempt to predict the attention shift across these objects.
3 OUR METHOD
In this work, we observe that scene perception can significantly influence visual behavior during free viewing, while attention in turn shapes how our brain understands scene, indicating that modeling of perception-viewing cycle can help facilitate the understanding of how our visual attention mechanisms operate in free-viewing real-world scenes. Inspired by this, we propose novel multi-task cyclical learning framework that synergizes Story Prediction (SP) and Guided Ranking (GR) to emulate this cognitive process for Salient Object Ranking (SOR).
Section 3.1 introduces our overall architecture that integrates image captioning (from SP) and saliency ranking (from GR). The Story Prediction (SP) module (Section 3.2) implements contextual
3

---

Published as a conference paper at ICLR 2026
Guided Ranking (GR) Module
Segmentation Head
Predicted Saliency masks and rankings
OJOOjeott queriss
Ranking Head
Query Decoder
feat
High
Low
Backbone
Og J SITA
CMQC
Interaction
Predicted image captions
Text
Decoder
(T) woman and man are preparing food the kitchon.
features
Caption Generator
Story Prediction (SP) Module
Qg Object queries
Text features
CMQC
Modsl Query Contextualization
SITA
Saliency-Infueed Textual Augmentation
Figure 2: Overview our proposed architecture. This framework comprises two key novel modules: the Guided Ranking (GR) Module and the Story Prediction (SP) Module. The GR Module generates saliency masks and rankings via object queries and cross-modal interactions. while the SP Module produces image captions by integrating text features with Object queries through the Story Prediction process. During the Interaction process, the two modules collaborate and iteratively update the visual and textual representations, achieving collaborative enhancement visual saliency ranking and image caption generation.
understanding as dynamic caption generation process, where saliency degree features from GR iteratively influence the refinement of the caption. Concurrently, the Guided Ranking (GR) module (Section 3.3) learns to predict the focused salient instance at each step, explicitly incorporating the text modality generated by SP as guidance to determine subsequent salient instances. Through simultaneous optimization, SP and GR jointly perform the SOR task by generating captions and estimating viewing orders. Finally, Section 3.4 details our training strategy.
3.1 OVERVIEW
As shown in Fig. we propose a rank and image description through iterative interaction between object queries and text features. The framework consists of vision backbone, Query Decoder, a Text Decoder, a Guided Ranking (GR) module, Story Prediction (SP) module, and the interaction between the GR and SP modules.
framework that achieves joint reasoning of saliency
The input image first fed into the backbone to extract a set of image features. These features are then used to construct feature pyramid feats; E xGxxH i {2, 4, via feature extractor (Cheng et al., 2022), where Ct, H,, W; denote the number of channels, height, and width of the i-th feature map, respectively. We initialize set of object queries Qo E TR. NDD with learnable parameters, where N and D represent the number and dimension of the queries. A transformer- decoder with L layers is employed to iteratively enhance the query representations by attending to multi-scale object features:
Q
i= (1-1) 1) mod 3 + 3,
where E 21,2,.... L} and Q: represents output object queries after th query decoder layers. The multi-layer object queries are then aggregated to obtain the global query Q2 E NxD which are fed into the ranking head to predict saliency scores S E N
S Linear (Q.).
(2)
Mcanwhile, the backbone visual features are projected obtain image embeddings Eing E DD where Np denotes the number of image patches. The embeddings Fing align with the dimensions of the textual space Dr, and serve the cross-modie context for autoregressive text generation. Starting from a special "[BOS]" token, the decoder autoregressively generates caption tokens. At each step it takes input the embeddings of previously generated tokens, Xx: [xo...., Xt-1] R along with the image embeddings Emn.. to produce the current hidden state h, E Dt:
h TextDecoder((xex Fang)-
4

---

Published as a conference paper at ICLR 2026
After generating all Lg tokens, we obtain H [hi:...; hy which can be projected to the vocabulary space to generate a descriptive text T, as:
R as the full text features,
T = arg max weV wEVocab
softmax((inear(H)
Cyclic Interaction. We establish a cyclic interaction between Q2 and H through two modules: the Guided Ranking (GR) module and the Story Prediction (SP) module. Our SP module learns to enhance text features via the Saliency-Inussed Textual Augmentation (SITA), and then generate descriptive language by caption generator. The enhanced text features (R) can be computed as:
H (k) = SITA(Q, ((*-1), " (()
where k is the index of iterations. Concurrently in the GR module, the enhanced text features (k) contextualize the global query Q2 through the Cross-Modal Query Contextualization (CMQC) mechanism, allowing it to learn image-perception features, as:
(k) CMQC(Q (H()).
(6)
This iterative process is repeated for K steps. The final object query Q₂ !K) fed into a ranking head to predict saliency scores, as described in Eq. 2. Meanwhile, the final text features H()) are decoded into natural language captions through generator, following Eq. 4. The overview of SITA and CMQC are shown in Fig. 3.
3.2 STORY PREDICTION (SP) MODULE
We propose the Story Prediction (SP) module to simulate the brain's perception process by formu- lating scene understanding as generating image captions. The SP module establishes the viewing- to-pereeption pathway. where saliency-related information influences the generation of linguistic descriptions. Specifically, it iteratively injects object query features into text features, enabling the model to ground textual narratives visually salient regions.
The SP module consists of a Saliency-Infused Tex- tual Augmentation (SITA) module for modality- aligned feature fusion and caption generator to progressively align linguistic descriptions with hu- man attention patterns. The SITA module takes the global query Q (k-1) E NxD and its associated saliency scores ((k- E gxx1 as inputs (Eq. 5). SITA first computes saliency-weighter visual con- text vector through an clement-wise multiplication of queries with saliency scores and then spatial averaging along the object dimension, yielding a compact visual representation Vsal E IR as:
H
GELU
Cross- Attention
Linear
Valo
Proj.
Lincar
gg
(h-1)
H
1)
GELU
MLP
Linear
Sigmoid
Proj.
Gate
HIM
H
N Σ N
Element-wise
Addition MMultipliaation
V
(7)
CMQC module
SITA module
where @ denotes element- multiplication. This vector Vai then projected to align with the text feature dimension D, and broadcast to match the text sequence length L,, resulting in Valign E RD.
Figure 3: Cross-Modal Query Contextualization (CMQC) module and Saliency-Infused Textual Augmentation (SITA) module. CMQC contextu- alizes object queries with text semantics via cross- modal attention, while SITA injects saliency-guiee guidee visual cues into text features through adaptive gating. Jointly, they enable iterative refinement visual- language representations for saliency ranking and caption generation.
These expanded features Valign are then processed sal by gating mechanism, which is structured as two-layer neural network with GELU activation and sigmoid normalization. The generated gate G E RL.x1 dynamically scales the outputs of MLP network applied to the original text features. A residual connection preserves the baseline linguistic patterns while allowing controlled infusion of saliency information. The whole process can be described as:
G o (GELU( Valalggaaaaaa
align W1 + b
Wa + b2).
(8)
5

---

Published as a conference paper at ICLR 2026
H MLP((((-))) o G ((11))
This gating mechanism cognitively inspired, which mimics neural gain modulation (Peelen et al., 2024) by adaptively scaling textual features based on saliency information, aligning with attentional modulation theories while ensuring saliency-driven augmentation of text features (see Appendix A.1).
3.3 GUIDED RANKING (GR) MODULE
The Guided Ranking Module predicts object viewing order by enhancing the object queries through perception- terpiiintto-vieiing pathway with two key components: (1) the Cross-Modal Query Contextualiza- tion (CMQC) that iteratively refines query representations using linguistic features, and (2) ranking head that outputs saliency scores after refinement iterations.
The CMQC module first projects nigh-dimensional text features into latent space commensurate with object query embeddings. Given input text features H E R RL.xD D: from the text decoder, a learnable linear transformation with layer normalization adapts them to the query dimension (IR 2), enabling cross-modal interaction while preserving the linguistic structure via normalization constraints.
Subsequently, we employ multi-head cross-attention mechanism iteratively refine semantic representations across steps. In each iteration k object queries (k) E R R interact with text features via scaled dot-product attention, enabling targeted alignment with relevant linguistic cues (e.g., associating clothing-related queries with tokens like "striped shirt"). The attention weights adaptively fuse contextualized textual semantics into the queries through residual updates:
0(++1) +(*) + MultiHeadAttn(Q. (*)).
(10)
This residual architecture preserves spatial priors while progressively integrating cross-modal se- mantics. Over K iterations, the queries evolve to emphasize contextual features while suppressing irrelevant linguistic noise. This iterative refinement process is analogous to predictive coding in the cognitive system (Rao & Ballard, 1999), where the residual update minimizes the prediction error between object queries and their expected values under textual guidance (see Appendix A.2). The final saliency ranking scores are computed following Eq. 2.
3.4 TRAINING Loss
Our model is trained in an end-to-end manner with the loss function consisting of four terms, as:
= Ltask + Lrank + Lim-
(11)
ftask follows the loss configuration of Mask2Former (Cheng et al., 2022). It includes Lmas for predicting instance masks and Cets for determining whether each instance salient object, as:
Lrack Lmack + Lete:
(12)
where Lmask adopts the binary cross-entropy loss and the dice loss. L.ts adopts the cross-entropy loss. Lyank is the saliency ranking loss (Liu et al., 2021a). fim is the cross-entropy loss compute the difference between the generated and the ground truth image captions.
4 EXPERIMENTS
Implementation Details. We employ Swin Transformer pre-trained on the MS-COCO (Lin et al., 2014) training set the backbone for feature extraction, and pre-trained BLIP's text decoder to generate ()). For each image, we randomly select one out of its five corresponding captions from the MS-COCO dataset as the ground-truth caption. Our model initialized with configuration parameters 2=200, K=5, and D=256, trained end-to-end without layer freezing across four RTX 3090 GPUs, with all input images resized to a 1024 x 1024 resolution. We employ the AdamW W optimizer with le -4 weight decay, and train model for 24,000 iterations with batch size of 4. The learning rate is initially set to 2.5e and reduced by 10 after 14,000 iterations. During inference, objects with confidence scores
0.7 regarded as salient ones for follow-up ranks prediction.
Evaluation Datasets. We conduct experiments on the publicly available SOR benchmark datasets, ASSR (Siris al., 2020) and IRSR (Liu al., 2021a). The ASSR dataset contains 7, 7,646 training

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative Comparison. SOD: Salient Object Detection task. SID: Salient Instance Detection task. INS: Instance Segmentation task. SOR: Salient Object Ranking task. Best results are marked in bold and second-best best results are underlined. indicates that the result is not available.
Methods
Venues
Tasks
Backbone
ASSR Dutaset
IRSR Dataset
SA-SOR
SOR
MAR.
SA-SOR 1
SOR
MAR.
S4Net (Pan at, 2019)
CVPR-2019
ResNes-50
0.451
0.649
14,4
12.1
VST (Lio et al., 20210)
ICCV-2021
SOD
T2T-VIT-T
0.422
0.643
9.99
0.183
0.571
8.75
MENet (Wang
2023)
CVPR-2023
SOD
ResNet-50
0.369
0.627
9.60
0.162
0.558
8.25
QueryInat Inst (Pang et al. 202th)
10CV-2021
INS
ResNet-101
0.596
0.865
8.52
0.538
7.13
Mask (Chong ot al., 2022)
CVPR-2022
INS
RosNet-101
0.635
0.867
7.31
0.521
0.799
7.14
RSDNer (Islam al. 2018)
CVPR-2018
SOR
ResNet- 101
0.386
0.692
18.2
0.326
0.663
18.5
ASRNet (Siris et al, 2020)
CVPR-2020
SOR
ResNat-101
0.590
0.770
9.39
0.346
0.681
9.44
PPA (Fang et al., 2002,
1CCV-2021
SOR
VoVNet.39
0.635
0.86.
8.52
0.521
0.797
8.08
(Liu 2022
TPAMI-2021
SDR
ResNet-50
0.650
0.854
7,79
OCOR ct.., 20222)
CVPR-2022
SOR
Swin-1
0.541
0.873
10.2
0.504
0.820
8.45
PSR (Sun et al, 2023)
ACMMM-2023
SOR
ResNet-50
0.644
0.815
9.59
0.454
0.752
8.07
HyperSOR (Qiao al., 2024)
TPAMI-2024
SOR
ResNet-10 101
0.653
SeqRank (Gian & Lan, 2024b)
AAAI-2024
SOR
Swin-1.
0.663
0.863
8.03
0.554
0.801
7.51
QAGNet (Deng al, 2024)
CVPR-2024
SOR
Swin-L
0.771
0.857
5.78
0.616
0.818
6.71
DSGNN
2024)
CVPR-2024
SOR
Swin-L
0.761
0.856
5.41
7.01
PoceSOR (Gumn & Lan, 2024a)
ECCV-2024
SOR
Swin-1.
0.667
0.856
7.87
0.551
0.812
7.32
Ours
SOR
Swin-L
0.787
0.869
5,28
0.624
0.822
6.89
images, 1,436 validation images, and 2,418 test images, with each image annotated with up to five salient instances ranked by saliency levels. The IRSR dataset includes 6,059 training images and 2,929 test images, with each image containing up to eight ranked salient instances.
Evaluation Metries. We employ three widely-used evaluation metries for the SOR task: (1) Mean Absolute Error (MAE), which measures pixel-level discrepancies between predicted saliency instance masks and ground truth annotations; (2) Salient Object Ranking (SOR) scores (Islam et al., 2018), which computes the Spearman's rank correlation coefficient to evaluate the consistency between predicted saliency rankings and ground truth rankings. This metric tends not to penalize the detection errors such missed false- possiivee instances; (3) Segmentation et al., 202 Ia) is proposed to correct the above limitation with the SOR score by combining Pearson correlation with detection penalties. It excludes unmatched predictions (missing real objects or detecting fake ones) through instance matching and score suppression, ensuring that the ranking score reflects both detection and ordering accuracy.
SOR (SA-SOR) (Liu
4.1 COMPARISON TO STATE-OF-THE-AN METHODS
Quantitative Comparison. As shown in Table 1, we conduct a comprehensive comparison of the proposed framework with state-of-the-art methods on the standard ASSR and IRSR benchmarks. For fair comparison, we retrain all these methods both the ASSR and IRSR benchmarks. Our method achieves state-of-the SA-SOR scores while maintaining competitive advantages in both SOR and MAE. Particularly, the ASSR dataset, when using the Swin-L backbone, our method outperforms the current best model, QAGNet (Deng et al., 2024), by 1.95% on the SA-SOR metric, while simultaneously reducing MAE by 8.65%. This validates the efficacy of our perception-viewing modeling, where text and query features are mutually influenced and updated during the cycle.
Qualitative Comparison. Fig. 4 demonstrates the superior segmentation and ranking performances of our model, compared to all other methods. Notably, in multiple challenging cases (e.g.. 2nd, 6th, and 8th rows), our method successfully predicts the whole salient object ranks while the results of competing methods exhibit consistent error patterns. The reason is that although other methods also utilize various cues such as shape (Wu et al., 2024) and human pose (Guan & Lau, 2024a), these cues derived solely from the image itself are inherently less representative. For example, in the 4th row, PoseSOR incorrectly shifts the focus from the two individuals in front of the TV to the television itself based on their poses. In contrast, our method leverages the perceptual cue "video games" to first drive our model's focus to the television, and then to the participants who are playing the video games. By integrating this cognitive process, our approach becomes more broadly applicable across a variety of scenarios compared to relying solely on cues such as shapes or human pose.
Another example is shown in the 7th row, where the scene- perceepion cue "carriage" initially directs our method's 's focus to the leading horse, and then shifts to the people on either side, culminating in the understanding that people are riding in carriage. These visual comparisons generally verify that
7

---

Published as a conference paper at ICLR 2026
*
#
Input
RSDNet
ASRNes
IRSR
OCOR
SeqRank
QAGNet
DSGGNN
PrxeSOR
Ours
GT
Figure 4: Visual comparison between results of our method and those of eight state-of-the-art methods. Our method produces more faithful salient object ranking results.
modeling perception- viewing cycle in our approach enables reasoning the sequence of attention shifts during scene comprehension. Refer to the Appendix B for more visual comparisons.
4.2 INTERNAL ANALYSIS
To validate the effectiveness of each proposed module and design choice, we conduct thorough ablation studies on the ASSR benchmark.
Analyzing the Model Components. We first
Table 2: Ablation analysis of different techniques the proposed SP and GR modules. ( denotes the saliency of object queries in each step.
evaluate each module's efficacy on the ASSR benchmark systematically through controlled ab- lation studies. Table 2 shows the results. We begin with the baseline method where object queries from the query decoder directly fed into segmentation and ranking heads (denoted "T"). We then gradually introduce ponents into the baseline method as: adding captioning supervision (denoted as "II"), incor- porating the CMQC module (denoted as "III"), exploiting the saliency reweighing and the gating mechanism separately (denoted as "IV" and "V". respectively) and jointly as the whole SITA module (denoted "VI"). We can see from Table 2 that while the simple baseline method may not perform well, gradually incorporating the proposed techniques (from II to VT) brings performance gains continuously under all three metrics.
SITA
Settings
Caption
CMQC
SA SASSOR1
SOR
MAE
S
Gate
U1667
0.722
0.547
6,83
1.729
0.549
IV
0.734
0.517
6.21
0.748
0.854
6.27
Analyzing the Numbers of Iterative Steps. We then evaluate the impact of different numbers of iterative steps within the proposed cyclic interaction. We report the ablation results in Table where "Selection of S" indicates whether saliency scores are derived from initial object queries before interaction (denoted as "First") or final refined queries after the interaction process (denoted "Last"). By comparing settings I and II, enhances the accuracy of salient object ranking, which verifies our core idea of building the cyclical perception-viewing interactions. The comparison among settings II, III, and IV shows that increasing
see that the interaction mechanism significantly
8

---

Published as a conference paper at ICLR 2026
the number of interaction steps tends to produce better SOR performances and achieve the best performance when the number of steps is set to 5, while we observe the MAE tends to be saturated.
Table 3: Ablation study on the number of iter- ative steps. "Selection of S indicates whether saliency scores are computed from object queries be- fore ("First") or after ("Last"" the interaction process.
Table 4: Ablation study on caption refinement.
Refine
Settings
Number of Steps
CIDEr
SPICE
Last
Selection of SA-SOR
Settings
Steps
SOR
MAB
First
Last
0.397
0.125
III
1
0.116
0.138
II
<
0.747
0.848
5.81
0.433
0.149
III
0,754
0.851
5.74
IV
0.767
0.856
5.73
VI
0.457
0.158
Analysis of Caption Refinement. We now analyze the impact of the number of iteration steps on caption generation. The results are shown in Table 4, where "First" and "Last" represent whether the hidden states for cap- tion generation are selected before or after the proposed interaction. We report the CIDEr and SPICE metrics for evaluating image cap- tion quality, with CIDEr focusing assessing content consistency and SPICE focusing on assessing the quality of semantic information. In setting upon the shared backbone, which does not interact with object queries (aligned with Table 2 Setting II.. From settings II to V. gradually increase the number of interaction steps. The comparison between settings and demonstrates that the interaction can enhance the hidden state representa- tion, while the comparison between II and III reveals the effectiveness of refining the hidden state. Settings IV to VI show that setting the number of iterations to 5 achieves the best caption generation performance, which we use in this work. We also discuss an early stopping criterion Appendix C. Fig. 5 shows comparison of generated caption before and after refinement.
Initial H
Stuffed
a table.
iitrations
Stuffed bears sitting around table.
RefinedH
Figure 5: comparison of generated caption before and after refinement.
independent text decoder branch
Analysis of Semantic Density In free- free-viewing, humans dynamically assign and adjust their attention to different objects in way that can maximize their contextual comprehension. Our method models this cyclical interaction between scene perception and viewing for saliency ranking. Consequently, method is expected to be more effective in semantically rich scenarios compared to simple ones. To verify this, we first define the semantic density p of an image as the ratio of the number words in the ground truth caption to the number of salient objects:
round Number of Words in Caption
(13)
Number of Salient Objects
Images with higher semantic density contain richer contextual information. We then randomly select 600 images from the ASSR test set and compute p for each image. We group these images based on values and calculate the mean SA-SOR score for each group. The results are shown in Table 5.
Table 5: Mean SA-SOR scores grouped by semantic density 600 images from the ASSR test set.
7
8
9
10
11
12
13
14
15
Mean SOR
0.796
0.747
0.774
0.838
0.819
0.629
0.725
0.808
0.90
0.909
0.880
0.895
0.944
1.0
further compute the Pearson correlation coefficient between p and SA-SOR across all 600 images. As shown in Table 6, o and SA-SOR exhibit strong positive linear relationship. indicating that model performs better on images with higher semantic density.
Model Efficiency Analysis We provide comprehensive analysis of the computational efficiency and runtime performance of our proposed method. All experiments are conducted on a single NVIDIA RT'X 3090 GPU. Table 7 summarizes the inference time and frames per second (FPS) for our method

---

Published as a conference paper at ICLR 2026
Table 6: Pearson correlation between semantic density p and SA-SOR on the ASSR test subset.
Dataset
Mean
Pearson
p-value
ASSR-test (600 images)
6.07
0.714
0.00416
under various configurations (backbone, input resolution, and number of cyclical steps K), alongside comparisons with other state-of-the -o-ttt---ttt-ttt methods for reference.
Table 7: Comprehensive runtime performance (inference time / FPS) RTX 3090.
Method
Backhone
512x512
768x768
1024x1024 (K
1024x1024 (K 5)
Swin-L Swin-B
70ms 14.3
118ms
191ms 5.2 136ms 7.3
205ms 148ms / 6.8
Ours
49.8ms 20.1
80.7ms / 12.4
PoseSOR (Guan & Lau, 2024a)
Swin-L
98ms / 10.2
152ms / 6.6
QAGNet (Dong et al,, 2024)
Swin-L
294ms / 3.4
384mms / 2.6
As shown in the table, the majority of the computational cost originates from the Transformer- based backbone, whose attention mechanism has a quadratic complexity with respect to spatial resolution (O((HW)²)). In contrast, increasing the cyclical interaction steps from K introduces minimal overhead, merely adding 14ms (approximately 7% at 1024x 1024 for Swin-L.). When configured for speed (Swin-B) backbone), our method achieves interactive speeds, reaching 20.1 FPS at 512x512 12 and 12.4 FPS at 768x768 resolution, with a modest performance trade-off (SA-SOR decreases by 2.0% and 3,5% at 768x768 and 512x512 12 compared to 1024x 1024, respectively). In high-accuracy setting (Swin-L 1024x 1024), method operates at 5.2 FPS (K 1) and 4.9 FPS (K = =), which is competitive among Transformer- baased SOR methods.
K
CONCLUSION
In this paper we have proposed to model the cyclical interaction between perception and viewing for SOR. Our method introduces two
Ours
GT
SOR: 0.5
key components: the Story Prediction (SP) module, which simulates human perceptual process through image caption generation, and
SA-SOR: 0.5
GT caption: A display at grocery store filled with fruits and vegetables next to jars.
Guided Ranking (GR) module, which pre- diets saliency ranking under SP's guidance. Through iterative cross-modal refinement, ob- ject queries GR and textual features in SP interact dynamically, effectively mimicking human-like perception-viewing cycles. Exten- sive experiments on SOR benchmarks demon- strate the superior performance of our method. Nevertheless, our method does have limita- tions. When the scene semanties is weak, the guidance provided by the perceptual process of our model may be limited. As illustrated in Fig. 6, our method may fail in scenes where the saliency of objects is mainly determined by low-level features such as colors, shapes, and positions.
Ours
GT
SOR: 0.4
SA-SOR: 0.4
GT caption: An ussortment of cooking utensils including measuring cup scissors.
Figure 6: Failure cases. Our model may fail to predict the correct saliency rank when the semantic information of the seene is relatively weak, in which the ground truth caption (from humans) may not provide much information related each salient object.
REFERENCES
Radhakrishna Achanta. Sheila Hemami, Francisco Estrada, and Sabine Susstrunk. Frequency-tuned salient region detection. In CVPR, 2009.
Rebecca Berman and Carol Colby. Attention and active vision. Vision Research, 2009.
10

---

Published as a conference paper at ICLR 2026
Bowen Cheng. Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer for universal image segmentation. In CVPR, 2022.
Ming-Ming Cheng, Niloy J Mitra, Xiaolei Huang, Philip HS Torr, and Shi-Min Hu, Global contrast based salient region detection. IEEE TPAMI, 2014.
Bowen Deng, Siyang Song, Andrew P French, Denis Schluppeck, and Michael P Pound. Advancing saliency ranking with human fixations: Dataset models and benchmarks. CVPR, 2024.
Lan Du, Lu Li, Di Wei, and Jiashun Mao. saliency-guidees single shot multibox detector for target detection sar images. IEEE TGRS, 2019.
Ruochen Fan, King-MMing Cheng, Qibin Hou, Tai-Jiang Mu, Jingdong Wang, and Shi-Min Hu. S4net: Single stage salient-instane instance segmentation. In CVPR, 2019.
Hao Fang, Daoxin Zhang, Yi Zhang, Minghao Chen, Jiawei Li, Yao Hu, Deng Cai, and Xiaofei He. Salient object ranking with position-preserved attention. ICCV, 2021a.
Yuxin Fang, Shusheng Yang, Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. Instances as queries. In ICCV, 2021b.
Huankang Guan and Rynson W. H. Lau. Posesor: Human pose can guide our attention. In ECCV, 2024a.
Huankang Guan and Rynson W.H. Lau. Seqrank: Sequential ranking of salient objects. In AAAI, 2024b.
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In
Shengfeng He, Jianbo Jiao, Xiaodan Zhang, Guoqiang Han, and Rynson WH Lau. Delving into salient object subitizing and detection. ICCV, 2017b.
Yaoqi Huang and Xiuying Wang. Hazards prioritization with cognitive attention maps for supporting driving decision-maie making... IEEE TITS, 2024.
Md-Amirul Islam, Mahmoud Kalash, and Neil Bruce Revisiting salient object detection: Simultane- ous detection, ranking, and subitizing of multiple salient objects. CVPR, 2018
Laurent Itti and Christof Koch. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision research, 2000.
Dominik A Klein and Simone Frintrop. Center-surround divergence of feature statistics for salient
object detection. ICCV, 2011.
Feiran Li, Qianqian Xu, Shilong Bao, Zhiyong Yang, Runmin Cong, Xiaochun Cao, and Qingming Huang. Size-invariance matters: Rethinking metrics and losses for imbalanced multi-object salient object detection. In ICML,, 2024.
Hao Li, Dingwen Zhang. Nian Liu, Lechao Cheng. Yalun Dai, Chao Zhang, Xinggang Wang, and Junwei Han. Boosting low-data instance segmentation by unsupervised pre-training with saliency prompt. In CVPR, 2023.
Jayyng Lin, Shuquan Ye, and Rynson WH Lau. Do multimodal large language models see like humans? arXiv:2412.09603 2024.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitiick. Microsoft coco: Common objects in context. ECCV, 2014.
Fang Liu, Yuhao Liu, Ke Xu, Shuquan Ye, Gerhard Petrus Hancke, and Rynson WH Lau. Language- guided salient object ranking. In CVPR, 2025.
Nian Liu, Junwei Han, and Ming-Hsuan Hsuan Yang. Picanet: Learning pixel-wise contextual attention for saliency detection. In CVPR, 2018.
11

---

Published as a conference paper at ICLR 2026
Nian Liu, Long Li, Wangbo Zhao, Junwei Han, and Ling Shao. Instance-level relative saliency ranking with graph reasoning. IEEE TPAMI, 2021a.
Nian Liu, Ni Zhang, Kaiyuan Wan, Ling Shao, and Junwei Han. Visual saliency transformer. In ICCV, 2021b.
Shravan Murlidaran and Miguel P Eckstein. Eye movements during free viewing maximize scene understanding. Journal of Vision, 2024.
Ulric Neisser. Cognitive psychology: Classic edition. Psychology Press, 2014.
Marius V Peelen, Eva Berlot, and Floris P de Lange. Predictive processing of scenes and objects. Nature Reviews Psychology, 2024.
Federico Perazzi, Philipp Krähenbühl, Yael Pritch, and Alexander Hornung. Saliency filters: Contrast based filtering for salient region detection. In CVPR, 2012.
Minglang Qiao, Mai Xu, Lai Jiang, Peng Lei, Shijie Wen, Yunjin Chen, and Leonid Sigal. Hypersor: Context-aware graph hypernetwork for salient object ranking. IEEE TPAMI, 2024.
Ricardo Ramos Gameiro, Kai Kaspar, Sabine U König, Sontje Nordholt, and Peter König. Exploration and exploitation in natural viewing behavior. Scientific Reports, 2017.
Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 1999.
Keith Rayner and Alexander Pollatsek. Eye movements and scene perception. Canadian Journal of Psychology, 1992.
Avishek Siris, Jianbo Jiao, Gary K.L. Tam, Xianghua Xie, and Rynson W.H. Lau. Inferring attention shift ranks of objects for image saliency. In CVPR, 2020.
Avishek Siris, Jianbo Jiao, Gary KL. Tam, Xianghua Xie, and Rynson WH Lau. Scene context-aware salient object detection. In ICCV, 2021.
Chengxiao Sun, Yan Xu, Jialun Pei, Haopeng Fang, and He Tang. Partitioned saliency ranking with dense pyramid transformers. In ACM MM, 2023.
Xin Tian, Ke Xu. Xin Yang, Baocai Yin, and Rynson WH Lau. Weakly-supervies salient instance detection. In BMVC, 2020.
Xin Tian, Ke Xu, Xin Yang, Lin Du, Baocai Yin, and Rynson WH Lau. Bi-directional object-context prioritization learning for saliency ranking. CVPR, 2022a.
Xin Tian, Ke Xu, Xin Yang, Baocai Yin, and Rynson WH Lau. Learning to detect instance-level salient objects using complementary image labels. IJCV, 2022b.
Xin Tian, Ke Xu, and Rynson Lau. Unsupervised salient instance detection. In CVPR, 2024.
Xinyu Tian, Jing Zhang, Mochu Xiang, and Yuchao Dai. Modeling the distributional uncertainty for salient object detection models. In CVPR, 2023.
Olga Veksler. Test time adaptation with regularized loss for weakly supervised salient object detection. In CVPR, 2023.
Wenguan Wang, Jianbing Shen, Xingping Dong, and Ali Borji. Salient object detection driven by fixation prediction. In CVPR, 2018.
Yi Wang, Ruili Wang, Xin Fan, Tianzhu Wang, and Xiangjian He. Pixels, regions, and objects: Multiple enhancement for salient object detection. In CVPR, 2023.
Jun Wei, Shuhui Wang, Zhe Wu, Chi Su, Qingming Huang, and Qi Tian. Label decoupling framework for salient object detection. In CVPR, 2020.
Yu-Huan Wu, Yun Liu, Le Zhang, Wang Gao, and Ming-Ming Cheng. Regularized densely-connectee pyramid network for salient instance segmentation. IEEE TIP, 2021.
12

---

Published as a conference paper at ICLR 2026
Zijian Wu, Jun Lu, Jing Han, Lianfa Bai, Yi Zhang, Zhuang Zhao, and Siyang Song. Domain separation graph neural networks for saliency object ranking. In CVPR, 2024.
Jeffrey M Zacks, Nicole K Speer, Khena M Swallow, Todd S Braver, and Jeremy R Reynolds. Event perception: mind-brain perspective. Psychological bulletin, 2007.
Fan Zhang, Bo Du, and Liangpei Zhang. Saliency-guinn unsupervised feature learning for scene classification. IEEE TGRS, 2014.
Lu Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu, and You He. Capsal: Leveraging captioning to boost semantics for salient object detection. CVPR, 2019.
Xiaoning Zhang, Tiantian Wang, Jinqing Qi, Huchuan Lu, and Gang Wang. Progressive attention guided recurrent network for salient object detection. In CVPR, 2018.
13

---

Published as a conference paper at ICLR 2026
A THEORETICAL JUSTIFICATION OF CMQC AND SITA
A.1 NEURAL GAIN MODULATION INTERPRETATION OF SITA
The gating mechanism in SITA (Eq. and Eq. 9 of main paper) is inspired by neural gain modulation in biological vision systems (Peelen et al., 2024). The gate G modulates the text features H based on waliency-weighted visual context. This mimics how attentional gain in the brain prioritize high- confidence, salient information and suppresses irrelevant or noisy signals.
A.2 PREDICTIVE CODING INTERPRETATION OF CMQC
Predictive coding theory posits that the brain minimizes prediction errors through iterative residual updates (Rao & Ballard, 1999). We show that the CMQC module (Eq. 10 of main paper) implements a similar mechanism for cross-modal alignment.
Let the optimal object query conditioned textual context be denoted as Qj E[], The cross-attention output in CMQC approximates the prediction error.
MultiHeaddAttml 0 (k))) QQ )
(14)
where a is an effective learning rate. Then the update becomes:
(++1) Q(hk aQ -,(²),,
(15)
which implies:
Q2 (1 a) Q7). (k+1)
(16)
After K iterations:
(K)
Q ~ (1 a
a (Q Qg)
(17)
As K - oc, Q + Q indicating that the residual update minimizes the prediction error. This process is equivalent to gradient descent the loss:
Cpc E [1Q, TE[Q, [[[2||AT]??]
(18)
MultiHeadAAtnnQ.. (k) (k)) - (2) LrC -77 2(0 - Q7),
(19)
where 7) an effective learning rate and corresponds Eq. 14. The proof above show how CMQC
residual achieves predictive error minimization.
B MORE VISUAL RESULTS
We provide more visual results in Figure 7, where our method produces more faithful salient object ranking results compared with state-of-the-art methods.
C EARLY STOPPING CRITERION
Our method employs fixed number of iterative cycles (K 5) to achieve optimal performance in saliency ranking. To accommodate didency-sensitvvy scenarios, we further explore an adaptive strategy that dynamically adjusts the number of iterations. Specifically, define adaptive stopping criterion as follows:
(k)
1)
1|
< E,
(20)
N
14

---

Published as a conference paper at ICLR 2026
where is convergence threshold. When 0.05, the average iteration count is reduced from to 4.1, with approximately 86% of images converging by the fourth cycle. This results a computational saving of approximately 4ms per frame while maintaining model accuracy, with performance degradation remaining negligible (SA-SOR drop below 0.5%)
D MORE ABLATION STUDIES AND EXPERIMENTS
D.1 INFLUENCE OF THE NUMBER OF QUERIES.
We report the results of using different numbers of object queries in the Table 8. Setting the number of queries to 200 is suitable for SOR task. Further increasing the number to 300 degrades the SOR performance as it would introduce background noise. Decreasing the number of queries to 100 also degrades the SOR performance impedes the model's capability to represent objects in complex scenes (for example, small or partially-ocee -cclluee objects).
Table 8: Impact of query numbers on SOR performance.
Query Numbers
SA-SOR1
SORt
100
0.7706
0.8565
200
0.7732
0.8561
300
0.7724
0.8543
D.2 SCALING UP WITH MORE POWERFUL TEXT DECODER
Table 9 shows replacing the BLIP-1 text decoder (in Setting II and VI of Table 2) with a stronger text decoder (i.e., BL.IP-2 OPT-2.7B) helps improve the performance.
Table 9: Comparison of different text decoders under Setting II.
Methods
SA-SOR Setting II (BLIP-1 -BLIP-2)
Setting II (Table of main paper)
0.722
0.739
Setting VI Setting VI (BLIP-1 -LLIP22
0.752
0.762
D.3 SOR-GUIDED IMAGE CAPTIONING.
We further studied whether SOR could help with image captioning. We adopt a strong off-the-shelf MLLM, Owen2.55VVL77B as the baseline model to generate image captions. This baseline model takes input an image and text prompt that asks the MLLM to generate a caption for the input image. We further construct the zero-shot SOR-guided caption generation model based the baseline. Specifically, addition to the input image, we also provide masks for salient objects and their saliency ranks. The prompt explains these saliency ranks to the MILMM (as human attention-shift sequence) and asks to generate the caption. The results (on the ASSR test set) in Table 10 show that SOR guidance brings consistent improvements across all captioning metrics, suggesting that salient object ranking can serve as a useful prior for enhancing image captioning.
Table 10: Comparison of captioning performance with and without SOR guidance.
Method
BLEU
METEORT
ROUGE-LI
CIDErt
SPICE
Baseline
0.0576 0.0795
0.2966
0,2454
0.0883
0.1791
SOR- guided
0.3285
0.2589
0.1395
0.2298
15

---

Published as a conference paper at ICLR 2026
E VISUALIZATION OF INTERACTION
We visualize the cross-modal interaction and results under different values of K.
Figure 8a plots the evolving trajectories of the top-8 ranked queries with the increasing number of interaction steps. We can see that the beginning interactions help our model gradually identify saliency ranks (e.g.. query # #138 is correctly ranked at step 2: query #195 overtakes query #166). The last few interaction steps tend to refine and smooth saliency rank predictions.
In Figure 8b, visualize the L2 norm of each query numerically stable across the interactions, with no sign of numerical explosion collapse, indicating that the cyclic update is stable in terms of feature magnitude.
(8) (1QQtt) 12). which shows that these queries
Figure 8c shows text-query attention heatmaps for the top-50 ranked queries across different in- teraction steps. At the beginning (Step O), most queries only attend sparsely to tokens across the entire caption, with slightly higher responses occasionally at the beginning positions of the sentences. As the interaction proceeds (Steps 1-3), attentions are shifted gradually to focus semantically important tokens, which tends to stable Steps 4-5. This indicates that the cyclic interaction effectively guides the salient instances to focus the words that most relevant to the scene description, rather than diffusing attention over irrelevant tokens.
small set of
Figure 9 visualizes the SOR results of example under different numbers of interaction steps, which shows that the interactions gradually adjust the saliency ranks of the man over the screen.
F THE USE OF LLM
This research does not involve use of Large Language Models (LLMs) its core contributions, such for model training fine-tuning. LLMs were used solely for the purpose of polishing the writing of the manuscript. These uses do not affect the originality or core methodology of the research and therefore do require detailed declaration.
16

---

Published as a conference paper at ICLR 2026
kii
Imput
SegRank
QAGNet
PosSSOR
Figure 7: Visual comparisons between our method and eight state-of-the-art methods.
17

---

Published as a conference paper at ICLR 2026
Saliency Trajectory [Top Queries)
14
Query 86
Query
Query 19
Query 195
Saliency Score
Query
Query 61
Query 63
Query
Interaction Step
(a) Evolving trajectories of top-8 ranked queries across different interaction steps.
Query Step
16.000
Intoraation
18.015
16,010
16.005
50
100
125
150
1.5
Dusrr 1D
(b) norms object queries over interaction steps.
Croce Mode Attection Bot Takner Object Quedes
531
6.4
123
5000
8405
12
(c) Text- attention heatmaps for the top-50 ranked queries across interaction steps.
Figure 8: Qualitative analysis of the cyclic interaction.
18

---

Published as a conference paper at ICLR 2026
High
Step SA-SOR: 0.7
Step SA-SOR 0.8
Figure 9; Visualization of results under different interaction steps.
19


Reviews:
[{"review_id": "9DoP0bVW2p", "reviewer": "Reviewer_oTJC", "summary": "The authors propose a Salient Object Ranking (SOR) approach that consists of two modules: the Guided Ranking (GR) module and the Story Prediction (SP) module, whose interaction enhances the overall performance of SOR.", "questions": "What training data are used for the segmentation head? Was it pre-trained on the COCO segmentation dataset? If it was trained only on the SOR dataset, would its segmentation generalization ability be affected?\n\nDoes the random selection of captions influence the results? It is recommended to include a discussion—for example, are the salient objects in the image always located in the main subject position described in the caption?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The design of this model aligns well with the human cognitive system, such as predictive coding, and the English writing is good and clear.", "weaknesses": "Some experiments and details are not clearly explained. For example, in the experimental section, how was the choice of 24,000 epochs determined, and why such a large number? Could this lead to overfitting?\n\nIn addition, it would be helpful to qualitatively present the interaction between object queries and text features, as well as the results under different values of K.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jcZBdRCZNl", "reviewer": "Reviewer_siW4", "summary": "This paper proposes a novel framework that models the cyclical interaction between perception and viewing for the Salient Object Ranking (SOR) task. The method introduces two key components: a Story Prediction (SP) module that simulates the human perceptual process through image caption generation, and a Guided Ranking (GR) module that predicts saliency rankings under the guidance of the SP module.", "questions": "(1) Explain the differences between the proposed method and Language-Guided Salient Object Ranking (CVPR 2025). Moreover, the performance of this method is still inferior to that of the existing work.\n\n(2) In Eq.(1), when $l=1$, what dose $Q_{l-1}$ refer to?\n\n(3) In Table 2, for Setting II (“independent caption generation”), there is a performance improvement even without interaction between caption and visual features, which is confusing. Could the authors clarify this behavior?\n\n(4) How is the ground-truth (GT) caption obtained?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "（1）Novel Cognitive-Inspired Framework. \nThe paper introduces a cyclical perception–viewing model inspired by human visual cognition, which is strongly supported by established cognitive and psychological theories. And the introduction is easy to follow.\n\n（2）Extensive experiments.\nThe paper conducts both qualitative and quantitative experiments, and also provides an analysis of inference time. Moreover, the visualized experimental results clearly and intuitively demonstrate the improvements achieved by the proposed method.", "weaknesses": "（1）The paper lacks a clear comparison with the recent top-down method, Language-Guided Salient Object Ranking (CVPR 2025), and its performance remains inferior to the results reported in that study.\n\n（2）In the Method Overview section, the symbols used in the equations do not correspond to those shown in Figure 2, which makes it confusing to understand the inputs and outputs of each module.\n\n（3）The experimental section mainly provides data and setup details but offers limited analysis or discussion to explain the observed results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dHaAmMQMun", "reviewer": "Reviewer_5n6p", "summary": "This paper proposed a method for saliency object ranking. A story prediction module predicts the caption of the image and a guided ranking module predicts the saliency rankings. The cyclical interaction module aligns and refines the caption and the ranking iteratively. The experimental results seemed to show the proposed method outperformed previous SOTA.", "questions": "- Is the segmentation head a pretrained segmentation model or else? A strong segmentation model could favor the MAE.\n- What is the impact of number of object queries on results? An ablation study will be beneficial to see the impact.\n- Would a stronger text decoder leads to better performance?\n- What is the reason of decreased performance of retrained QAGNet? Did the authors use different training details or different evaluation settings or else?\n- Intuitively, the proposed method could also improve the performance on image captioning task. I am wondering if salient object ranking could help with image captioning. It will be interesting to see results compared with SOTA image captioning methods.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The cyclical interaction uses caption to guide saliency object ranking.\n- Ablations shows the effectiveness of the SITA and CMQC in the proposed method.", "weaknesses": "- The segmentation head is unclear. The performance increase could potentially due to using a strong pretrained segmentation model.\n- The retained QAGNet has lower scores across metrics compared to the ones reported in the original paper. This is critical since the results of the proposed method does not outperform the reported results of QAGNet.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)