Decomposition of Concept-Level Rules in Visual Scenes
Authors: Fan Shi
Yuxuan Liang
Xiaolei Chen
Haiyang Yu
Xu Li
Yi Zheng
Rui Zhu
Xiangyang Xue
Bin Li
Venue: ICLR 2026 Poster (2026)
Keywords: Compositionality
Concept-Level Rules Decomposition
Large Vision-Language Models
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Human cognition is compositional, and one can parse a visual scene into independent concepts and the corresponding concept-changing rules. By contrast, many vision-language systems process images holistically, with limited support for explicit decomposition. Previous methods of decomposing concepts and rules often rely on hand-crafted inductive biases or human-designed priors. We introduce a Concept-Rule Decomposition (CRD) framework to decompose concept-level rules with Large Vision-Language Models (LVLMs), which explains visual input by leveraging LVLM-extracted concepts and the rules governing their variation. The proposed method operates in two stages: (1) a pretrained LVLM proposes visual concepts and concept values, which are employed to instantiate a space of concept rule functions that model concept changes and spatial distributions; (2) an iterative process to select a concise set of concepts that best account for the input according to the rule function. We evaluate CRD on an abstract visual reasoning benchmark, a spatial reasoning benchmark, and a real-world image caption dataset. Across both settings, our approach outperforms baseline models while improving interpretability by explicitly revealing underlying concepts and compositional rules, advancing explainable and generalizable visual reasoning.

Published as a conference paper at ICLR 2026
DECOMPOSITION OF CONCEPT-LEVEL RULES IN VISUAL SCENES
Fan Shi; Yuxuan Liang, Xiaolei Chen, Haiyang Yu, Xu Li, Yi Zheng, Rui Zhu, Xiangyang Xue, Bin Li Shanghai Key Laboratory of Intelligent Information Processing College of Computer Science and Artificial Intelligence, Fudan University
{fshi22 22, yxliang25, chenx1233 fudan. edu. cn hyyu200 fudan. edu.cn {xu_li23, zhengy23, rzhu24 ( J.m. duddanedd... cn {xyxue, libin] fudan. en
ABSTRACT
Human cognition is compositional, and one can parse a visual scene into indepen- dent concepts and the corresponding concept-changing rules. By contrast, many vision-language systems process images holistically, with limited support for ex- plicit decomposition. Previous methods of decomposing concepts and rules often rely on hand-crafted inductive biases or human-designed priors. We introduce a Concept-Rule Decomposition (CRD) framework decompose concept-level rules with Large Vision-LLanguage Models (LVL.Ms), which explains visual input by leveraging L.V.MM-xtracted concepts and the rules governing their variation. The proposed method operates in two stages: (1) pretrained LVLM proposes visual concepts and concept values, which are employed to instantiate space concept rule functions that model concept changes and spatial distributions; (2) iterative process to select a concise set of concepts that best account for the input according to the rule function. We evaluate CRD ing benchmark, a spatial reasoning benchmark, and a real-world image caption dataset. Across both settings, our approach outperforms baseline models while improving interpretability by explicitly revealing underlying concepts and com- positional rules, advancing explainable and generalizable visual reasoning.
abstract visual reason-
INTRODUCTION
The compositionality of human cognition enables us to understand complex scenes by isolating independent concepts and their changes (Lake al.) 2017). A visual concept (Lake al. 2015) is high-level, interpretable visual property (i.e., atta-attribute) that describes specific semantic categories or abstract traits present (e.g., concept Color describes attributes such as Blue, Red, or Green). A Rule specifies the allowable pattern of how concept's 's values vary across space (e.g.. The colors of rainbow are arranged from red to violet.) (Tenenbaum et al.) [201 Kemp Tenenbaum, 2008). Many visual scenes involve such decomposition into concepts and rules. for example, matrix reasoning problem (Raven & Court 1938) consists of human-defined meta-attributes and logical rules, while the entities in physical video move under physical laws (McCloskey 1983).
Early approaches decompose concepts and rules through hierarchical Bayesian inference, enabling the analysis of spatial arrangement rules in structured data (Kemp & Tenenbaum 2008 TTnenbaum
et al. 2011) as well as the compositional structure of handwriting characters (Lake et al., 2011
2015). More recent approaches aim disentangle the visual perception process from the high-level rule inference process by designing specialized modules (Zhang et al. 2021a), where structured generative priors, such as latent Gaussian processes (Shi et 2021) and algebraic reasoning back- ends (Zhng et al.) 2021b), have been incorporated to provide effective inductive biases for abstract visual reasoning tasks. Another line of work enforces concept-speciff latent functions capture independent factors of variation (Shi et al. 2023), which is evaluated on three different visual scenes.
Equual contribution Corresponding author

---

Published as a conference paper at ICLR 2026
Prior works often rely on strong inductive biases manually designed priors to extract interpretable structure. Although these inductive biases can lead to interpretable results, they also limit their adaptability to various visual scenes. Therefore, building frameworks that cover such compositional structure remains challenging. Motivated by these limitations, we lever- age Large Vision-Language Models (LVLMS)) as rich, data-driven priors for concept discovery and rule induction (Liu et al. 2023). LVLMs encode extensive world knowledge and fine-grained visual- linguistic correspondences (Wang al., [025b] Tong et al. 2024). and we could exploit these capac- ities to automatically perceive scene content, propose semantically meaningful concept candidates, and estimate their patch-wise values without manual attribute taxonomies or rule templates.
automatically dis-
In this paper, we propose a Concept-Rule Decomposition (CRD) framework that leverages pre- trained L.LM to decompose visual concepts and rules from visual inputs. First, CRD introduces a pre-trained LVLM to propose a Visual Concept Set (VCS) of the input, which contains a set of meta- attributes and the corresponding local attribute values. And we instantiate a Concept Rule Function (CRF) space to capture the change or spatial distribution of visual concepts. Second, given the CRF space grounded in the visual concepts, CRD performs an iterative sampling process to update the VCS that best accounts for the visual input. Through the sampling process, CRD converges set of visual concepts and associated rule functions that provide an interpretable explanation of the patterns observed in the visual data. To evaluate the proposed CRD, construct subset from VStar Bench (Wu & Xiel 2024), consisting of high-quality, open-domain real-world images paired with their corresponding meta-attributes.
The main contributions are summarized as follows:
We propose CRD, a novel and general framework to decompose concept-level rules from visual inputs based on the perception and reasoning ability of LVLMs.
We demonstrate that CRD can automatically extract and decompose concept-level rules from both natural and abstract images, improving the attribute extraction and spatial rea- soning abilities of various LVLM baselines.
We show that CRD enables LVLMs to surpass both traditional concept-rule decomposition methods and standard LVLM baselines on abstract visual reasoning tasks, highlighting its ability decompose the visual concepts and rules.
RELATED WORKS
Decomposition of Concepts and Rules. Earlier approaches analyze spatial arrangement rules on grid-structured image panels via Hierarchical Bayesian inference (Tenenbaum et al., 2011 Kemp & Tenenbaum 2008). In parallel, character modeling approaches decompose whole characters into strokes through predefined compositional processes and recombine them into new characters accord- ing to specific rules (Lake et al.) 2011 2015 Shi et al., 2025). Some methods leverage hand-crafted or learned feature representations address abstract visual reasoning tasks (Lovett & Forbus 2017 Little et al.) 2012 Lovett et al.] 2010). Recent works model concepts and rules more explicitly by combining latent encodings with probabilistic induction or adversarial learning (Pekar et al.) 2020
Zhang et al.) 2021a|b) by Shi et al., 2024 (026). while alternative approaches capture concepts and concept-changing rules through latent functions (Shi al.) [2021] (023). Nevertheless, these meth- ods still depend on auxiliary supervision or task-specific inductive biases, often involving human- injected knowledge like the specific form of rules.
Large Vision-Language Models. Large Vision-Language Models (LVLMss) have evolved signifi- cantly, starting with early models (Liu al., 2023 [i et al. 2023 Chen et al., 2025) that connected pre-trained visual encoders (c.g., CLIP-based ViTs (Radford et al. 2021]) to language models for open-ended visual question answering. These models paved the way for later LVLMs (Liu et al. 2024 Chen et al.] 2024b) Yin et al. [025]] Yu et al.] 2025 Liang et al.) 2023 2026), which im- proved input image resolution and enhanced vision-language alignment. Notably, InternVL-3.5 (Wang et al.) 2025a) and QL2. (Bai et al. 2025) exemplify the latest advancements, with both introducing new training strategies and frameworks to optimize reasoning performance. Sev- eral recent approaches have sought to leverage the capabilities of LVLMs to perform visual rea- soning (Tong et al.] 2024 Wang et al., [2025b] Chen et al. 2024a). These innovations have closed the performance gap with proprietary models such GPT-4o (Achiam et al. 2023), making open
2

---

Published as a conference paper at ICLR 2026
LVLMs competitive in rision-language tasks and offering impressive improvements in multimodal reasoning. Despite substantial progress, current LVLMs remain limited in visual rule extraction and abstract visual reasoning. Trained largely for pattern recognition and caption-style objectives, they receive little supervision for inferring compositional rules or conducting systematic relational rea- soning. Empirical analysis reports weak compositional understanding of concept-relation bindings (Anis et al. 2025) and persistent failures on abstract rule-induction tasks (Ahrabian et al. 2024)) This motivates our approach, which harnesses LVLM priors while introducing decomposition mechanism into interpretable concepts and rules, enabling more robust abstract visual reasoning.
METHOD
In this section, we describe how CRD decomposes visual inputs into visual concepts and rules. We first introduce the probabilistic definition of the visual concepts and rules in an overview, and then describe the two-stage process by which the model learns both components from raw data.
3.1 VISUAL CONCEPT SET
The set of candidate visual concepts contains a vast collection of possible concepts, often covering the majority of words in the vocabulary. As result, the concept space is highly diverse, making exhaustive enumeration impractical. Let [M] = f1....,M, .M} denote the universe of all M candidate visual concepts (we use their indices for convenience). Each element in [M] corresponds to prim- itive visual concept that could potentially be used to describe the input data. While [M] defines the full concept vocabulary, only very small portion of these concepts is related to the input.
Definition 1 (Visual Concept Set). Given the candidate concepts [M), Visual Concept Set (VCS) of size K is a subset G C [M] with cardinality |G] K.
According to Definition I CRD seleets a subset from to form the VCS, which contains K con- cepts that are actually relevant for explaining the given visual input X. For each visual concept i E [M]. we assume that p; E (0, 1) is the probability that is included in the VCS G, and the logit 0g log 1-p.
P._ The probability distribution of G given 0, is (see Appendix A for the proof)
PK (G (G ()
1
II iEG
where Z = Σ
II 3
(1)
Z
SC JES
|S|=
higher 0g indicates a higher prohability that concept i is included in G. Furthermore, 0 reflects the likelihood that the concept values are supported by specific rules, which establishes the connection between VCS and the rules. If the change pattern of a concept follows a certain rule, the concept latent factor that appropriately explains the visual input. In this case, its corresponding score 0, should be higher, resulting in a greater probability of the concept in G. In summary, Px (G 8) considers the underlying rules, biasing the selection toward concepts that exhibit clearer rules for structured decomposition of visual inputs. In the following section, we will introduce how CRD bridges the visual concepts and rules.
is more likely to serve
3.2 CONCEPT RULE FUNCTION
The rules in CRD capture the spatial distribution of concept values within an image. While concepts describe attributes such as color, the rules specify how these concept values are arranged and interact spatially. CRD learns rules by analyzing patch-wise concept values, capturing value changes and their spatial dependencies. As illustrated in Figure CRD first splits the input image X into N non-overlapping patches {=I,.... NNN ordered in a raster-scan manner from left to right and top to bottom. A LVLM is employed to generate set of concepts G,ule according to X. For each concept i E Grute- the LVLM further extracts concept values each patch. Let Pr. denote the spatial position of Fn, and Ug,n is the observed value of concept i on the n-th patch.
Definition 2 (Concept Rule Function). For given input image X, the values of concept extracted by the LVLM are U. = [Ui,I....,, Vi,N The position vector of patches is p = [P.....,PN]] The Concept Rule Function (CRF) of concept is a mapping f p + U,, The function space F of the
3

---

Published as a conference paper at ICLR 2026
(a) VCS Extraction
Batrect mrto-attrilotee from the image stogte Buglish words only pàruses compound no hyphenated words). Retone evorlyy one word per attritate..
Gom image entimate the the attribute (Golf Water,
Pathway, Humans) oppeurs gri
[mmage]
[Prompt]
(Prompt]
Large Vision- Langugge Model
Riolf Water, Pathway,, Humanj
Visual Concepts Set (VCS)
Crop
Innput Imagel
(b) CRF Learning Stage
VCS Sampling Stage
Visual Concept Set G m [Golf, Water, Tree, Pathnway, Human]
Extraction
Accept
Reject
Visual Concept Set
LVLM based Concept Proposal
yes
=17
Likelihood
80
with Deep Kernel
&
Visual Concept 6'=fttolf, Sun, Humon]
Beroulli Samling
GP with Deep Kernel $
Concept Rule Functions (CRFs)
#(G, G')
Acceptance Rate Computation
Likelihood
Overview of the proposed CRD framework. The process consists of two stages. In the CRF learning stage, visual concepts are extracted from images and Gaussian process prior is used to construct the function space over CRFs. In the VCS sampling stage, iterative sampling procedure is applied generate concept subsets from the distribution px (G 8).
Figure 1:
mapping is Gaussian Process (GP) Williams & Rasmussen 1995) with deep kernel:
f ~ (P( k)(:))), where ko (pi.P)) = exp (-2 (190(p))) so(p))li). i
1
and g6 is a neural network that maps imput positions into high-dimensional representations.
The marginal likelihood of the concept values is Gaussian p(v; P. Φ) = (N 0, K2) under the GP prior, where Kg RNN is the kernel matrix with entries (Ko).z ks P,). The logarithmic marginal likelihood is
N
LIMI. (p. v.) -. K v, log del (Ks)
log(2
(2)
Intuitively, a CRF characterizes how visual concepts vary across spatial positions of an image. Since the concepts with specific patterns considered more related to the input, CRD defines 0; through a CRF and the function space F, i.e., 0; LLML (p, v.).
3.3 LVLM-BASER TWO-STAGE LEARNING PROCESS
Based Definitions 1 and 2
the main challenges that CRD needs address twofold:

---

Published as a conference paper at ICLR 2026
1. how construct the function space F to fit the rules on input images;
2. how efficiently sample G from the complex probability distribution pk (G ().
To tackle both challenges, we propose a LVLM- learning process shown in Figure In the CRF learning stage, a LVLM is taken to extract visual concepts and learn an appropriate function space F over CRFs. In the second VCS sampling stage, according to the learned function space, we employ sampling procedure like Netropolis-Hastingg to generate G from px PG | [). In the following sections, we will provide detailed description of this two-stage process.
3.3.1 CRF LEARNING STAGE
Given a batch of input images, we construct the patches and extract the patch positions and the concept values using a LVLM, forming the training set for the corresponding CRFs. The training set is denoted as D {Prt vx) )))),= where p; denotes the spatial positions of patches and vs is the predicted concept values. To train the CRF, we minimize the negative logarithmic marginal likelihood defined in Equation the observed concept values. The parameters & are optimized by gradient-based methods. We compute the gradient of the negative logarithmic marginal likelihood with respect to d. This gradient is then used in standard gradient descent or adaptive optimizers (e.g., Adam (Kingma & Ba 2014)) to update ø. By iteratively processing batches of images, computing the marginal likelihood, and performing gradient-based updates, the deep kernel parameters are learned such that the GP prior over CRFs captures the underlying rules of visual concepts across spatial positions.
3.3.2 VCS SAMPLING STAGE
Once the distribution px (G' () and the function space F have been defined, the second stage of framework aims to obtain samples of G that are consistent with both the probabilistic distribu- tion. Direct sampling from Px(GG 0) is computationally intractable due to the combinatorial size of the candidate concepts. To address this, we design Sectrpolis-Hastinn (MH) sampling proce- dure (Chib & Greenberg, 1995), denoted as LVLM-MH, which leverages the proposal distribution informed by LVLM.
Starting from a current VCS G, we propose new VCS G' = G' (i) by replacing the concept i G with the candidate concept j E [M]] G. The transition probability of the concept replacement process is decomposed as
= = G) G,G),
where r(1 G) denotes the probability of selecting concept i of G for replacement, and q( G) denotes the proposal probability of selecting the candidate concept its replacement. We consider the following design choices for r(i G)- First, select i E G uniformly at random, i.e., r(i G) 1/IG). Second, weight the selection by the inverse importance of concepts, i.e., r(i G) x - such that less related concepts are more likely to be replaced. CRD determines the replaced concept G uniformly random, which avoids dependence on the score assigned to each concept. This choice removes the necessity of evaluating 0 over the entire set G and brings a computational advantage. At the same time, uniform sampling guarantees an exploration of the concepts, ensuring that every candidate has equal opportunity to be selected.
by selecting
After determining the replaced concept i, the distribution q i Gi instantiated by the LVLM to propose a target concept from the candidate concepts [M] G. The distribution leverages the semantic prior captured by the I.V.M, thereby assigning higher probability to concepts that are semantically visually more consistent with the input image. This mechanism ensures that the sampling process is guided by high-level semantic knowledge, facilitating the discovery of more meaningful candidate concepts. To avoid degenerate cases where the LVLM assigns an extremely small probability to certain concepts, we impose a constraint on the logits before normalization. The output logits produced by the LVLM are clipped to ensure that concept receives probability arbitrarily close to zero
With the transition probability Q(G, G'), the acceptance probability of L.LLM-MH is computed by
5

---

Published as a conference paper at ICLR 2026
Pk () r(j ()
aGG,"" min 1,
min
Pk (0)
Pk
r(i
G(
(4)
q(i j
min 1,
Ui G))
Bernoulli variable is sampled with the acceptance rate (: G'). The proposal G - G' will be executed if the sampled result is otherwise, the current VCS is retained. For an input image, we run multiple iterations of the LVLM-MH sampler. At each iteration, a candidate concept is proposed and accepted or rejected with probability ((G, G*). Over repeated iterations, the VCS converges to the target distribution PK (G' 8), thus providing samples of concept sets consistent with the complex probability distribution of VCS.
EXPERIMENTS
4.1 BASELINES
To assess robustness and model-agnostic generality, we benchmark CRD against a diverse panel of large vision-langen -anooggg models that span families and parameter scales. Specifically, we include Intern 3.5 in 4B/8B ( (Wang et al. 2025a), Qwen2.5-VL. in 3B/7B (Bai et al. 2025) and DeepSeek- VI.2-Tiny (Wu et al. 2024p. We use publicly released checkpoints and official inference pipelines without any additional fine-tuning, To ensure comparability,, we harmonize evaluation along three axes: (I) prompting standardized instruction templates for all baselines (detailed in Appendix [ (2) decoding deterministic generation with temperature O; and (3) adopting each model's 's native im- age preprocessor (including any built-in in high-resolutin tiling or sub-image partition mechanisms). We also compare method with six approaches based deep learning in abstract visual reason- ing tasks: PrAE (Zhang et al. 2021a). I.PP (Shi et al. 2021). and CLAP-NP (Shi et al.) 2023),
LEN (Zheng et al.] 2019), ReTNet+DRT (Zhang al. 2019), and SRAN (Hu et al. (021). These traditional deep learning methods are not gencral-purpose models in the sense of LVLMs. They
are typically designed for single dataset a limited set of tasks, with architectures and train- ing objectives specifically designed to capture dataset-dependent rules and concepts. In contrast, LVLMs provide a universal modeling paradigm, pretrained on large-scale multimodal corpora, and are capable of zero-shot or few-shot reasoning across diverse tasks.
4.2 DATASETS
Meta-Attribute Extraction. We derived VSB-MA by selecting all high-quality images from VS- tar Bench (Wu & Xie, 2024), aiming to establish standardized set of meta-attributes for general visual scenes. Each image was first processed with a GPT-4o (Achiam et al., (023) model using carefully engineered prompt (detailed in the appendix) to automatically extract an initial pool of descriptive attributes. These raw attributes were then rigorously reviewed and cleaned by human annotators to ensure conceptual uniqueness of each attribute and clear inter-attribute distinctiveness. The cleaning process involved removing redundant or ambiguous descriptors, merging semantically overlapping terms under unified labels, and verifying consistency across similar scenes. As a result, we developed a curated and standardized meta-attribute set for each image, which serves as a high- quality reference for assessing the ability of models to extract visual features and rules in complex real-world scenarios.
Abstract Visual Reasoning. We evaluate our framework on RAVEN (Zhang et al., 2019) and I- RAVEN (Hu et al. (021), two widely used datasets for abstract visual reasoning. Both datasets consist of seven image configurations with diverse layouts (e.g., single-object, inside-outside, and grid-based), where images are governed by compositional rules over attributes such as shape, size, and position. In addition, both datasets introduce noisy attributes (c.g., random rotations, colors, and object positions in grids), which increase the difficulty of learning concept-rule composition only from the raw data. In this task, we fill each candidate option into the question matrix and select the one with the highest CRD rule likelihood as the final answer.
Spa Benchmark. We evaluate our framework on SpatialEval, a benchmark designed to probe spatial reasoning in LVLMss. SpatialEval spans four complementary tasks that cover core

---

Published as a conference paper at ICLR 2026
Table 1: Meta-attribute extraction performance on VSB-MA. We compare the standard LVLM
baselines with the corresponding CRD instantiations.
Method
Avg. Sim.
Precision
Recall
F1
AUPRC
ROC-AUC
DeepSeek-VL22 VL2-Tiny
16.8
21.1
27.4
36.3
50.8
DeepSeek-VL24 + CRD
20.4
44.8
23.2
30.6
40.1
58.3
Qwen2.5-VL-33 -LL-3B
31.5
77.1
26.9
39.9
42.5
65.8
Qwen2.5-VL-33 CRD
36.7
77.3
32.8
46.1
47.9
68.1
Qwen2.5-VI--7B
46.9
73.7
38.0
50.2
54.1
74,6
Qwen2.5-VI-7BB CRD
51.6
76.3
44.4
56.1
58.0
75.7
IntemVL-33.54B
38.5
75.1
35.3
48.0
48.8
68.7
IntemVL--3.5-4P CRD
44.5
76.4
42.7
54.8
52.4
70.1
IntemVL-33.5-8B
75.7
51.2
61.1
65.2
83.9
IntemVL-3.5-8B + CRD
64.0
77.4
55.6
64.7
68.3
84.8
Human
77.4
84.7
74.6
79.3
79.0
87.7
facets spatial intelligence, including spatial relationship understanding on maps, goal-directed navigation in mazes, position querying and counting in structured grids, and spatial reasoning grounded in real images with long dense captions. A key feature of the benchmark is that each problem provides both an image and textual description that are individually sufficient to answer the question, enabling controlled evaluation under text-only, vision-only, and vision+text settings, and making possible to analyze how models use ignore visual evidence when rich textual cues are available. In our evaluation, CRD concatenates the extracted meta-attrin aetribatti and patch scores as an additional prompt prefix to the original question and then obtains the model's 's response.
4.3 META-ATTRIBUTE EXTRACTION
Metrics. Let A and A denote the sets of predicted attributes and ground-truth attributes, respectively. For any pair (i.j) E Á A, the cosine similarity score s(i,j) [-1, 1] is computed from the sentence-transformee (Reimers & Gurevych] (019) embeddings of the two attributes. We obtain a one-to-one alignment matrix M C A x (e.g., through the Hungarian algorithm (Japrapto 2010)) that maximizes total similarity. Average Similarity is the mean similarity over aligned pairs:
AvgSim
Σ s(i.j).
(5)
M
(K))MM
providing graded measure of alignment quality across the matched concepts. We report Precision, Recall, F1, AUPRC, and ROC-AUC derived from the similarity scores, providing comprehensive evaluation of the attribute extraction performance in terms of accuracy, coverage, and separability.
Quantitative Results. As shown in Table [ CRD consistently boosts meta-attribut extraction per- formance for every model and scale evaluated. For each architecture, method outperforms its base counterpart in all metrics. For example, the VLter.VL-35555 rises from 38.5 44.5 and its Fl score from 48.0 54.8 after using CRD. We observe simi- lar improvements for the larger VL-3-5--33.588 and Sowen2.5-VL-1 when optimized with CRD. Even smaller models benefit: Owen2.5-VL-3B and DeepSeek-VL2-Ting both show notable gains in all metrics after using our method. We also include the performance of human experts on the veta-attribute extraction task as a reference. Collectively, these results suggest that CRD enables the models to more fully tap into their vast pre-trained knowledge, substantially strengthening their ability to interpret complex visual rules and to extract the relevant attributes with higher fidelity. More experimental results are provided in Appendix D
Average Similarity (AvgSim)
Ablation Study. Table 2 indicates the impact of removing each component from the acceptance probability a (G, G") in our VCS sampling stage. Removing any single component of the acceptance probability causes a notable drop in performance. This consistent decline confirms the effectiveness of framework. The rule-based CRF score (the term cd;- derived from the Concept Rule Function) effectively guides the update of the concept set, while the LVLM proposal ratio (based on the LVLM's proposal probability) helps the large model thoroughly explore the concept space.
7

---

Published as a conference paper at ICLR 2026
Table 2: Ablation study on acceptance probability components in VCS sampling stage. We analyze the impact of different components in the acceptance probability a(G, G') using InternVL- 3.5-8B as the baseline model on VSB-MA dataset.
Method
Avg. Sim.
Precision
Recall
F1
AUPRC
ROC-AUC
Intern VI.-3.5-8B + CRD
64.0
77.4
55.6
64.7
68.3
85.0
w/o LVLM Proposal Ratio
61.0
77.1
51.3
61.6
65.8
84.2
w/o CRF Score Term
59.2
74.6
49.8
57.4
64.8
84.1
InternVi VL-3.5-8B
59.9
75.7
51.2
61.1
65.2
84.5
CRD dedign > 'plantt?
Judger Agroct
CRD house' bbilding?
Judger
Meta Attribute?,
Final Meta Attribute
design house sky clay
plant hause sky
plant house sky
grey
plant shadow building road sky facilities
LVLMs
road
road
clay
clay
CRD Wky tolee?
Juciger Disagrect
X
Figure 2: Qualitative case studies of meta-attribute extraction on the VSB-MA dataset.
Together, these components drive significant performance gains over the baseline. Furthermore, we compare the two ablated variants in detail. Removing the LVLM proposal ratio results performance decline (e.g., Avg Sim drops from 64.0 61.0), remaining above the baseline (59.9). strong contributor to performance. The LVLM
This suggests that the CRF score term alone proposal ratio provides additional benefit by allowing some proposals with lower CRF scores to be accepted. This encourages a broader exploration of the concept space and yields a higher overall performance. In contrast, removing the CRF baseline on most metrics. Without the CRF score to steer the sampling toward concept sets, unguided exploration can degrade the results. Appendix Prpendix[]provve provides more analyses of our framework.
causes
drop, and even below the
Qualitative Results. As shown in Figure 2 given natural image, the pretrained LVLM first pro- poses an initial pool of candidate meta- attriiutts. CRD then performs an iterative propose-judge update loop. In each iteration, CRD suggests refinement or replacement aimed at increasing se- mantic abstraction and rule consistency (e.g., replacing the vague design with the concept plant, or lifting the instance-level house to the category-level building). Proposals inconsistent with the meta-attribute definition (e.g., sky - blue, which collapses to a specifie color instance) are rejected. Repeating this process converges to concise, interpretable meta-attriuute set that better aligns with the scene's underlying organization. More qualitative results can be found in Appendix
4.4 ABSTRACT VISUAL REASONING
In the abstract visual reasoning tasks, we compare our method with deep-learning methods, which require -ptaset--peeific hyperparameter tuning (e.g., represetation dimensions and number of rules) to adapt to different data. We instantiate CRD with Intern Qwen2.5-VL, and DeepSeck- VL2, and compare the resulting models against their original baselines well as other representative LVLMs. In this setting, the context panel and all candidate images are concatenated into a single problem image, where the candidates are annotated from A to H. The complete prompt construction procedure is described in Appendix C To solve abstract visual reasoning tasks with LVLM-CRD, the right-bottom panel the problem matrix is iteratively replaced with each candidate image. For each replacement, the corresponding CRF score is computed to evaluate how well the completed matrix conforms to the learned rules. Finally, the candidate with the highest CRF score among all eight options is selected the predicted answer for the model evaluation.
8

---

Published as a conference paper at ICLR 2026
Table 3: The performance on the abstract visual reasoning task. We show the accuracy (%) of selecting answers on subsets of RAVENI-RAVEN, Both Qwen2.5-VL-33 and Qwen2.5-VL-77 achieve much higher accuracy on RAVEN but drops near-random accuracy on I-RAVEN, despite the two datasets sharing the same context panels and differing only in the candidate set, raising the concern of data contamination. InternVL-CRDD indicates the CRD instantiated with Intern 3.5-8B, and Qwen-VL-CRD indicates the CRD instantiated with Qwen2.5-VL-77. We use the 7B checkpoint of LLaVA-NeXT and the tiny version of DeepSeck-Vie VL2.
RAVEN LRAVEN
Modcls
Center
L-R
U-D
O-IC
O4G
2Grid
3Grid
Average
PrAE
14.5/22.6
7.1/21.2
11.1/26.5
7.1/16.9
9.5/24.4
13.1/214
11.1/18.9
10.5/21.7
LGPP
9.2/20.1
4.7/18.9
5.2/21.2
4.0/13.9
3.1/12.3
8.6/13.7
10.4/13.9
6.5/16.3
CLAP-NP
30.4/42.9
13.4/35.1
12.2/32.1
16.4/37.5
9.5/26.0
16.0/20.1
24.3/35.8
17.5/32.8
ResNet+DRT
14.1/13.2
11.9/134
12.8/12.1
13.6/12.1
13.1/13.3
16.8/12.4
16.1/12.8
14.1/12.8
SRAN
75.8/89.6
31.0/67.6
33,2/70,9
39.3/75.7
68.0/52.2
66.9/38.6
79.3/32.2
56,2/61.0
LEN
69.3/15.3
74.5/14.6
74.2/15.5
72.8/12.8
77.6/15.7
65.0/15.1
73.5/16.1
72.4/15.0
InterVLL322B
9.0/9.0
17.0/13.0
14.0/16.0
13.0/10.0
10.0/13.0
13.0/9.0
10.0/12.0
12.3/11.7
+Instruct
13.0/800
15.0/11.0
15.0/18.0
14.0/11.0
7.0/13.0
13.0/12.0
14.0/17.0
13.0/12.9
IntemVL3--8
16.0/15.0
6.0/14.0
12.0/13.0
11.0/13.0
17.0/14.0
13.0/110
6.0/17.0
11.6/13,9
+Instruct
12.0/16.0
12.0/10.0
11.0/11.0
12.0/11.0
16.0/15.0
10.0/10.0
8.0/18.0
I 11.6/13.0
GPT-4o
16.0/13.0
13.0/13.0
10.0/6.0
13.0/110
10.0/17.0
8,0/13.0
11.0/12.0
1 11.6/1211
LLaVA-NeXT
14.0/14.0
13.0/13.0
20.0/17.0
10.0/9.0
10.0/12.0
9.0/15.0
13.0/12.0
12.7/13.1
DeepSeck-VLL VL2
18.0/77.0
22.0/15.0
11.0/110
21.07.00
20.0/15.0
17.0/11.0
10.0/11.0
17.0/12.4
Qwen2.5-VI..3B
68.0/17.0
$4.0/17.0
33.0/11.0
71.011.0
54.0/15.0
$5.0/5.0
44.0/8.0
14.1/
Qwen2.5-VL-77
78.0/19.0
53.0/18.0
55.0/14.0
73.0/18.0
65.0/12.0
$2.0/9.0
42.0/15.0
59.7/15.0
IntemVL-CRD
22.0/24.0
19.0/29.0
31.0/37.0
33.0/44.0
47,0/49.0
33.0/20.0
36.0/32.0
31.6/33.6
Qwen-VT-CCRD
7..0/8.0
97.0/93.0
95.0/95.0
87.0/89.0
98.0/98.0
84.0/80.0
88.0/89.0
89.4/89.3
Quantitative Results. Table 3 reports the experimental results on the classical abstract visual rea- soning datasets RAVEN and I-RAVEN, where the models are evaluated by the accuracy of selecting the correct answer from eight candidates. The deep learning-base baselines achieve higher aceu- racy than most LVLMs, while LVLMs exhibit limited performance on both RAVEN and I-RAVEN. Open-source models such as InternVL3 and LLaVA-Next, as well as the closed-source GPT-4o, achieve accuracies close random guessing (12.5%). An exception is Qwen2.5-VI. which reaches over 60% accuracy on RAVEN, substantially outperforming other LVL.ss. Notably, when applied to Qwen, CRD already achieves performance that surpasses several of these task-specific mod- els, especially on I-RAVEN, Recent work (Jiang et al. 2025) has also observed similar anomalies. Qwen2.5-VL performs well on the final answer selection task, which the original RAVEN task, but struggles on simpler intermediate reasoning subtasks, leading the authors to suspect potential data contamination. We observe similar discrepancy in our experiments. The proposed method mostly outperforms both deep learning-based baselines and LVLMS on these benchmarks, demonstrating its advantage in abstract reasoning. We hypothesize that the poor reasoning performance of L.V. stems from their lack of explicit problem decomposition ability. Unlike CRD, which transforms the problems into structured concept-rule learning process, LVLMs tend to rely on holistic pattern matching, making difficult to capture the underlying abstract rules behind problems. Appendix provides additional qualitative results and analysis.
4.5 SPATIALEVAL BENCHMARK
Table reports the results on SpatialEval Benchmark (Wang al. 2024) to evaluate four spatial reasoning abilities: MpatialMap, MazeNav, SpatialGrid, and SpatialReal. We compare the original LVLMs with two variants of our method. CRD-meta uses only the extracted meta-attributes as additional prompts. While CRD-full further incorporates patch-level scores. Across both Qwen2.5- VL-3B and 0wen2.5-VL-75. CRD-meta already provides clear gains the base models, and CRD-full yields further improvements, particularly in SpatialReal and MazeNav. These results show that both types of CR Cer-vveeeed signals contribute meaningful guidance spatial reasoning tasks.

---

Published as a conference paper at ICLR 2026
Table 4: Performance on SpatialEval Benchmark. SpatialEval includes four spatial reasoning tasks (Spa MazeNav, SpatialGrid, SpatialReal). CRD-meta denotes that only the meta- attributes extracted by CRD are used as prompts to guide the LVLM's responses. CRD-full further incorporates both the meta-attributes and the patch scores additional prompts for the LVLMs.
Model
SpatialMap
MazeNav
SpatialGrid
SpatialReal
Overall
Sp.
48.87
22.60
87.03
34.81
50.27
Qwen2.5-VI.
50.93
27.40
83.33
91.85
54.00
Qwen2.55-V1:3B--RRDD-
50.93
29.07
85.20
96.30
56.27
Qwen2.5-VV-33+<CRD-I
51.67
29.00
86.40
96.30
56.87
Qwen2.5-VI-78
63.00
28.93
85.60
91.11
60.11
Qwen225-VL-77BCCRD- Qwen2.5-VV--BB-CRD--i
64.20
29.13
87.93
95.56
61.45
65.26
33.33
88.46
97.04
63.37
Table 5: Efficiency analysis the VSB-MA benchmark. We report the average latency per token (ms), computational cost (TFLOPs), GPU memory usage (GiB), KV-Cache size (MB), and input token count for different models with and without CRD. We also analyze the impact of patch granularity on computational efficiency by varying the number of patches (2x2, 3x3, 4x4) for LntemV--3.5-8D on the VSB-MA benchmark.
Model
Lalency Per Token
TFLOPs
GPU Memory
KV-Cache
Input Token
Intern VL-3.5-8B
77.3
39.02
17.02
1287.0
2288
+CRD-2x2
234.3
42.93
17.46
1405.7
2499
+CRD-3x3
236.1
43.23
17.67
1414.7
2515
+CRD-4x4
239.5
43.71
17.95
1429.3
2541
Qwen2.5-VL--7B
39.7
6,47
16.49
171.1
447
+CRD-2x2
190.1
8.91
17.32
234.7
613
DeepSeek-VL.22TTHg +CRD-2x2
59.6
1.06
11.82
114.8
1224
264.1
1.16
12.53
124.9
1332
4.6 EFFICIENCY ANALYSIS
shown Table we summarize the inference efficiency of different models with and without CRD. The Ga-based CRF and sampling procedure do introduce additional computation compared to direct LVLM inference. However, this extra cost comes with more accurate concept-rule decompo- sition, which in turn leads significantly improved performance the evaluated reasoning tasks. In our setting, the number patches per image is kept small. The default CRD configuration uses 2 x 2 spatial grid (i.e., N 4 patches), the theoretical O(N3) cost of GP inference remains negligible in practice. We provide a complexitylattenc table reporting latency and memory usage under different patch counts, showing that runtime grows modestly and remains practical within the patch range we consider. Scalable GP variants such as SKI (Wilson & Nickisch] (2015) are mainly beneficial when N is very large (e.g., thousands of patches); since our method operates low-N regime by design, approximate inference is not considered here.
CONCLUSION
We presented CRD, model-agnostic framework that leverages pretrained LVLMs to propose vi- sual concepts, which employed learn rule functions that capture how these concepts vary and organize spatially. Followed by an iterative sampling process, CRD selects a proper set of visual concepts and concept-level rules for the input. Across natural-image and abstract visual reasoning evaluations, CRD improves performance over LVLM baselines, demonstrating its ability to decom- pose concepts and rules. By minimizing hand-crafted inductive biases and harnessing data-driven priors, CRD offers general route explainable, compositional visual understanding. In future work, we plan to extend the framework to temporal and broader settings and to explore richer rule spaces and sampling strategies.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (No.62176060) and the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303. 08774, 2023.
Kian Ahrabian, Zhivar Sourati, Kexuan Sun, Jiarui Zhang, Yifan Jiang, Fred Morstatter, and Jay Pujara. The curious case of nonverbal abstract reasoning with multi-modal large language models. arXiv preprint arXiv:2401. 12117, 2024.
Ahmad Mustafa Anis, Hasnain Ali, and Saquib Sarfraz. On the limitations of vision-language models in understanding image transforms. arXiv preprint MxXi:22503.098377 2025.
Shuai Bai, Keqin Chen. Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvim: Endowing vision- -anguaee models with spatial reasoning capabilitiess Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 14455-14465, 2024a.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. In ECCV, pp. 370- 387,2025 2025.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic -ingul-iin tasks. In CVPR, pp. 24185-24198, 2024b.
Siddhartha Chib and Edward Greenberg. Understanding the hutroplis-hhstting algorithm. The american statistician, 49(4):327-335, 1995.
Sheng Hu, Yuqing Ma, Xianglong Liu, Yanlu Wei, and Shihao Bai. Stratified rule-aware network for abstract visual reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1567-1574, 2021.
Benjamin Agus Japrapto. Hungarian algorithm. 2010.
Yanbei Jiang, Yihao Ding, Chao Lei, Jiayang Ao, Jey Han Lau, and Krista A Ehinger. Be- yond perception: Evaluating abstract visual reasoning through multi-stage task. arXiv preprint arXiv:2505.218800 2025.
Charles Kemp and Joshua B Tenenbaum. The discovery of structural form. Proceedings of the National Academy of Sciences, (56611):0687-1069 2008
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 141266441..6900 2014.
Brenden Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua Tenenbaum. One shot learning of simple visual concepts. Proceedings of the annual meeting of the cognitive science society, volume 33, 2011.
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 330062666::333--1338 1332-1338, 2015.
Brenden M Lake, Tomer D Ullman, Joshua Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253, 2017.
11

---

Published as a conference paper at ICLR 2026
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-imad image pre-training with frozen image encoders and large language models. In International conference machine learning. pp. 19732.19742. PMLR, 2023.
Yuxuan Liang, Xu Li, Xiaolei Chen, Haotian Chen, Yi Zheng, Chenghang Lai, Bin Li, and Xi- angyang Xue. Global semantic-guided sub-image feature weight allocation in high-resolution large vision-language models, 2025. URL CtLhstts: www.v.org/abs......ooooooa/ abs///lll//21/ 14276
Yuxuan Liang, Xu Li, Xiaolei Chen, Haotian Chen, Yi Zhen, Zhe Liu, Rui Zhu, Bin Li, and Xi- angyang Xue. Pyramid token pruning for high-resolution large rision-language models via re- gion, token, and cnaruuctionguuddee importance. IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2026. doi: 10. 10.11990CSV222226336666
Daniel R Little, Stephan Lewandowsky, and Thomas L Griffiths. A bayesian model of rule induction in raven's progressive matrices. In Proceedings of the Anmual Meeting of the Cognitive Science Society, volume 34, 2012.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NeurIPS, pp. 34892-34916, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, 2024. URL https:
/ //1lava-vl.ght wiw.ll....t........... iamhub..o/// 1o/bo///// 20111//244-01----11 1011--00-laaaeet.
Andrew Lovett and Kenneth Forbus. Modeling visual problem solving as analogical reasoning. Psychological review, 124(1):60, 2017.
Andrew Lovett, Kenneth Forbus, and Jeffrey Usher. A structure-mapping model of raven's 's progres- sive matrices. In Proceedings of the Annual Meeting of the Cognitive Science Society, 2010.
Michael McCloskey. Intuitive physics. Scientific american, 222-41 1983.
Niv Pekar, Yaniv Benny, and Lior Wolf. Generating correct answers for progressive matrices intel- ligence tests. arXiv preprint trXiv:2011.00490 2020.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. pp. 8748-8763, 2021.
John C Raven and JH Court. Raven's progressive matrices. Western Psychological Services Los Angeles, CA, 1938.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https: /
org/ abs///////// /19///90.... 10084
Fan Shi, Bin Li, and Xiangyang Xue. Raven's ' progressive matrices completion with latent gaussian process priors. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pp- 9612-9620, 2021.
Fan Shi, Bin Li, and Xiangyang Xue. Compositional law parsing with latent random functions. In The Eleventh International Conference on Learning Representations, 2023.
Fan Shi. Bin Li, and Xiangyang Xue. Towards generative abstract reasoning: Completing raven's progressive matrix via rule abstraction and selection. In The Twelfth International Conference on Learning Representations, 2024.
Fan Shi, Haiyang Yu, Bin Li, and Xiangyang Xue. Cola: Chinese character decomposition with compositional latent components. arXiv preprint arXiv:2506.03798. 2025.
Fan Shi. Bin Li, and Xiangyang Xue. Abstracting concept-changing rules for solving raven's 's pro- gressive matrix problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp-
1-12, 2026. doi: 10.1 2..109//PAMM..202..66555
12

---

Published as a conference paper at ICLR 2026
Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah D Goodman. How to grow mind: Statistics, structure, and abstraction. science, (316602))1279-12855 2011.
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal Ilms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9568-9578, 2024.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open-souree multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508. 2508. 2025a.
Yikun Wang, Siyin Wang, Qinyuan Cheng, Zhaoye Fei, Liang Ding, Qipeng Guo, Dacheng Tao, and Xipeng Qiu. Visuothink: Empowering Ivlm reasoning with multimodal tree search. arXiv preprint ar Xix::2500..00 00:::2004.9930. 2025b.
Christopher Williams and Carl Rasmussen. Gaussian processes for regression. Advances in neural information processing systems, 8, 1995.
Andrew Wilson and Hannes Nickisch. Kernel interpolation for scalable structured gaussian pro- cesses (kiss-gp). In International conference on machine learning, pp. 1775-1784. PMLR, 2015.
Penghao Wu and Saining Xie. V?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 13084-13094, 2024.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma. Chengyue Wu, Bingxuan Wang, al. Deepseek-v12: models for advanced multimodal understanding. arXiv preprint arXiv:2412.10300 2024.
lisionllanguage
Weijie Yin, Yongjie Ye. Fangxun Shu, Yue Liao, Zijian Kang, Hongyuan Dong, Haiyang Yu, Dingkang Yang. Jiacong Wang, Han Wang, et al. Sail-v12 technical report. arXiv preprint 250.::2509.44333 2025.
Haiyang Yu, Mengyang Zhao, Jinghui Lu, Ke Niu, Yanjie Wang, Weijie Yin, Weitao Jia, Teng Fu, Yang Liu, Jun Liu, et al. Eve: Towards end-to-end video subtitle extraction with vision-language models. arXiv preprint arXiv:2503.0400
2025.
Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chur Zhu. Raven: dataset for relational and analogical visual reasoning. In Proceedings of the IEEE//VF Conference Computer Vision and Puttern Recognition, pp. 5317-5327, 2019.
Chi Zhang, Baoxiong Jia, Song-Chin Zhu, and Yixin Zhu, Abstract spatial-temporal reasoning via probabilistic abduction and execution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9736-9746, 2021a.
Chi Zhang, Sirui Xie, Baoxiong Jia, Ying Nian Wu, Song-Chun Zhu, and Yixin Zhu. Learn- ing algebraic representation for systematic generalization in abstract reasoning. arXiv preprint arXiv:2111. 2021b.
Kecheng Zheng, Zheng-Jun Zha, and Wei Wei. Abstract reasoning with distracting features. Ad- vances Neural Information Processing Systems, 32, 2019.
13

---

Published as a conference paper at ICLR 2026
A PROOFS
A.1 PROOFS OF EQUATION
We consider the probability distribution of visual concept set G, where each concept i [M] is included in G with probability ps (0, 1). Denote |G| K and define 0; = log 1-p;" The goal is to derive
Pi
Pk (G (0)
1
II TTe,
where Z
Σ II
Z
iEG
SCMM
s||=
Proof.
Without the constraint [G] K, the probability of G is product of independent Bernoulli
distributions:
II P:
II Pa).
iEG
k&G
Imposing the constraint |G| K, the conditional probability is
IL Ps [e pi)
Pk(G) Esquiss 1l Pj 11 -p); p2)
Factor out the term C(p) = II[ M (1 Pr):
Σ II Pj
[I p2) = C(p)
Σ
Pj C(p) Σ
II
e
1 1 Pj
SC[M jES
1$8
SCIM]
jES
SCIM jES -
|
|8|=K
II
II p.) C(p)
II
C(p) II
P
SEG
iEG
iEG
IEG
Then, obtain
px ( 10)= - E Ils
B DATASETS
B.1 ABSTRACT VISUAL REASONING
We evaluate the methods two commonly used abstract visual reasoning datasets: RAVEN (Zhang ct al. 2019) and I-RAVEN (Hu et al. 202T). Both datasets contain seven distinct image configu- rations, illustrated in Figure 3 The Center configuration contains single central object, while L-R and U-D consist of two objects arranged horizontally or vertically. O-IC and O-IG adopt inside- outside layouts, and 2Grid and 3Grid contain 2x2 and 3x3 object grids, respectively. While most configurations involve rules applied to single component, O-IG and grid-based configurations ad- ditionally introduce rules defined over object grids.
Each configuration involves abstract rules that govern visual attributes, including four main cate- gories: Constant, Progress, Arithmetic, and Distribution Three
1. Constant: the attribute keeps unchanged in rows;
2. Progress: the attribute increases or decreases with the same stride in rows;
3. Arithmetic: the attribute the third image is computed from the attributes of the first two
images via specific arithmetic operations (e.g., addition and subtraction operations);
4. Distribution Three: the attributes in rows are three fixed values in different orders.
In addition, both datasets contain noise attributes, which are randomly sampled from the feasible set (e.g., object rotation in non-grid settings, rotation, color, and grid position in grid-based settings).
14

---

Published as a conference paper at ICLR 2026
Center
L-R
UDD
O-C
Candidates
Candidates
Candidates
Candidates
O-IG
2Grid
3Grid
Candidates
Candidanes
Candidates
Figure 3: Visualization of the RAVEN dataset. The seven configurations on RAVEN/I-RAVEN. The images with red borders are correct answers that fix the rules defined in the panels.
beach, velicle, person, dress, umbeelln, road,
building, street, people, vehicle brand, shediwy, faciltiies
faiities, plant buildin, sky, roed, character
valicde
vehide, skg bridge, plant, eign
paper,
Figure 4: VSB-MA dataset examples. Sample images from the VSB-MA dataset derived from VStar Bench, showing diverse visual scenes with their corresponding meta-attributes for concept- rule decomposition evaluation.
B.2 META-ATTRIBUTE EXTRACTION
As shown in Figure 4 VSB-MA is a full set of all high-quality natural images sampled from VS- tar Bench, created provide a standardized reference for meta-attribute-basee concept-level rules evaluation. For each image, we first elicit an initial attribute pool with GPT-4V using a tightly engineered prompt (specification included below). We then conduct rigorous human curation to enforce conceptual uniqueness and inter-attribute separability: redundant or ambiguous descriptors are removed, semantically overlapping items are merged under unified label, and cross-scene con- sistency is verified. Meta-attributes are defined high-level, interpretable properties (e.g., vehicle, building, furniture, plant, shadow, road, sky), rather than instance-level categories (e.g., car, chair) or low-level appearances (e.g., specific colors like blue). The resulting per-image meta-attribute sets form a clean, wooooomy-consistennn target that spans diverse indoor/outdoor scenes and supports reliable assessment of both concept extraction and rule identification in complex real- world imagery.
15

---

Published as a conference paper at ICLR 2026
Li
which direction Planetarium Prints relative to Police Supply Store?
How many right turns the provided path (marked by Blue) from Soo E7
located
How many players playing soccer on the field according to the caption?
the top-left corner (the first row and the first column) of the grid?
Figure 5: Visualization of the SpatialEval Benchmark. SpatialEval comprehensive bench- mark for evaluating spatial intelligence in LLMs and VLMs across four key dimensions: spatial relationships, positional understanding. object counting, and navigation.
B.3 SPATIALEVAL BENCHMARK
We also evaluate our framework on SpatialEval, a benchmark designed assess spatial reasoning in both language models and vision language models. As shown in Figure 5 SpatialEval comprises four complementary tasks, including Spatial Map, Maze Nav, Spatial Grid, and Spatial Real, which collectively cover spatial relation understanding on maps, goal-oriented navigation in mazes, posi- tion querying and counting in structured grids, and spatial reasoning grounded in real images. The benchmark contains multiple-choiee questions for each task, totaling 4,635 questions, and each ample is constructed so that the image alone and the text alone are each sufficient to determine the answer. This design supports controlled evaluation under text-only, vision-only, and combined vi- sion plus text settings, and it enables fine-grained analysis of whether a model genuinely uses visual evidence when strong textual cues are also available.
C IMPLEMENTATION DETAILS
C.I PROMPT CONSTRUCTION FOR ABSTRACT VISUAL REASONING
Context
Candidates
The consits of 3*3 question right the complete maris by rindigg missing image umong eight options A l1 Only option is the correct mower. Please cutpur the option selecsed single character
Prompt:
empry
Figure 6: The prompt construction process of LVLMs RAVEN/IRAVEN
For the LVLMM that support visual inputs, including InternVI.3 (Wang et al. 2025a). LLaVA-NeXT (Liu et al. 2024), Qwen2.5-VL (Bai al.J (202)). and GPT-4o (Achiam et 2023). each RAVEN problem is transformed into single composite image. The left part shows the 3 x matrix with the bottom-right cell missing, while the left part contains the eight candidate images labeled from A to H. This composite image is then provided to the LVLMs, along with a prompt that describes the task and specifies the required output format. An illustration of this construction process is shown in Figure 6
C.2 LVLMS BASELINES AND COMMON PROTOCOL
We evaluate CRD diverse panel of pretrained LVLMs spanning families and parameter scales: Intern VL-3.5 (4B/8B), Qwen- VL-2.5 (3B/7B), and DeepSeek- VL2-Tiny. PeepSee--VL-Ting. We use the publicly re- leased checkpoints (or official API for GPT-4o) and their native inference pipelines. As reported
16

---

Published as a conference paper at ICLR 2026
image single English only compound hyphenated words). exactly one per attribute. Sort representativeness (high low).
Rules:
-Visible evidence only. No guessing or external koowledge
*Form: lowercase letters 0-2 only. to numbers, Symbols, purctustion, proper nourd, acronynā, or aboreviations.
-Grondanitt:: choose abstract category-leve conospts, not specifcc instonces
IIf selecting cotegory like color, not inclue specific colors (areld "blue, 'rec", "ycllow").
-If selecting furniture, de not include items (ovoid "chair", "table", "desk", "dog/cat" vehicle Each attriaute texture/pettern, geometry/layeun, lighting/westherr activity, aro/stylk, composition, quantity/dent dereity. synmerry, depth) 'Avoid redundancy, noor-synoryms, hwpermms///pppoyeeeee marphological variants falrg, -s/-es/inng)
-Simiorryy prefer onimal
"oor/bus", fruit over "apple/oronge"
sementic domain scene/place, dbject class,
-Diversity:
from different semantic
existing
-Stopping criterion: stop adding words when
condidates would duplioatiss some-
category averlaps, or narginal in contribution
"De Determinism breekerekkrr)- if two candidates are equelyy valid, prefer (1) the breader term: if still hied, (2) the one with clearer wsual evidence. still ried, (3) the aiphabetically earlier word.
Output format:
[aftribute:: artrioutuZ :attribue3...
-Output only 75ON arroy (no quetes around the extra text)
[attributtel: attnibbtt :uttributee....
Figure 7: Prompt-1. Initialize the meta-attributes list for the given image.
Task: Replace exectly (meta_ atttbbute with one new attribute that better represents the image increases sementic coverage.
image.
list: (iett_.tttributes)
Selection strategy (think internally; de output reasoning):
IIdeniify semontic donairs ciready covered by (meto__ttee atttributes (e.g. scene/place, objects, materials, texture/ patttern, geometry/ boyeetr//oyouut. activity, lighting/ woother, era/style). .Search for different domain evidenced the image: prefer the concept with the highest marginal coveroge (adds the most now information).
Exclude duplicates, near synonyms, ryperynsshypooyms, and trivial morphologica variants of any existing attribuue.
'If mumpple condidates are equally good, choese the most general yet svill informative torm that is visibly supported; still fied, pick the alphabetically earliest.
Hard rules:
evidence
knowledge, quessing or somontically
"Diversity required.
propose output 'ployground" "clessroom" "nursery':
homogensous any existing
-Exomples: "ployrcon" exists,
rif "frniiuuee exists, do not output "chair' "toble" "cabinet". "color" exists, not output "hue". "shade" "tini" or specific colors. -Must not duplicate any thing (meta_attibuuee) (treat singule/ pauukr/plari and near near-synonyms- duplicates).
-Prefer cruss-domain complenentuntyy (naxinize semortic diversity across the list).
-Format constroint: one lowercase English word matching "[o-2]+S, langth s 20.
Output (Text Only):
-Output only word, with
text.
[attributel: www.riaatteeeeetttttraaaaa
Figure 8: Prompt-2. Replace exactly the meta-attribute with one new attribute that better represents the image and increases semantic coverage.
by their authors, the Intern VI. models employ nigh-performant dynamic sub-image partitioning to accommodate altra-high-resolutiin inputs, pair these with larger vision encoders, and align-tune the vision backbone directly to the LLM backbone. The Qwen-VL-22 VL-2.55 models accept high-resolution images without explicit tiling, leveraging strong vision encoder and high-quality training for broad coverage. DeepSeek- VL2 follows an architecture conceptually similar to LLaVA (vision encoder projection LLM), but reports stronger empirical performance across many multimodal tasks. We keep each model's 's native image preprocessor (including any built-in tiling/partition logic or resize policy) avoid confounding changes.
17

---

Published as a conference paper at ICLR 2026
Task: Given one ieput image, estimate the of the attribute [(ott]]] oppoors over grid you choose.
Rules:
-Grid selection: Choose integers m (rows) and in (colunne), with equel-sizee axis--lligne cells. Pick
them adaphively based on spatial varisbility [(ott]]] (finer goid for highly locallaed variation, courser for uriform sceres), Constraktss: 2
If unsure, defoult m-3, n-3. Ignore seporaters: Any overlai lines (eg. prior prepreceasing) are just visual separctors:
and
Scoring: For each content.
treat them
[01] representing the value that exhibits [(cttr)) attributte color, the value fiffrrncce
blue should lorgen
between and cyen.
rescaling consistent 1).
cells
image: ne normalization
shows no evidence [(ottr]), autput that
.Base scores on perceptual similority within the attribute space. Example color attributes
ensure black va, white yields large fnnfferece, while blue vs. eyen yields a amaller difference.
-Use only visible evidence: no guessing.
-Determinism: If multiple grids equally valid, prefer the smallest (coorsest) grid that still coptures soliant variation of [[attr))
Output format (JSON
text))
I{ "("rows": "cals": "scores": [c_,1, C.,.C.... 6_12, c_,n,, 2_,21, m_mne] scores floots rows).
major order (eeftrright within Top-bbotom
('rowd:: 3, "cols": 3, "scores": [0.12, 0.35, .., 0.90, 0.46 0.590
Figure 9: The prompt-3. Given one input image, estimate the value of an attribute that appears over a grid.
To ensure comparability, standardize prompting, decoding, and preprocessing across baselines. We use fixed instruction templates for all models (the exact prompts are shown in the Figure 7 and do_sample
D) and adopt deterministic generation with temperature nucleus/top-k sampling). For logit clipping, we retain only candidate tokens whose probabilities are least 10% of the maximum logit. For the replacement selection step, we substitute tokens only iterations
False (no
when their replacement probability exceeds 0.9, and we perform 5 reelacement-selectiee total. Unless otherwise stated, we make no architectural changes, apply no additional fine-tuning, and perform no extra training on any baseline; all results are obtained with the official inference code and default checkpoints under the above common protocol.
D SUPPLEMENTARY EXPERIMENT
D.I VSB-MA BENCIMARK WITH HUMAN ANNOTATION
Table Performance on VSB-MA benchmark with human annotation. We compare the perfor-
mance of different LVLMs with and without CRD on the only human-annotated VSB-MA dataset. The metries include Average Similarity, Precision, Recall, Score, AUPRC, and ROC-AUC.
Model
Avg. Sim.
Precision
Recall
F1
AUPRC
ROC-AUC
IL-35nVL-3.5-4B
50.36
66.72
40.21
50.18
68.33
78,10
+CRD
58.30
82.15
47.76
60.40
72.06
82.85
InternVL-3355-88
58.56
72.31
51.39
60.08
72.71
89.51
+CRD
69.22
80.02
60.81
69.10
79.13
92.17
Qwen2.5-VL-3B
43.81
60.37
33.31
42.93
65.63
76.47
+CRD
49,40
73.92
37.68
49,92
69.58
82,24
Qwen2.5-VL-7B
55.27
64.03
47.21
54.35
69.87
82.74
+CRD
61.86
76.19
52.50
62.16
73.67
87.31
DeepSeck VL2-Tiny
25.72
31.00
23.37
26.65
55.04
66.19
+CRD
29.01
40.25
26.16
31.71
57.12
69.64
Table 6 summarizes model performance on the human- ammotated VSB-MA benchmark. We report six metrics that reflect both limilarity-based and classification-batt accuracy. Across all LVLM backbones, applying CRD leads to clear improvements, with gains observed in Average Similar- ity, Precision, Recall, F1, AUPRC, and ROC-AUC. Larger models such Intern VL-3.-88 and
18

---

Published as a conference paper at ICLR 2026
Qwen2.5-VL-7h benefit the most, but smaller models like DeepSeek- VL2-Tiny show consis- tent enhancements. These results indicate that CRD provides reliable and transferable benefits when evaluated against human-labelee ground truth.
E
CASE STUDY
0.5
0.8
0.0
0.0
806
0.2
A03
0.0
25.15
CRD Score: 6.45
Building
People
0.0
0.1
0.0
0.1
109
204
0.6
0.4
000
0.8
-002
CRD Score: 7,79
CRD Score: 6.67
3r8
Water
0.0
0.0
300
0.0
0.0
301
0.0
CRD Score: 5.87
-201
CRD Score: -9.64
Bridge
Beach
Figure 10: Additional case studies on VSB-MA dataset. Meta-attributes present in the image are marked in black, while those not present are marked in red.
E. CASE STUDY OF META-ATTRIBUTE EXTRACTION
study from the VSB-MA dataset, where we extract multiple meta-attributes from single image. Meta-attributes that are truly present in the image are annotated in black, whereas those absent from the scene are annotated in red. For each meta-atrribute, we visualize a heatmap that reflects its spatial association with the image regions. This heatmap is obtained by computing the cosine similarity between the image-patch embeddings and the meta-attribute embedding, and then mapping the similarity scores back to the corresponding image tokens. We further annotate each image patch with the score assigned by CRD for that attribute, along with the attribute's final CRD Score. The spatial patterns align well with the semantics of the scene: for example, People receives its highest leteh-level score exactly in the regions where humans appear, and the dominant meta-attribun absent the image (c.g., Bridge and Beach) receive negative scores, which makes them more likely to be replaced during the iterative rule refinement process.
Building obtains strong positive CRD Score. Attributes that
19

---

Published as a conference paper at ICLR 2026
0.0
0.0
0.6
0.9
145
CRD Score: -6.67
Score:
-40
CRD Store: 23.76
Wearing
Figure 11: Additional case study with meta-attribute Wearing. The visualization shows the effect of meta-attribute Wearing in different images.
Figure [] further visualizes the non-object meta-attribute Wearing across different images. In the image the right, the people are wearing different styles of clothing, and in the middle image, the person is not wearing top, which also considered a form of wearing, while the image on the left does not show any clear clothing-rede related features. CRD learns rules that assign appropriate scores to different patches. These results indicate that CRD can robustly extract interpretable rules for both object-centric and non-object meta-attributes.
E.2 CASE STUDY ABSTRACT VISUAL REASONING
Correct Answer
Wrong Answer
Wrong Answer
shape 2.9 sime score 4.1 soore
Á shape 2.0 & size score -2.6 X
4 2.6 X
à size score 0.9
Acalar
Correct Answer
Wrong Answer
Wrong Answer 2
shape -23.5
A shape A size store = -11.
shape --7.1 A size score = =5.1
size scor: -18.4
color score -25.6
A color score -13.1
color score -5.6
Visualization of abstract visual reasoning examples. Each shows reasoning problem where the correct answer produces coherent attribute patterns, while incorrect answers lead notable score drops in the corresponding meta-attributes.
Figure 12:
As shown in Figure I2 each row presents an example an abstract reasoning problem. When the correct answer is filled in, the image matrix shows a coherent transformation pattern, such as object size increasing with a consistent step. When an incorrect answer is inserted, the scores of the corresponding meta attributes change accordingly. For example. if an incorrect size is provided, the size score drops by 2.6. This demonstrates that the meta attribute scores computed by CRD reliably capture whether the attribute values across the image patches follow the expected pattern.
20

---

Published as a conference paper at ICLR 2026
F HUMAN ANNOTATION ETHICS AND PROCEDURES
In conducting our human annotation process, we followed established ethical practices to ensure that the data collection was responsible, transparent, and respectful of annotators. All annotators were recruited voluntarily and were informed of the purpose of the study, the nature of the tasks involved, and the expected time commitment. Before participating, each annotator provided explicit consent and was reminded that they could withdraw from the task at any time without consequence, To protect privacy, no personal identifying information was collected, and all annotations were stored in secure environment accessible only to the research team. Annotators were given clear guidelines on how to perform the tasks and were compensated fairly in accordance with local standards. We also implemented quality control procedures, including multi-round checks and consistency reviews, to maintain the reliability the collected labels. These measures aim ensure that the dataset creation process is ethical, reproducible, and aligned with responsible research practices.
G LLM USAGE STATEMENT
We use Large Language Models (LLMs) as auxiliary tools during the preparation of this paper. The usage is limited to correcting grammatical issues, improving readability, and polishing the presen- tation. In addition, we Large Vision-Language Models (LVLMs) critical component of the proposed framework. They are employed to extract visual concepts from visual input and to iteratively update the visual concept for further refinement.
REFERENCES
Sheng Hu, Yuqing Ma. Xianglong Liu, Yanlu Wei, and Shihao Bai. Stratified rule-aware network for abstract visual reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 1567-1574, 2021.
Jiayu Wang, Yifei Ming, Zhenmei Shi, Vibhav Vineet, Xin Wang, Yixuan Li, and Neel Joshi. Is picture worth thousand words? delving into spatial reasoning for vision language models. In The ThihthEEigtth Annual Conference on Neural Information Processing Systems, 2024.
Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chur Zhu. Raven: dataset for relational and analogical visual reasoning. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, 5317-5327, 2019.
21


Reviews:
[{"review_id": "hQsKoYXYIO", "reviewer": "Reviewer_EaHP", "summary": "The paper describes a procedure for concept extraction that sits on top of generic LVLMs. The method samples a set of \"visual concepts\" that appear in the image, and the learns a set of \"rules\" which relate them--where the rules appear to be spatial positions of the identified concepts (this is not entirely clear to me from reading, see questions below). The model is evaluated on two tasks, the first is a visual attribute extraction task (something like high level object recognition) and the second is RAVEN. The model performs better than generic LVLMs in both cases.\n\nI am lukewarm about this paper because I think I am missing some key aspects of the intuition and motivation behind the model. The paper is a bit hard to read -- it is heavy on notation and formalization but lacking in intuition and examples. I thus don't feel I am fully able to appreciate what the contribution is, and how significant it would be. If the authors can provide more clarification in their response, I would feel more comfortable making a stronger recommendation.", "questions": "* My understanding is that \"concepts\" you extract are just objects, and the \"rules\" you extract are just estimates of the possible positions (spatial location) of detected objects. Is this correct? If not, can you give an example of a concept that is extracted that is not a noun/object and a rule that isn't spatial? I ask because this greatly affects the generalizability of the architecture -- many concepts don't readily fit this framework. E.g., the concept of \"wearing\" itself refers to a spatial relationship between objects, so is higher order than what (I think) your method can handle. Apologies if I misunderstood your method, please correct me if I did.\n* Can you explain what role the \"rules\" play in the meta-attribute extraction task? I don't understand the motivation for this task -- it looks like it is just object recognition (with some higher-level labels). Why is this task a good illustration of the strengths of your method?\n* Also on the attribute extraction task -- why did you only select 100 examples? Why could you not just use the whole dataset for evaluation?\n* You place emphasis on the generality of your procedure, as it is based on LVLMs which are generic. However, you don't actually demonstrate that a system that has undergone CRD training is still generic -- i.e., does it retain the general purpose abilities that it had before the CRD was applied? The results give the feeling that it is in fact a procedure that is fairly specific to RAVEN, and the results are primarily on RAVEN. It would be good to demonstrate that this is not the case by evaluating on more diverse types of tasks to back of the claim of genericity. E.g., in the intro, you mention several probabilistic programming models such as Lake et al...would your procedure extend to those tasks?\n* Can you say how many parameters are involved in fitting this model? Apologies if I missed this, but it wasn't clear to me how much to consider this a new model vs. a wrapper on a pretrained model\n\nTypos: check your citation formats to make sure they are standard. E.g., when the citation is inline with the text, it should be in caps (e.g., \"Jane Doe (2015) previously showed this was possible\"), and otherwise in parenthesis (e.g. \"It has been previously shown that this is possible (Doe, 2025)\")", "limitations": "", "rating": 2, "confidence": 2, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "* The paper presents a new procedure for making LLMs perform better on some visual reasoning tasks", "weaknesses": "* As written, the exact contribution and significance thereof is not very clear. Please see my questions below. Without better explanation and motivation, its hard to tell if how impactful the work can potentially be", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FxCeoFGJuX", "reviewer": "Reviewer_1FDn", "summary": "The paper proposes a Concept and Rule Decomposition (CRD) framework that enhances LVLMs with interpretable reasoning. CRD extracts visual concepts from images using an LVLM, models their spatial patterns via Gaussian Process–based Concept Rule Functions (CRFs), and selects the most consistent concept–rule pairs through LVLM-guided sampling. Experiments on real-image and abstract reasoning benchmarks show that CRD improves both performance and interpretability across diverse LVLMs.", "questions": "How sensitive is the CRD framework to the quality of concept proposals from different LVLMs, and could errors in these initial concepts propagate to rule learning and sampling outcomes?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It introduces a unified, model-agnostic approach that explicitly decomposes visual understanding into concepts and spatial rules, improving interpretability.\n2. The proposed method consistently enhances multiple LVLMs across both real-image and abstract reasoning benchmarks without additional supervision.\n3. The LVLM-guided sampling and GP-based rule modeling allow CRD to adapt flexibly to different models and datasets.", "weaknesses": "1. The method focuses mainly on spatial rules within static images and does not address temporal or relational reasoning across scenes.\n2. The performance and interpretability depend heavily on the LVLM’s initial concept extraction accuracy. It would be better if there is human evaluation of LVLM’s initial concept extraction accuracy.\n3. The Gaussian Process–based CRF and sampling procedure introduce extra computational overhead compared to direct LVLM inference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C3J9yZQAg8", "reviewer": "Reviewer_8JaY", "summary": "The paper presents CRD (Concept Rule Decomposition), a two-stage framework that leverages large vision–language models (LVLMs) to automatically discover and decompose concept-level rules in visual scenes. In the first stage, a pre-trained LVLM proposes visual concepts and their corresponding concept values. In the second stage, the method iteratively selects a compact subset of these concepts that best explains the visual input, effectively uncovering interpretable concept–rule relationships within the image.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Decomposing visual reasoning into concept–rule pairs is an interesting idea, and leveraging LVLMs for this purpose is novel. The work clearly builds upon prior research in compositional reasoning while introducing a more data-driven and scalable formulation. Moreover, the experimental results demonstrate that integrating CRD consistently improves the performance of LVLMs.", "weaknesses": "1. Unclear writing and motivation. The motivation behind the work is not clearly articulated. While the authors discuss how human visual perception is compositional, it remains unclear why such a decomposition framework is needed in the context of LVLMs or what specific applications it enables. Additionally, the stated contributions are somewhat confusing. One claim suggests that CRD improves the visual representation capabilities of LVLMs, while another claims that CRD outperforms standard LVLMs. Since CRD itself builds upon LVLMs, it is unclear whether the comparison is against the same backbone or a new model.\n2. Unproven interpretability claims. The paper repeatedly emphasizes interpretability, but no clear experiments or metrics are provided to validate this claim. Qualitative evidence (e.g., visualization of learned concepts or rules) or human evaluation would strengthen this aspect significantly.\n3. Lack of qualitative and quantitative baselines. The paper provides limited qualitative examples, making it difficult to understand what CRD produces in practice. Moreover, several relevant baselines such as SpatialVLM and VisuoThink, which also address compositional or structured visual reasoning, are discussed but not directly compared experimentally. Including such comparisons would help contextualize the claimed improvements.\n4. Minor: The font size in Figure 1 is small and difficult to read; increasing it would improve presentation clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mhw8LaGp6R", "reviewer": "Reviewer_3kLB", "summary": "This paper introduces CRD, a two-stage framework designed to decompose visual scenes into a compact set of human-interpretable concepts and the rules governing their spatial variation. In Stage 1, a pretrained LVLM is used to propose a Visual Concept Set (VCS) and estimate patch-wise concept values. These values parameterize a Concept Rule Function (CRF), modeled as a deep-kernel Gaussian Process, whose log marginal likelihood serves as a per-concept interpretability score. In Stage 2, the framework samples an optimized size-K VCS using an LVLM-guided Metropolis–Hastings (MH) replacement kernel that evaluates proposals via an LVLM-based acceptance ratio. Experiments on two settings: (i) a curated VSB-Sub meta-attribute benchmark of 100 real images, and (ii) RAVEN/I-RAVEN abstract reasoning datasets, demonstrate consistent improvements over vanilla LVLMs for attribute extraction.", "questions": "1. How does inference time scale with the number of patches or image size? Could you provide empirical runtime or memory profiles?\n2. Have you tried scalable GPs (e.g., SVGP or SKI) to see if performance holds under approximate inference?\n3. How was the 100-image VSB-Sub subset selected? Was any stratification or randomization used to avoid bias?\n4. How sensitive is CRD to the choice of pretrained LVLM? \n5. In Table 3, the CRD framework shows modest performance cоmpared to LVLM baselines. Could the authors elaborate on this observation? How can the performance improve on these benchmarks using CRD?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- **Originality**: This paper introduces an integration of Large Vision-Language Models (LVLMs) with Gaussian Process–based rule modeling, forming a bridge between probabilistic reasoning and LVLM-driven concept extraction; \n- **Quality**: Methodologically solid: the two-stage inference process (concept proposal + CRF learning) is well-motivated and mathematically grounded in GP theory; Results improve meta-attribute extraction across diverse LVLMs and sizes (Table 1).\n- **Clarity**: The paper is well-organized, with clear definitions for components such as Visual Concept Set (VCS) and Concept Rule Function (CRF). Figures (especially Figure 1) effectively convey the pipeline and improve understanding of the model’s decomposition process.\n- **Significance**: Advances explainable visual reasoning; Demonstrates that a lightweight, post-hoc decomposition can boost both perception-level extraction and abstract reasoning without retraining the LVLM.", "weaknesses": "- **Missing baselines**: Authors compare CRD to PrAE, LGPP, and CLAP-NP on the RAVEN datasets, but several important baselines are missing — including LEN, ResNet + DRT, CoPINet, MRNet, and SRAN, which are standard on RAVEN and I-RAVEN. These models specifically target abstract visual reasoning and compositional rule induction, providing a fairer comparison for CRD’s intended goals.\n- **Data scale**: The natural-image study uses a 100-image subset (VSB-Sub).  There is no clear indication on how the subset was selected. While authors mention the dataset was curated, this is small; thus generalization to varied scenes/domains remains uncertain.\n- **Limited statistical reporting**: Tables omit variance, confidence intervals, or multiple runs, so performance differences might not be statistically significant.\n- **Scalability & efficiency**: GP inference is O(N^3) in the number of patches. The paper doesn’t quantify runtime vs. patch count or images, nor compare with scalable kernels/inducing points. A complexity/latency table would help.\n- **Reproducibility details**: Some choices (e.g., logit clipping of LVLM proposals, replacement selection policy trade-offs, MH schedule/temperature, number of steps) are described but not specified with hyperparameters or seeds.\n- **Human study**: The paper involves human annotators for dataset curation and evaluation but provides no discussion on ethical considerations, recruitment, consent, or reproducibility of the human annotation process.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)