Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs
Authors: Hengwei Ye
Yuanting Guan
Yuxuan Ge
Tianying Zhu
Yijia Zhong
YiJing Zhang
Han Zhang
Yingna Wu
Zheng Tian
Venue: ICLR 2026 Poster (2026)
Keywords: Benchmark
MLLM
Reasoning
Primary area: datasets and benchmarks
TL;DR: KidGym is a comprehensive 2D grid- based benchmark for evaluating five core capabilities of MLLMs based on the Wechsler Intelligence Scale.

Abstract: Multimodal Large Language Models (MLLMs) combine the linguistic strengths of LLMs with the ability to process multimodal data, enabling them to address a broader range of tasks. Because MLLMs aim at more general, human-like competence than language-only models, we take inspiration from the Wechsler Intelligence Scales — an established battery for evaluating children by decomposing intelligence into interpretable, testable abilities.
Inspired by the Wechsler Intelligence Scales, we introduce KidGym, a comprehensive 2D grid-based benchmark for assessing five essential capabilities of MLLMs: execution, perception reasoning, learning, memory, and planning. 
The benchmark comprises 12 unique tasks, each targeting at least one core capability, specifically designed to gauge MLLMs' adaptability and developmental potential, mirroring the stages of children's cognitive growth. Additionally, our tasks encompass diverse scenarios and objects with randomly generated layouts, ensuring more accurate and robust evaluation of MLLM capabilities. 
KidGym is designed to be fully user-customizable and extensible, allowing researchers to create new evaluation scenarios and adjust difficulty levels to accommodate the rapidly growing MLLM community.
Through evaluation of state-of-the-art MLLMs using KidGym, we identified significant insights into model capabilities and revealed important limitations of current status. We release our benchmark at: https://kidgym.github.io/KidGym-Website/.

Published as a conference paper at ICLR 2026
CHILDREN'S INTELLIGENCE TESTS POSE CHALLENGES FOR MLLMS? KIDGYM: A 2D GRID-BASED REASONING BENCHMARK FOR MLLMS
Hengwei Yel, Yuanting Guan', Yuxuan Gel, Tianying Zhu', Yijia Zhong', Yijing Zhang', Han Zhang', Yingna Wu', Zheng Tian I+ 'ShanghaiTech University
ABSTRACT
Multimodal Large Language Models (MLLMs) combine the linguistic strengths of LLMss with the ability process multimodal data, enabling them address broader range of visual tasks. Because MILMS aim at more general, human-like competence than language-only models, we take inspiration from the Wechsler Intelligence Scales - an established battery for evaluating children by decom- posing intelligence into interpretable, testable abilities. We introduce KIDGYM, a comprehensive 2D grid- brid-based benchmark for assessing five essential capabilities MLLMs: Execution, Perception Reasoning, Learning, Memory, and Plan- ning. The benchmark comprises 12 unique tasks, each targeting at least one core capability, specifically designed to gauge MI.LMs² adaptability and developmen- tal potential, mirroring the stages of children's 's cognitive growth. Additionally, our tasks encompass diverse scenarios and objects with randomly generated layouts, ensuring more accurate and robust evaluation of MILM capabilities. KIDGYM is designed to be fully user-customizaale and extensible, allowing researchers to create new evaluation scenarios and adjust difficulty levels accommodate the rapidly growing MLLM community. Through the evaluation of state-of-the-ar MLLMs using KIDGYM, we identified significant insights into model capabili- ties and revealed several limitations of current models. We release our benchmark at: htttps://kiiiie githubtttuuuu. woo/oi///ii Webpyynn/Wb/b//////e
INTRODUCTION
Large language models (LLMs) have demonstrated significant success across language-based tasks (Brown et al., 2020; Sharan et al., 2023). laying a strong foundation for advancements in artificial intelligence. Following this success, multimodal large language models (MLLMs), which integrate multiple data modalities such as images (Wang et al. 2024f) and videos (Cai et al. 2024 Wang et al., 2024c), are also experiencing rapid growth and development. By fusing diverse infor- mation sources, MLLMs enable AI to learn and reason (Gao et al. 2024) across different modalities, bringing it closer numan-like cognition (Du et al. (2024).
Human cognitive testing frameworks (Smith & Gasser 2005) have been well-established and re- fined over time, and insights from cognitive developmental psychology have consistently provided valuable guidance in advancing artificial intelligence (Lake et al., 2016) Wu et al.) 2024a] [Sumers) jet al. 2024; Gaeera) 2025). Research in psychometric Al and universal psychometrics shows that ability-orientdd batteries adapted from validated human tests offer a principled way to gauge general reasoning, outperforming mask-specific benchmarks (Voudouris al. 2024 2025). Within this context, the shift from LLMs to MLLMs calls for an evaluation framework that profiles multiple coordinated abilities rather than language alone (Wang al. 2024a Han et al. 2025 Gopnik al.)
2009), aligning well with child intelligence assessments such as the Wechsler scales, suggests that evaluating MI.MSs with fhiuuee cognitive frameworks is particularly promising approach.
While Wechsler framed human intelligence as a constellation of interrelated abilities that vary in degree, the cognitive profile of MLLMs cannot mapped one-to-one onto these human con-
"Correspondence Zheng Tian haanrhoon@@saangaaiccc......

---

Published as a conference paper at ICLR 2026
(a) Classification
(b) Selection
(c) Sorting
(d) Maze
(e) Filling
Puzzle
Placement
(h) Counting
ii) Decode Maze
ii) Memory Maze
(k) Memory Decode
(l) Memory Filling
Previews of 12 tasks in KIDGYM. The circular chart in the upper-right corner of each subfigure represents the cognitive abilities required by the task: E for Execution, M for Memory, L for Learning. P for Planning, and PR for Perception Reasoning.
Figure 1:
structs (Bender & Koller 2020). Consequently, each MILM capability requires bespoke definition that respects the model's architectural and functional particularities (Binz & Schulz 2023).
In this work, we introduce KIDGYM, a new benchmark specifically designed for evaluating MLLMs' cognitive abilities. Drawing inspiration from the Wechsler Intelligence Scales (Guertin ct al. 1966) Zhu et al.) 2004 Zeigler-Hill et al., (220). widely recognized children's 's intelligence test, we summarized and defined five essential capabilities that MLLMS require in the current state: Execution, Perception Reasoning, Memory, Learning, and Planning.
KIDGYM comprises 12 carefully designed tasks: focused on testing individual capabilities and 6 on assessing integrated dual capabilities. To ensure robust and reliable experimental results, our tasks cover wide range of scenarios and objects with randomly generated layouts. Furthermore, to evaluate the performance limits of various MI.LMs, each task is presented at three difficulty levels (L.1, 1.2, I.3) from easy to hard. In order to support customization, we built the benchmark based on the Gym API (Brockman et al. 2016), allowing researchers to create new evaluation scenarios to accommodate the rapidly growing MLLM community.
We benchmark representative set of state-of-the-art MLLMs on KIDGYM, including closed- source models: o3 (OpenAl 2025a), GPT-5 (OpenAl 2025b), GPT-4o (OpenAl 2024). Gemini- 2.5-Pro (DeedMind (025), Gemini-2.5-Flash (DeepMind 2025). Claude-3.7-Some Sonnet (Anthropic
2025
and strong open-source models: Deepseek (Wu et al. 2024c), Qwen VL-2.5 (Bai et al.
2025
Intern (All ab 2025) with different sizes.
2

---

Published as a conference paper at ICLR 2026
Through systematic experiments, closed-source models achieve near pear-perfect scores on specific tasks and excel particularly in learning tasks. Among these, o3, GPT-5 and Gemini-2.5-PP signifi- cantly outperforms the other models by significant margin across all capability dimensions.
However, we identified 4 key challenges for current MLLMs not captured in previous benchmarks:
First, models show limitations in reasoning over non-semantic, abstract visual information.
Second, models are sensitive to item quantity.
Third, models struggle with composite capacity tasks involving the interaction of multiple rules.
Finally, models perform relatively poorly on perception reasoning and planning tasks.
Our contributions can be summarized as follows:
1) We propose an assessment framework for MLLMs, incorporating five core abilities based on the Wechsler Intelligence Scale.
2) We introduce KINGYM, unified 2D benchmark for MLLMs, featuring diverse environments, randomized layouts, graded difficulty levels, and customization options.
3) We conduct systematic evaluation of state-of-the-art MLLMs, highlighting empirical strengths and weaknesses, and providing insights for future development.
RELATED WORKS
2.1 MULTIMODAL LARGE LANGUAGE MODELS
LLMs (Ouyang et al. 2022: Touvron et al. 2023
Chung et al. 2024) have evolved from processing
solely text-based inputs to exhibiting multimodal capabilities. This advancement has significantly expanded the applicability of MLLMs in areas such as image description (Liu et al. 2016 Tan et al., 2024), image reasoning (llevskii & Feng 2017 Wang et al. 2024e Xiao et al. 2024). ) and visual question answering (VQA) (Gaur et al. 2024 Wang et al. (2024b), bringing closer to the ultimate goal of AI research: general artificial intelligence (AGI) (Zhong et al. 2024). which aims to develop systems capable of matching surpassing human-level performance across diverse domains.
2.2 MLLM BENCHMARK
Numerous benchmarks have been developed over time to assess the capabilities and performance of MLLMs. Initially, these benchmarks primarily focused on evaluating MLLMs' ability to pro- cess and understand multi-modal data, such image comprehension and analysis (Li et al. 2023 Xu et al., 2023; Yin et al., 2023 Yu et al., 2023; Fu et al. 2024). As MILMS demonstrated pro- ficiency in recognition tasks (Kuchibhotla
al.) 2024), attention shifted toward evaluating their reasoning abilities (Shi et al.) 2024 Han et al. 2023). including inductive, deductive, and abductive reasoning (Huang & Zhang 2024). Advancing further, a diverse range of specialized benchmarks has emerged. focusing on various aspects of MLLM capabilities across different application scenar- ios (Li et al. 2024). including creativity in image generation (Fang al. 2025). information pro- cessing in long contexts (Song et al. (024) and human-level planning in real-world problems (Chen)
al.
2024). highlighting the rapid evolution of MLLM field.
However, current MLLM benchmarks predominantly evaluate static tasks where information re- mains constant throughout (Amini-Naieni al., 2024] Cao et al. (024), rather than dynamic tasks requiring continuous environmental interaction and adaptation (Xu et al.) 2024). Dynamic tasks require the agent to follow a trajectory or execute sequence of actions through continuous interac- tion to ultimately complete the task, rather than answering in the simple question-and and-answer format typical of static tasks (Gonzalez) 2005). Furthermore, most existing benchmarks typically assess iso- lated capabilities (Krishna et al. 2025), providing limited insight into how the diverse competencies of MLLM compare or interact in real-world contexts (Tihanyi et al. 2024). Positioning KIDGYM against this backdrop, Table provides structured comparison with representative benchmarks.
One promising approach to addressing these limitations is the use games as benchmarking tools (Juliani al. 2019 Samvelyan et al. 2021 Gan et al. 2021). Games offer dynamic, multi-
3

---

Published as a conference paper at ICLR 2026
Table Comparison of KIDGYM with existing benchmarks across target paradigm, difficulty-Ieeel support, user extensibility, evaluated capabilities, and dynamic vs. static settings.
Benchmarks
Target
Difficulty Level
User Extensible
Capabilities
Dynamic/Static
Crafter
RL
X
x
Dynamic
MiniGrid
RI.
Dynamic
Learning
LogicGame
LLM
V
X
Planning
Static
Execution
EgoPlan
MLLM
X
X
Planning
Dynamic
MileBench
MLLM
X
X
Memory
Static
Countgd
MLLM
X
X
Counting
Static
CompBench
MILMM
X
X
Reasoning
Static
MaRs-VQA
MLLM
X
X
Reasoning
Static
ARC-AG1-2
MLLM
X
X
Reasoning (Abstract)
Static
KIDGYM
MLLM
All Above
Dynamic
dimensional environments that can better simulate complex, interactive tasks. For instance, Mini- Grid (Chevalier-Boisse et al. 2023) provides a suite of goal-oriented game environments, but it was originally designed for reinforcement learning. SmartPlay (Wu et al. 2024b) incorporates six classic games, including Minecraft (Johnson et al. 2016) and Crafter (Hafner 202T), converting gameplay scenarios into text-based descriptions to evaluate key LLM capabilities such instruction following and error correction. While tabletop games (Costarelli et al.) (202). logical games (Gui et al., 2024) and board games (Topsakal et al. 2024) have been utilized for model evaluation, these approaches predominantly focus on text-based assessments and are less suitable for evaluating MILMs.
To fill these gaps, introduce KIDYYM, suite of interactive and dynamic scenes, which not only assess individual capabilities but also enable the simultaneous evaluation of multiple dimensions of intelligence (e.g., memory and planning). This integrated evaluation offers more accurate and comprehensive understanding of MLLMs' strengths and weaknesses.
CAPABILITIES
Humans and MLLMs differ fundamentally in their embodiment and interaction modalities, so a literal transfer of abilities and subtests is inappropriate. Given these limitations, do not simply copy the Wechsler test. Throughout the development of KIDGYM, we collaborated with co-authors who experts in child brain science and translated the most important Wechsler indicators into five core competencies that are critical for MLLMs.
Children's 's behavior is widely viewed the realization prior intentions (Scarle) 1983), In cognitive science, this capacity is captured by executive function (Diamond (013) the con- scious regulation of thought and action. Analogously, MLLMs must translate internal representa- tions of goals into concrete behaviors to produce meaningful outcomes. We therefore define execu- tion the capability of an MILM fulfill task on the basis of its inferred goals and constraints. Whether the model is navigating virtual world, manipulating physical objects, or coordinating with other agents, robust execution bridges the gap between abstract objectives and verifiable behavior, ensuring that understanding is consistently transformed into successful action.
Execution:
Memory: Memory allows the human to encode, store, and retrieve information so that past experi- ences guide current decisions (Atkinson & Shiffrin 1968). MLLMs, however, can reread the entire interaction history at every step; their "memory" therefore emphasizes maintaining long-rang con- textual dependencies rather than reconstructing fragmented episodes. Throughout this paper define an MLLM's memory as its capacity to retain previously perceived information, integrate that information into coherent context, and exploit the evolving context refine subsequent reasoning
4

---

Published as a conference paper at ICLR 2026
Background
Game item
Agent
Hint Bar
Backpack
Figure 2: A KIDGYM task frame comprises a scene map, a backpack, and hint bar. We provide varied agent skins, backgrounds, and scene-specific items; backpack slots and in-scene items are letter/number-lbelle for identification. Resolution and grid layout are specified in Appendix B.
and actions (Wang et al., 2024d). Such persistent memory is indispensable for tasks that demand sequential understanding and consistent decision-making making across multiple turns (Zhang al. 2024).
Learning: Learning is the process by which am individual acquires new knowledge, skills, attitudes or behaviors through experience, practice or formal education. The capacity to learn is central cognitive ability that distinguishes humans from most other species. In MLLMs, learning refers to model's capacity to ingest previously unseen information or rules and effectively apply them in decision-makien and problem-solving (Huo & Tang 2025 Tai et al. 2024). A major challenge emerges when the incoming information conflicts with, or supersedes, what the model has already stored. Without further fine-tuning, MLLM must reconcile such inconsistencies on-the- on-the-fly. Real- world tasks are dynamic: new constraints, updated facts and evolving user goals continually arise. Therefore, MILM that can learn, adapt and deploy fresh knowledge without frequent and costly retraining will be more flexible, robust and economically viable practice.
Planning: In human intelligence, planning serves fundamental cognitive process that enables individuals to anticipate outcomes, formulate strategies, and sequence actions to achieve desired objectives. Within the context of MILMs,, planning constitutes the capacity systematically orga- nize tasks, predict action consequences, and implement multi-step strategies for complex problem- solving (Zheng al., 2024)). This capability transcends mere reactive decision-makinn by incor- porating foresight-requirigg models to balance immediate actions against long-term goals while navigating the inherent trade-offs between short-term responses and strategic outcomes.
Perception Reasoning: In the Wechsler, perceptual reasoning measures the ability of children to solve purely visual problems, integrating spatial perception, visual organization, and nonverbal rea- soning. By analogy, we define perception reasoning in the context of MILMs as the capability to draw inferences and make decisions directly from visual inputs (Xiao al., 2025). This capability goes beyond object recognition: the model must analyze visual evidence, construct coherent chain of logic, anticipate plausible outcomes, and choose actions that follow from those predictions.
MECHANICS
The tasks in KIDGYM have been specifically designed with several mechanisms (see Figure 2 that take into account both the strengths and weaknesses current MLLMs.
Diverse Semantic Scenes:
In real-world applications, tasks of the same type often vary based on their contextual scenarios. To capture these variations. we have designed a range of environments, including supermarkets, canteens, and farms, along with corresponding items create immersive,
5

---

Published as a conference paper at ICLR 2026
context-rich scenarios. By evaluating the model in original contexts, where the context is ran- domized for most task types, we can assess whether it has acquired the targeted abilities and can apply them effectively across varying scenarios, rather than relying memorization of similar en- vironments from pretraining data. This helps mitigate data leakage or contamination to some extent.
Randomness: In addition to diverse semantic scenes, variability in task layouts is crucial for sessing MILM robustness. While semantic diversity introduces novel contexts across tasks, layout stochasticity generates distinct configurations within the same task and scene. Each episode initial- izes with randomized element arrangements (e.g.. item locations, agent spawn), ensuring no two rounds are identical. This randomness reduces evaluation variance and ensures more consistent performance estimates. The exact question counts are provided in Appendix [B2].
Backpack and Hint Bar: Current MLLMs often struggle maintain contextual consistency (Chou et al. 2024). particularly when dealing with hidden details not explicitly represented in visual in- formation. For example, an agent may successfully "pick up the key' in one step but fail to recall possessing it in later steps. To address this problem, we designed backpack and hint bar as components of the task state, enabling agents to retrieve crucial information through out the task.
High-level Actions: MLLMs are not well-suited for executing atomic actions such as "go one step forward" or "turn left" in tasks requiring high operability. suited for handling macroscopic concepts and executing high-level actions. Building this, each task in KIDGYM presents MLLMs with high-level actions. For instance, the agent can directly perform actions such as "pick up the basketball" instead of navigating step-by-step its location and interacting with it. This reduction in operational granularity enables the model to focus on actions that are directly tied meaningful outcomes, avoiding low-level controls.
contrast, MILMS are better
Each item in KIDGYM'S 's task scenes is assigned unique identifiers. These identifiers enable the MI.LMs to associate visual elements with text-based descriptions in high-level actions or goals, such as "put the item from backpack A into item number 2." These labels not only optimize information retrieval but also reduce ambiguity in task execution, ensuring that the agent interprets and interacts with the environment accurately.
Identification:
TASKS
We design 12 tasks to evaluate MLLMs: 6 targeting a single capability and 6 targeting composite capabilities. Each task includes three difficulty levels, from easy to hard. KIDGYM follows standard psychometric practice like Wechsler by constructing tasks which one (or two) target abilities are dominant by design. Detailed information for each task can be found in Appendix F
5.1 SINGLE CAPACITY TASK
Classification (CL): In CL task, the agent is required place each item into its designated container based on specific instructions, such as "placing the cherry in the yellow basker" (see (a) in Figure I It is designed to evaluate the MLLM's Execution ability, which involves translating an understand- ing of goals into effective actions. The agent's performance in this task measures its accuracy in following instructions within structured environment.
Selection (SE): In SE task, several random items will appear in the left hint bar at first (see (b) in Figure Once the task starts, these items will be hidden, and the agent need to select the items that appeared in the hint bar before. This task evaluates the MLLM's Memory capability by requiring it to remember and recall the items previously shown.
Sorting (SO): In SO task, the agent is presented with rule that may contradict real-world knowl- edge. For instance, the agent might be instructed that "the faster the animal, the heavier it is". The agent is expected to correctly rank the animals based the given rule (see (c) in Figure I]. This task evaluates the MLLM's Learning abilities, as it requires the agent to comprehend a novel rule that may conflict with its prior knowledge.
Maze (MA): This task is inspired by Procgon (Cobbe et al. 2019), where the agent must obtain the diamond in maze with several locked doors. The agent needs collect the corresponding colored

---

Published as a conference paper at ICLR 2026
keys to unlock these doors (see (d) in Figure D This task primarily evaluates the MLLM's Planning ability, as the agent should carefully devise strategy to reach the diamond with the fewest steps.
Filling (FI): In FI task, the agent will be presented with image in which a quarter section has been removed, such as "a goldfish with a missing head" (see (e) in Figure Then it needs to restore the image by selecting the correct missing piece from set of distractors in the backpack. This task primarily evaluates the MLLM's Perception Reasoning ability, as it requires the agent to develop a holistic understanding of the image and infer the missing part.
Puzzle (PU): In PU task, a target image composed of 4 puzzle pieces is displayed in the hint and the agent needs to assemble the scattered puzzle pieces from its backpack to reconstruct the target (see (f) in Figure [. This task primarily evaluates the MLLM's Perception Reasoning in abstract visual mode, as requires the agent to grasp the image's overall structure, which cannot be easily conveyed through language.
5.2 COMPOSITE CAPACITY TASK
Placement (PL): In PL task, the agent required to place the item in the opposite position based on the given goal. For instance, if the rule states "place the toy car on the north side of the toy train" (see (g) in Figure the agent actually needs to place it on the "south" side. This task primarily evaluates the MLLM's abilities in Learning and Perception Reasoning, as it necessitates understanding of placement rules and the awareness of spatial orientation.
Counting (CO): In CO task, the scene contains several piles of items, with quantities ranging from 1 to 3 (see (h) in Figure D. At the start of the task, the agent is given a target number and then must collect exactly that number of items. This task primarily evaluates the MLLM's Perception Reasoning and Planning abilities, focusing on the agent's awareness of item quantities and its strategic decision making regarding how many items collect single time.
Decode Maze (DMA): This task follows the same rules as the "Maze", with an added challenge. The agent can longer use same-colored key open door. Instead, it must learn the "key-door" correspondence shown in the hint bar (see (i) in Figure D. such as "use the blue key to open the yellow door". This task evaluates the MLLM's Learning and Planning abilities, requiring the agent to leverage the hint information to make correct choices and formulate a series of plans to obtain the diamond as few steps as possible.
Memory Maze (MMA): This task follows the same rules as the "Maze", with an added challenge. Before the task begins, the agent is shown the location of the diamond, but once the task starts, the diamond in the scene will be hidden and several treasure chests will appear (see (i) in Figure [}. To succeed, the agent must correctly open the chest containing the diamond. This task primarily as- sesses the MLLM's Memory and Planning abilities, as the agent must recall the diamond's s location and devise an effective strategy to retrieve
Memory Filling (MFI): This task follows the same rules as "Filling", with an added challenge. The agent must additionally remember the target, which will disappear once the task starts (see (k) in Figure D. This task primarily evaluates the MLLM's abilities in Perception Reasoning and Memory, as it necessitates recognizing the overall image and recalling specific details to identify the correct piece.
Memory Decode (MDE); In MDE task, the agent is provided with a hint bar, which contains a certain number of association rules between different items (see (I) in Figure I and it must remem- ber the item relationships because these will be hidden once the task starts. This task evaluates the MLLM's abilities in Memory and Learning. as it requires the agent to retain and utilize the information from the hint bar to make accurate selections.
EXPERIMENTS
6.1 EXPERIMENTAL SETUP
We evaluated of MLLMs on KIDGYM, covering both closed-source and open- source models. The closed-source models are: o3 (OpenAl 2025a), GPT-5 (OpenAl 20256).

---

Published as a conference paper at ICLR 2026
GPT-40 OpenAA 2024), Gemini-2..-Prr Pro-Pro (DeedMind 2025).
Gemini-2.5-Flan -lashh (DeepMind 2025 Slaude3.7-Sommee (Anthropic ).25). while the open-source models are DeepSeckVL-2 Team
2024). VL-25 (Bai et al. 2025), and InternVL-3 (AlLab 2025). For comparison, we also
provide human and random baselines (see Appendix
6.2 EXPERIMENTAL. METRICS
We evaluate closed-source models via their official APIs and open-source models using NVIDIA RTX A6000 GPUs. For each task, we ran 100 zero-shot rounds and evaluated every model on the identical set (see Appendix [B66 detailed evaluation procedure). We also tested chain-of-thought (CoT) and in-context learning (ICL) methods on part of the tasks and models, with results presented in Appendix
6.3 EXPERIMENTAL RESULTS
In this section, we compare the performance of MLLMs on KIDGYM, presented in Table The performance here is measured by the success rate under the ground-truth optimal solution.
From the results, the overall performance of closed-source MILMs is significantly higher than that of open-source ones and most models perform better in tasks that examined a single ability than in tasks that involved composite abilities. Notably, three of the currently most powerful closed-source models (o3, GPT5 and Gemini-2.5-Pro) are able to achieve near-perfect scores on a few specific tasks, such as CL, SE and MDE. However, from difficulty perspective, success rates generally decrease from L1 to L3, validating the effectiveness of our task taxonomy. Through quantitative analysis, we identified 3 main challenges of current MLLMs.
Challenges in Reasoning over Non-Semantic Visual Information. Both Fl and PU tasks require the model reassemble pieces match target image. In Fl task, the target image contains rec- ognizable and nameable objects (e.g. animals). In PU task, however, the model must reconstruct an arbitrary shape made of random blocks. The highest success rate for the FI-LI task is 0.83 (o3), while the highest success rate for the PU-LI task is only 0.30 (GPT-5), which is merely 5 percentage points higher than the random success rate, Across all models, performance of PU task is consistently worse than of FI task, suggesting that frontier MLLMs still struggle with abstract, non-semantic images.
Challenges in Identifying the Quantity of Items. The goal of the CO task is to collect a specific number of items in the scene, which very easy for humans (the human success rates for all three difficulty levels is 1 1.00). However, the CO task appears to present significant failure for current MLLMs, even the best model, Gemini 2.5-Pro, achieves only a 0.72 success rate on the easiest level (LI). In most failure cases, the model conflates a small cluster of items (typically two or three) and identifies them as a single object. Furthermore, we conducted a small-scale experiment and found that increasing image resolution improved several models' accuracy humans can complete the task without enhanced image clarity. This suggests that current MILMs insufficiently sensitive to quantitative information and tend to rely high-resolution visual cues rather than robust numerosity representations. Detailed results are provided in the Appendix B.]
the CO task. In contrast,
Challenges in Dealing with Composite Tasks. Compared with tasks that call for single capability, success rate drops markedly on several tasks that demand a combination of abilities. For example, MMA/MFI tasks extend the original MA/FI tasks by adding memory requirement. The success rates of all the models in MMA/MFI tasks are significantly lower than that of the MA/FI tasks, which assesses single capability. Therefore, for MLLMs, it remains a challenge to process multiple types of information at once or to take into account interrelated rules simultaneously.
Reasoning Method has Significant Impact on Different Tasks. By comparing 3 methods (zero- shot, CoT, ICL), we observe significant differences in their success rates across various tasks. For in- stance, when using COT, the Gemini-2.5-Flash model demonstrates remarkable improvements over zero-shot. However, for o3, which inherently integrates the CoT, no substantial difference is ob- served between zero-shot and COT. In the case of ICL, where scene and item layouts are randomly generated, its performance may even be inferior to zero-shot in certain tasks that emphasize mem- ory and learning. This could be attributed to the model's tendency to overemphasize examples, potentially neglecting the dynamic changes within the scene.
8

---

Published as a conference paper at ICLR 2026
Table 2: Zero-shot performance comparison of MILMS across 12 KIDGYM tasks. "I." denotes the task level. Performance is measured by the success rate over 100 rounds under the ground-truth optimal solution, rounded to decimal places.
Methods
I
CL
SE
SO
MA
FI
PU
PL
CO
DMA
MMA
MDE
MFT
Closed-Source Models (API)
1,00
1,00
0.97
0.87
0.83
0.26
1.00
0.30
0.90
0.44
1.00
0.81
o3
0.98
0.95
0.42
0.52
0.11
0.98
0.26
0.47
0.18
1.00
0.50
3
0.92
1.00
0.97
0.27
0.30
0.06
0.71
0.13
0.17
0.05
1.00
0.37
I
1,00
1,00
1,00
0.97
0.74
0.30
1.00
0.36
0.95
0.62
1.00
0.77
GPT-5
2
0.96
1.00
0.99
0.43
0.60
0.06
1.00
0.18
0.47
0.10
1.00
0.61
3
0.92
0.99
0.94
0.11
0.41
0.01
0.88
0.16
0.24
0.01
1.00
0.40
0.46
1.00
0.48
0.33
0.66
0.26
0.71
0.00
0.58
0.00
0.95
0.64
GPT-4o
2
0.24
0.76
0.31
0.19
0.28
0.09
0.37
0.00
0.14
0.00
0.98
0.26
3
0.13
0.50
0.08
0.00
0.15
0.03
0.20
0.01
0.00
0.00
1.00
0.18
1
0.99
1.00
0.99
0.95
0.81
0.19
1.00
0.72
0.93
0.66
1.00
0.81
Gemini-2.5 2.5
2
1.00
1.00
0.99
0.18
0.66
0.13
1.00
0.36
0.24
0.49
1.00
0.66
Pro
3
1.00
1.00
0.93
0.03
0.36
0.07
0.74
0.19
0.16
0.00
1.00
0.36
0.83
1,00
0.69
0.86
0.64
0.19
1.00
0.30
0.81
0.19
1.00
0.75
Gemini-2.5
0.71
0.79
0.38
0.15
0.24
0.05
1.00
0.10
0.13
0.00
1.00
0.15
Flash
3
0.50
0.53
0.21
0.01
0.22
0.03
0.58
0.06
0.05
0.01
1.00
0.11
1
0.98
0.97
0.85
0.64
0.57
0.22
0.97
0.54
0.60
0.00
0.98
0.43
Claude-3.7
0.92
0.68
0.71
0.15
0.32
0.14
0.63
0.33
0.04
0.00
0.98
0.23
Sonnet
3
0.81
0.51
0.37
0.05
0.18
0.01
0.44
0.27
0.01
0.00
0.99
0.10
Open-Souree Models (Large)
0.48
0.98
0.68
0.42
0.41
0.29
0.62
0.00
0.65
0.12
0.96
0.38
QwenVL-2.5
0.29
0.28
0.18
0.24
0.08
0.27
0.00
0.17
0.00
0.93
0.18
(72B)
3
0.01
0.65
0.09
0.03
0.09
0.04
0.15
0.00
0.00
0.00
0.92
0.08
1
0.43
0.99
0.59
0.47
0.48
0.26
0.63
0.01
0,48
0.00
0.91
0.41
Vn-
0.20
0.48
0.29
0.03
0.22
0.10
0.15
0.01
0.11
0.00
0.83
0.13
(78B)
3
0.05
0.17
0.09
0.01
0.08
0,06
0,09
0.02
0.00
0.00
0.75
0.05
Open-Soue Source Models
(Middle)
0.64
1.00
0.62
0.91
0.60
0.29
0.67
0.00
0.68
0.15
0.94
0.61
QwenVL-2.5
2
0.35
0.81
0.44
0.15
0.30
0.04
0.33
0.02
0.09
0.00
0.92
0.19
(32B)
3
0.05
0.52
0.11
0.03
0.09
0.06
0.20
0.00
0.00
0.00
0.94
0.08
1
0.45
0.88
0.46
0.85
0.58
0.28
0.43
0.00
0.38
0.01
0.85
0.57
VL-3
0.22
0.40
0.29
0.35
0.27
0.09
0.25
0.00
0.12
0.00
0.76
0.32
(38B)
3
0.24
0.21
0.17
0.03
0.12
0.03
0.18
0.03
0.02
0.00
0.65
0.08
Open-Son Source Models
(Small)
0.23
0.79
0.42
0.00
0.34
0.24
0.21
0.00
0.24
0.00
0.25
0.31
VwenV
2
0.07
0,31
0.20
0.00
0.16
0.07
0.10
0.00
0.04
0.00
0.20
0.11
(7B)
3
0.01
0.16
0.07
0.00
0.07
0.05
0.11
0.00
0.01
0.00
0.18
0.05
1
0.23
0.41
0.51
0.06
0.19
0.18
0.33
0.00
0.30
0.01
0.39
0.31
Intern InternVL-3 VL-3
0.05
0.11
0.26
0.02
0.13
0.09
0.19
0.01
0.02
0.00
0.33
0.14
(8B)
3
0.02
0.03
0,06
0.00
0.06
0.04
0.09
0.04
0.00
0.00
0.25
0.03
1
0.18
0.47
0.51
0.34
0.33
0.24
0.25
0.12
0.26
0.03
0.34
0.27
DeepSeckVL-2
2
0.06
0.09
0.14
0.01
0.12
0.09
0.12
0.04
0.03
0.07
0.25
0.10
3
0.01
0.03
0.04
0.00
0.04
0.04
0.12
0.06
0.01
0.08
0.17
0.04
Human Baseline
1
0.98
1.00
1.00
1.00
1.00
1.00
0.97
1.00
1.00
1.00
1.00
1.00
Human
0.95
0.98
1.00
1.00
0.95
1.00
0.97
1.00
0.95
3
0.93
1.00
1.00
0.97
1.00
1.00
0.93
1.00
0.95
1.00
0.95
1.00
Random Choose
0.24
0.25
0.25
0.38
0.25
0.25
0.05
0.25
0.50
0.25
0.25
0.15
Random
2
0.04
0.07
0.17
0.16
0.08
0.08
0.00
0.17
0.08
0.08
0.13
0.05
3
0.02
0.02
0.13
0.12
0.04
0.04
0.00
0.13
0.04
0.04
0.13
0.03

---

Published as a conference paper at ICLR 2026
6.4 CAPABILITY RADAR MAP
To provide deeper insights into the capabilities of MLLMs, as discussed in Section 3 we calcu- lated capability scores across dimensions and visualized them through radar map for each MLLM (Figure 3). The specific calculation methodology and formulas are detailed in Appendix D
In KIDGYM, execution a shared prerequisite: every task ultimately requires the model translate its inferred goal into concrete, correct actions. To keep the other capability scores interpretable, we therefore measure execution explicitly only with the Classification (CL.) task. In the remaining tasks, exccution is still unavoidable, but the scoring is designed to emphasize the task's 's target capability rather than re-estimating execution in a more confounded setting. In practice, the CL score provides the clearest reference for execution readiness: models with weaker CL performance are more likely to accumulate action-level errors that depress performance and reduce the reliability of their scores on other capabilities. Notably, results suggest that closed-source source models generally exhibit strong ecution, whereas open-source models tend to lag, which can in turn degrade both their performance and the trustworthiness of their evaluations on other tasks.
Overall, closed-source models perform relatively well in learning and memory capabilities, yet a substantial gap compared with human performance remains. For open-source MLLMs, the perfor- mance of models in the same category generally improves as the number of parameters increases. However, there remains significant gap between open-source and closed-source MLLMs, with o3, GPT-5 and Gemini-2.55 Pro standing out as particularly remarkable, dominating across all measured dimensions. As shown the capability radar map, evaluated MILMs generally score lower in perception reasoning and planning capabilities. While these models have progressed beyond basic recognition tasks, they still struggle with more complex forms of visual cognition, particularly ab- stract and sem-eemattic ones. Similarly, the planning dimension requires further development to enable models to systematically organize tasks, predict the consequences of actions, and implement multi-step strategies for solving complex and composite problems.
Feecution
Flncution
Owe 2-22
Ove W.-25-3300
C775
Perception Resdoning
Memory
Perception Reagoning
Welmory
Planning
Learn
Banningg
Leaming
Figure 3: Five-dimensional capability radar chart. The chart on the left shows the capability scores of the dlosed-souree models, while the chart on the right shows those of the spen-source models.
7 CONCLUSION
In this work, we propose an assessment framework for MLLMs, incorporating five core capabilities based on the Wechsler Intelligence Scale. And we introduce KidGym, a comprehensive 2D grid- based benchmark for evaluating these capabilities of MLLMs. Experiments indicate that although some closed-source MLLMs can achieve a relatively high success rate on some simple tasks, they still exhibit obvious deficiencies in those compound tasks that require multiple capabilities. Specifi- cally, more effort needs to be devoted to further enhancing models' abilities to handle non-semantic and quantitative visual information. Although the current number of tasks is limited, we believe that our open- open-source and scalable framework, KidGym, offers vast potential for the MLLM community and will drive further adyancements in the field of AGI.
10

---

Published as a conference paper at ICLR 2026
8 ACKNOWLEDGEMENT
This work is supported by Shanghai Sailing Program (23YF1427600),
REFERENCES
AlLab. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025. URL Ut//htttsss//aarii.oo//////////s .002250444409999 10479
Niki Amini-Naieni, Tengda Han, and Andrew Zissemaan. Countgd: Multi-modal open- world count-
ing, 2024. /ttp///ttee/aaaa...oo///////...0
NeurIPS 2024.
Anthropic. Claude 3.7 sonnet system 2025. m/ http:///eeeeeetrooocaaaaaaaaa/aa
2025.
Richard C Atkinson and Richard M Shiffrin. Human memory: A proposed system and its control processes. In Psychology of learning and motivation, volume 2, pp. 89-195. Elsevier, 1968.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng. Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. URL https: /ttp///axvvvv... org/ abs 2as./25522 2502. 13923
Emily M. Bender and Alexander Koller. Climbing towards NLU: On meaning, form, and under- standing in the age of data. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.
Online, July 2020. Association for Computational Linguistics. doi: 10. 18633vvvl 2020 Dacaacc--maii4440 URL
org/20gg///00/ acl -mainn 4 463
Marcel Binz and Eric Schulz. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120(6), February 2023. ISSN 1091-6490. doi: 10. 1073//aas.... 2218523120. URL http:///t://x..... httppp://ae..dl..org110111111100
221 2218523120
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, et al. Openai gym, 2016. URL https://arxiv: wwtg:/ogg//aooeoooooo abw. 01540
T.B. Brown, Benjamin Mann, Nick Ryder, et al. Language models are few-shot learners. arXiv: Computation and Language,arXii: Computation and Language, May 2020.
Weitong Cai, Jiabo Huang, Shaogang Gong, Hailin Jin, and Yang Liu, Milm video narrator: Mitigating modality imbalance in video moment retrieval, 2024.
abs / /24/2246..2 7880
Xu Cao, Yifan Shen, Bolin Lai, Wenqian Ye, Yunsheng Ma, Joerg Heintz, Jintai Chen, Meihuan Huang, Jianguo Cao, Aidong Zhang, and James M. Rehg. What is the visual cognition gap between humans and multimodal 1lms?, 2024. URL
10424 COLM 2025.
Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu. beeplan- Benchmarking multimodal large language models for human-level planning, 2024. URL
Maxime Chissei Bolun Dai, Mark Towers, Rodrigo Perez-Vicent Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworld: Mod- ular & customizable reinforcement learning environments for goal-oriented tasks. Advances in Neural Information Processing Systems, 36.73383-73394, 2023.
Shih-Han Chou, Shivam Chandhok, James J. Little, and Leonid Sigal. Mm-r3: On (in connsisssyy of multi-modal large language models (mllms), 2024. URL https: // /awxivivivv.. org/ abs/ 2410. 04778
11

---

Published as a conference paper at ICLR 2026
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling cnaruction-finetune lan- guage models. Journal of Machine Learning Research, 25(70):1-53, 2024.
Karl Cobbe, Christopher Hesse, Jacob Hilton, and John Schulman. Leveraging procedural gen- eration to benchmark reinforcement learning. CoRR, abs/1912.01588, 2019. URL http:
///axii..rr//aa//111.... 01588
Anthony Costarelli, Mat Allen, Roman Hauksson, Grace Sodunke, Suhas Hariharan, Carlson Cheng, Wenjie Li, Joshua Clymer, and Arjun Yadav. Gamebench: Evaluating strategic reasoning abilities of Ilm agents, 2024. Relhttps: : //tpx//aavvv.v//aa//aa
abs wwtp.//////bb//..eeee/e/ 06613
DeedMind. Gemini-2.5-pro system card, 2025. /tt/ www.hrdowww.thoooggg....oo
com/ atom///assstts/boouueettttt//s
ni 11-2..5-rra 5
Google DeepMind.
Gemini 2.5 flash
model card, 2025.
URL
https:
Adele Diamond. Executive functions. Annual Review of Psychology, 64(Volume 64, 2013):135- 168. 2013. ISSN 1545-2085.
doi: www..e/ww...orr.ccottttt////rrrr
13311--43750.
URL https: //::::/wwww wwwp///wwww/.. annua revi www.eeeeeereeeooeeeeeoee
1146/
snnurev-psycoo 11:0h-11331111 143750
Changde Du, Kaicheng Fu, Bincheng Wen, et al. Human-like pbhjeccccooccpllpppeeessttttnnno naturally in multimodal large language models, 2024. URL /rTps://arxlv.org/aa3.
2407 01067
Xinyu Fang. Zhijian Chen, Kai Lan, Lixin Ma, Shengyuan Ding, Yingji Liang. Xiangyu Zhao, Farong Wen, Zicheng Zhang, Guofeng Zhang, Haodong Duan, Kai Chen, and Dahua Lin. Creation-mmbenchh Assessing context-aware creative intelligence in mllm, 2025. URL https:
//arxiv.org/abs//2/ wbw./or////bb/./o..ooo..../..a.a
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models, 2024. URL https:://axxi..rgg////
Fu.
2306. (506.13394)
Chuang Gan, Siyuan Zhou, Jeremy Schwartz, Seth Alter, Abhishek Bhandwaldar, Dan Gutfreund, Daniel L. K. Yamins, James J DiCarlo, Josh McDermott, Antonio Torralba, and Joshua B. Tenen- baum. The threedworld transport challenge: A visually guided tast-and-motion planning bench- mark for physically realistic embodied ai, 2021. URL htttps://arxii.. wwwp:/iiioorrooooooo attpgg//abb///aa// 14025
Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yumhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, and Rongrong Ji. Cantor: Inspiring multimodal tf-inuhf- of mllm, 2024. htRs:tts::/ putttps://axxiv arxiv. http//a////....100000
Manu Gaur, Darshan Singh S. and Makarand Tapaswi. Detect, describe, discriminate: Moving beyond vqa for milm evaluation, 2024. URL hta.a//ps///axx..oo///////////// 15122
Cleotilde Gonzalez. Decision support for real-time, dynamic decision-mmkinn tasks. Organizational Behavior and Human Decision Processes, 96(2):142-154 2005. ISSN 0749-5978. doi:
http://dbb.og//110.016oo..obpppp 1.002 1.002. (ttttt::: htw///www/s. www.sciinnce www.scciencedieec.. com/ scienee/aarLieeee/1///8444533333
A. Gopnik, A.N. Meltzoff, and P.K. Kuhl. The Scientist in the Crib: What Early Learning Tells
Us About the Mind. HarperCollins, 2009. ISBN 9780061846915. URL https: / /bookk..
google. com. wtw.kkkkokkioooooooo (tt.:o/s222.---522222222222 6RA22MWCOYC
W. H. Guertin, C. E. Ladd, G. Frank, A. I. Rabin, and D. S. Hiester. Research with the wechsler intelligence scales for adults. Psychological Bulletin, 66:385-4PP 1966. doi: 10.10///00020410.
12

---

Published as a conference paper at ICLR 2026
Jiayi Gui, Yiming Liu, Jiale Cheng, Xiaotao Gu, Xiao Liu, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. Logicgame: Benchmarking based reasoning abilities of large language models, 2024.
http://xxivviii..... / 15778
William H. Guss, Brandon Houghton, Nicholay Topin, Phillip Wang, Cayden Codel, Manuela Veloso, and Ruslan Salakhutdinov. Minerl: large-scale dataset of minecraft demonstrations, 2019. URL https: htta::/aaxxvv.oom wtw.xiv.org//aaa. abs 9070/99007700 13440
Danijar Hafner. Benchmarking the spectrum of agent capabilities. CoRR, abs/2109.06780, 2021.
URL https://arxiv..rrt http/abs/2210 06780
Jiaming Han, Kaixiong Gong, Yiyuan Zhang. Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, and Xiangyu Yue. Onellm: One framework to align all modalities with language, 2025. URL https: arxia. www....ogg/////////222222222 http//abb///..e.ee// 03700
Xiaotian Han, Quanzeng You, Yongfei Liu, Wentao Chen, Huangjie Zheng. Khalil Mrini, Xudong Lin, Yiqi Wang, Bohan Zhai, Jianbo Yuan, Heng Wang, and Hongxia Yang. Infimm-eval: Complex open-ended reasoning evaluation for multi-modal large language models, 2023. URL http:///aaxxii...aaaaao///se
11567
Jiaxing Huang and Jingyi Zhang. A survey on evaluation of multimodal large language models, 2024. btLpt /ttp://aaivvv....//ia/eeeeeaweva org abs
wttp://.rraaab////00000 1470991.99999 15769
Yukang Huo and Hao Tang. When continue learning meets multimodal large language model: A survey, 2025. URL http::tts///axiioorrr wttp:/iww..rr//////53333.3 01887
Ilija Ilievski and Jiashi Feng. Multimodal learning and reasoning for visual question an- swering. Guyon, U. Von Luxburg, S. Bengio, et al. (eds.), Advances Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL
Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. The malmo platform for arti- ficial intelligence experimentatiion. In International Joint Conference on Artificial Intelligence, 2016. URL UtLLttts:://ppp wwtp://icaaaaannnooaooooaa wwwp:/lla...rr//oooooooo ort.ooouu.rrrrrrroisII.. : 9953039
Arthur Juliani, Ahmed Khalifa, Vincent-Pierre Berges, Jonathan Harper, Ervin Teng, Hunter Henry, Adam Crespi, Julian Togelius, and Danny Lange. Obstacle tower: A generalization challenge in vision, control, and
2019.
Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. Fact, fetch, and reason: A unified evaluation of retrieval- augmented generation, 2025. URL /ttl::tt////////..rrrrrg
wtt.://///////...1///// 12941
Hari Chandana Kuchibhotla, Abbavaram Gowtham Reddy, Sai Srinivas Kancheti, and Vineeth N. Balasubramanian. Fine-grained visual recognition in the age of multimodal L.MMs. Adaptive Foundation Models: Evolving Al for Personalized and Efficient Learning, 2024. URL. https:
//openreview. net for um? id- 1e--ii2pphHeBB..
Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. Build- ing machines that learn and think like people. CoRR, abs/1604.00289, 2016. URL http: //
00289
Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed- bench-2: Benchmarking multimodal large language models, 2023. URL https:///rxxi:
org/aba/24 2obl/2311..0 17092
Jian Li, Weiheng Lu, Hao Fei, Meng Luo, Ming Dai, Min Xia, Yizhang Jin, Zhenye Gan, Ding Qi, Chaoyou Fu, Ying Tai, Wankou Yang, Yabiao Wang, and Chengjie Wang. A survey benchmarks multimodal large language models, 2024. URL https:: //as:vv///.vvv../ org/abs/
2408. 08632
13

---

Published as a conference paper at ICLR 2026
Chang Liu, Changhu Wang, Fuchun Sun, and Yong Rui. Image2text: A multimodal image captioner. In Proceedings of the 24th ACM International Conference on Multimedia, MM '16. pp. 746-748, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 978 178343033 doi: 10. 10.1145/2964284227793 2973833.. URL http::t:: /doii. org/10.
.2..88....98... 29938311
Akira Miyake, Naomi P. Friedman, Michael J. Emerson, Alexander H. Witzki, Amy Howerter, and Tor D. Wager. The unity and diversity of executive functions and their contributions to complex "frontal lobe" tasks: A latent variable analysis. Cognitive Psychology, 41(1):49-100, 2000. ISSN 0010-0285. doi: https://bb..ogg//////////ooogg/
19tp:////coopp999077744 URL http:://twwww
sciencedirect. com/ htttt///s/..////t.tt../////////0
OpenAl. Gpt-4o system card, 2024.
URL
com/
gpt 40 system card.pd.
OpenAl.
Openai o3 and 04-mini system card, 2025a
URL https://
cdn wtteenai.coomm/f/221333333 03 23-anG-o4--ibl--yyttemmmmmo
OpenAl. Openai gpt-5 system card, 2025b. URL http::::/sxx....pp..//o
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models fol- low instructions with human feedback. Advances in neural information processing systems, 35: 17730-27744 2022.
Rommel Salas-Guerra Cognitive ai framework: advances in the simulation of human thought, 2025.
URL httpt://azxxxi wtt.:oggg/.ab.....o. /tEp:://aaaav.orrr/aa///55. 04259
Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, et al. Minihack the planet: A sandbox for open- ended reinforcement Jearnon research. CoRR, abs/2109. 13202, 2021. URL httppttss://axiiit
http//bbb2219..3222
John Searle. Intentionality: an essay in the philosophy of mind. Cambridge University Press, 1983.
S P Sharan, Francesco Pittaluga, Vijay Kumar B G, and Manmohan Chandraker. Llm-assist: En- hancing closed-loop planning with language-based reasoning. 2023. URL https://aaxxvv org/ htg//ba/220 aba//4aa1120 2401. 00125
Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng. Lidong Bing, and Roy Ka-Wei Lee. Math-Ilava: Bootstrapping mathematical reasoning for multimodal large language models, 2024.
http://bb/2244... 17294
Linda Smith and Michael Gasser. The development of embodied cognition: Six lessons from babies. Artificial Life, 11(1-2):13-29, 2005. doi: 10.1162/106454465227777
Dingjie Song, Shunian Chen, Guiming Hardy Chen, Fei Yu, Xiang Wan, and Benyou Wang.
Milebench: Benchmarking milms long context, 2024. URL htRphtsss///arxi..rrrg
2404. 18532
Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths. Cognitive archi- tectures for language agents, 2024. https:// org/ abs/ 2309 02427
Yan Tai, Weichen Fan, Zhao Zhang, and Ziwei Liu. Link-context learning for multimodal 1lms. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp- 27 27176-27188 2024.
Wentan Tan, Changxing Ding, Jiayu Jiang, Fei Wang, Yibing Zhan, and Dapeng Tao. Harnessing the power of milms for transferable text-to-image person reid. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 17127-17137, June 2024.
DeepSeek-AN AI Team. Deepseek Ilm: Scaling spen-souree language models with longtermism, 2024. URL htttps://arxiiv wtw./o/g///aaaoaoooooooooo. 2201 02954
14

---

Published as a conference paper at ICLR 2026
Norbert Tihanyi, Tamas Bisztray, Richard A. Dubniezky, Rebeka Toth, Bertalan Borsos, Bilel Cherif, Ridhi Jain, Lajos Muzsai, Mohamed Amine Ferrag, Ryan Marinelli, Lucas C. Cordeiro, Merouane Debbah, Vasileios Mavrocidis, and Audun Jøsang. Dynamic intelligence assess- ment: Benchmarking Ilms on the road to agi with a focus on model confidence. In 2024 IEEE International Conference Big Data (BigData), pp. 3313-3321, 2024. doi: 10.1 BigData62323 22002a06222232202400882253
Oguzhan Topsakal, Colby Jacob Edell, and Jackson Bailey Harper. Evaluating large language mod- els with grid-based game competitions: An extensible Ilm benchmark and leaderboard, 2024.
URL htttss://aaatttt .wwp//i.ooorooooooooo.. org/ abs/2407 07796
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL https:://///... http///axvvv...///ee/eeeeee.aev/ wttp://gp//....o..////// 13971
K. Voudouris, Slater, L. G. Cheke, W. Schellaert, J. Hernández-Orallo, M. Halina, M. Patel, I. Alhas, M. G. Mecattaf, J. Burden, J. Holmes, N. Chaubey, N. Donnelly, and M. Crosby. The animal-ai environment: A virtual laboratory for comparative cognition and artificial intelligence research. Behavior Research Methods, 57(4):107, Feb 2025, doi: 10.3758/s13428 10.37588/13428--2220222
Konstantinos Voudouris, Lucy G. Cheke, and Marta Halina. The future is computational comparatiie cognition. Comparative Cognition &appamp: Behavior Reviews, 19: 19:1005mdashh:110, 2024. ISSN 1911-4745, doi: 10.3819//cb:22244499000000 190009. URL httt::::/dk.diiiorr//11.....8 del..ici.ooo
CCBR. 2024. 190009
Chunwei Wang, Guansong Lu, Junwei Yang, Runhui Huang, Jianhua Han, Lu Hou, Wei Zhang, and Hang Xu. Illume: Illuminating your Ilms to see, draw, and self-enhance, 2024a. URL
https:/ // /arxivv...... otg/abs/////oooooooooo....o..oar abs/ 0tlo:/2112.066.. 06673
Guanqun Wang, Xinyu Wei, Jiaming Liu, Ray Zhang, Yichi Zhang, Kevin Zhang, Maurice Chong, and Shanghang Zhang. Mr-mllm: Mutual reinforcement of multimodal comprehension and vision perception, 2024b. URL
15763
Han Wang, Yanjie Wang, Yongjie Ye, Yuxiang Nie, and Can Huang. Elysium: Exploring object-level perception in videos via mllm, 2024c. URL httpp::/aax...orgg//aaaaa/
16558
Hengyi Wang, Haizhou Shi, Shiwei Tan, Weiyi Qin, Wenyuan Wang, Tunyu Zhang, Akshay Nambi, Tanuja Ganu, and Hao Wang. Multimodal needle in a haystack: Benchmarking long-context capability of multimodal large language models, 2024d. URL wwt.hhtss//rssii.or////o///
2446611230
Zefeng Wang, Zhen Han, Shuo Chen, Fan Xue, Zifeng Ding, Xun Xiao, Volker Tresp. Philip Torr, and Jindong Gu. Stop reasoning! when multimodal Ilm with chain-of-thought reasoning meets adversarial image, 2024e. URL
14899
Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. Genartist: Multimodal Ilm agent for unified image generation and editing, 2024f. URL https:://aax....rgg/////// wtt.:/orrrabb/////0//////////
05600
John O. Willis, Ron Dumont, and Alan S. Kaufman. Wechsler Intelligence Scales (WAIS, WISC, WPPSI), pp. 1-8. John Wiley Sons, Ltd, 2015. ISBN 9781118625392. doi: https://oii.o orrg// 10. 001000/97811182252222ebepppp abs/10.1002/910 alc/10010002/7781112228333 htw.://066222222.we0000663 wbecp405
hwt.atpsss//onnnnellaaaaaa wiley. com/ doi/
Siyu Wu, Alessandro Oltramari, Jonathan Francis, C. Lee Giles, and Frank E. Ritter. Cognitive 1mss: Towards integrating cognitive architectures and large language models for manufacturing decision-making, 2024a. URL org. abs 2408 09176
Yue Wu, Xuan Tang, Tom M. Mitchell, and Yuanzhi Li. Smartplay: A benchmark for 1ms as intelligent agents, 2024b. URL htt.:ht.ss//aaxvv.or//////////// 01557
15

---

Published as a conference paper at ICLR 2026
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang. Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie. Yuxiang You, Kai Dong, Xingkai Yu, Haowei 7hang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseek-v12: Mixture-of- experts vision-language models for advanced multimodal understanding, 2024c. https:
//axxiv. wwtp://pgw.b.......... 10302
Tong Xiao, Xin Xu, Zhenya Huang. Hongyu Gao, Quan Liu, Qi Liu, and Enhong Chen. Perception- rl: Advancing multimodal reasoning capabilities of milms via visual perception reward, 2025. URL
org. a55/ 07218
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal Ilm logical reasoning benchmark in visual contexts, 2024. URL htt.hhttp////c http:::/aaxvvvvvo.a wws/abs////////ooo.///..a///..ae http://bs/24000 /4077 240..049773 04973
Peng Xu, Wenqi Shao, Kaipeng Zhang, Peng Gao, Shuo Liu, Meng Lei, Fanqing Meng, Siyuan Huang, Yu Qiao, and Ping Luo. Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models, 2023. URLpttss://axxi http://awxx...ogg////////0/ 09265
Xinrun Xu, Yuxin Wang. Chaoyi Xu, Ziluo Ding, Jiechuan Jiang, Zhiming Ding, and Börje F. Karls- son. survey on game playing agents and large models: Methods, applications, and challenges, 2024.
abs 2203.102444
Zhenfei Yin, Jiong Wang. Jianjian Cao, Zhelun Shi, Dingning Liu, Mukai Li, Lu Sheng, Lei Bai, Xiaoshui Huang, Zhiyong Wang, Jing Shao, and Wanli Ouyang. Lamm: sanguage-assister multi- modal instruction-tuning dataset, framework, and benchmark, 2023. URL bttps://arx: org/ 2055/bb//50 /bs//306. 2306. 06687
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities, 2023.
https: //t///aaxvvvi/ org/ abs /tadb///3300 2308 02490
Virgil Zeigler-Hill, Todd K Shackelford, Emily J Hangen, and Andrew Elliot. Encyclopedia of personality and individual differences. In Encyclopedia of personality and individual differences. Springer Cham, 2020.
Chunhui Zhang, Yiren Jian, Zhongyu Ouyang, and Soroush Vosoughi. Working memory identi- fies reasoning limits in language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.
Miami, Florida, USA, November 2024. Association for Computa- tional Linguistics. doi: 10.18653/v1/2022
emnlp wapp-mmii.999. 938. URL /tapsttts///aclltthologog
org/202. emn ep- 9388
Yupeng Zheng. Zebin Xing, Qichao Zhang, Bu Jin, Pengfei Li, Yuhang Zheng, Zhongpu Xia, Kun Zhan, Xianpeng Lang, Yaran Chen, and Dongbin Zhao Planagent: A multi-moda large lan- guage agent for closed-loop vehicle motion planning, 2024. URL
abs/2406.0155 01587
Tianyang Zhong, Zhengliang Liu, Yi Pan, et al. Evaluation of openai ol: Opportunities and chal- lenges of agi, 2024. /RLpattss://arxiv wttp:/wwwwww////////ee/////
Jianjun Zhu, Lawrence Weiss, Aurelio Prifitera, and Diane Coalson. The wechsler intelligence scales for children and adults. Comprehensive handbook of psychological assessment, 1:51-75, 2004.
16


Reviews:
[{"review_id": "DsLdJF9Bal", "reviewer": "Reviewer_dWpT", "summary": "This paper proposes a new benchmark, KidGym, aimed at evaluating the cognitive capabilities of Multimodal Large Language Models (MLLMs). The inspiration comes from the Wechsler Intelligence Scale for Children. KidGym includes 12 2D grid-based tasks covering five core capabilities: execution, perception reasoning, memory, learning, and planning (Section 3). These tasks are divided into single-capability and composite-capability types, with three difficulty levels (L1–L3), and each round features randomized environments. KidGym is built on the Gym API and supports full customization. The authors evaluate 7 representative SOTA MLLMs (such as GPT-4o, o3, Gemini, QwenVL, InternVL, etc.) and point out that current models still have significant shortcomings in quantity perception, abstract graphic reasoning, and multi-rule composite tasks.", "questions": "Here are some concerns I would like to raise. Thanks.\n\n1. Has there been a deeper analysis of the error types made by the models? Besides success rate, were failed cases (e.g., recognition errors, logical confusion, memory lapses) analyzed? This information is crucial for understanding model behavior. Although Section 6.3 mentions that CoT improves Gemini's performance, is there a systematic analysis of the strengths of the three reasoning paradigms (Zero-shot, CoT, ICL) for each capability or task type?\n\n2. In the tasks, which plays a dominant role: visual or language information? Especially in tasks like Sorting or Decode, is the model's success primarily dependent on language understanding (rules, prompts), or on the parsing of visual images?\n\n3. How much does randomized layout affect evaluation results? In Section 4, it is mentioned that each round has randomized layouts. Has the variance in task results been evaluated to verify the robustness of the benchmark?\n\n4. Does model scale lead to linear performance improvement? Is there any observation that larger models perform better across the five capability dimensions? Is there a saturation point for capability scaling?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors align the task design with the Wechsler intelligence testing system and the concept of Executive Function from child cognitive psychology (Appendix A.1, Section 3), providing a structurally grounded framework with human cognitive analogy, in line with the current trend in AI evaluation toward psychological testing methods.\n\n2. KidGym evaluates five core capabilities, with 12 tasks of varying difficulty, and distinguishes between single and composite abilities (Section 5), making it more systematic than existing MLLM benchmarks. For example, Table 1 clearly shows KidGym’s advantages in capability dimensions compared to other benchmarks.\n\n3. The benchmark is built using the Gym API, allowing users to create new task scenarios and rules, making it highly extensible.\n\n4. A systematic comparison was conducted on 7 mainstream MLLMs, including both closed-source models (such as o3, Claude, Gemini, etc.) and open-source models (such as QwenVL, InternVL, etc.), and multiple evaluation paradigms were introduced.", "weaknesses": "1. Limited novelty of experimental findings: The core findings of the paper—that MLLMs perform poorly in abstract reasoning, precise counting, and multi-step planning—largely confirm known conclusions from other established benchmarks. For example, the difficulty of abstract visual reasoning has already been well documented in benchmarks like ARC-AGI-2 [1]; similarly, the poor performance in the Counting (CO) task is similar to that found in studies [2][3]. Therefore, although the KidGym framework is novel, its final experimental conclusions do not offer significant new insights into the limitations of MLLMs, but rather reaffirm them within a new task set.\n\n2. Limited diagnostic clarity of the benchmark: The paper distinguishes between “Single Capacity” and “Composite Capacity” tasks but does not provide a rigorous methodology for this classification. In Section 5.1, several tasks labeled as “single capacity” clearly involve multiple cognitive abilities according to their own descriptions, yet the paper offers no explanation as to why they are mapped to only one capacity in Table 2. For example, the Puzzle (PU) task, described as “assembling pieces to reconstruct an image,” can reasonably be considered to involve planning, but it is exclusively mapped to perception reasoning, without any explanation as to why other components are excluded. This missing explanation may make the capacity mapping appear somewhat arbitrary, forcing readers to guess the authors’ assumptions and weakening the benchmark’s claim to clearly isolate and measure specific cognitive abilities.\n\n3. Superficial connection between psychometric concepts and actual evaluation metrics: The paper claims to be inspired by the Wechsler Intelligence Scale (Appendix A.1), but the correspondence between these concepts and the actual implementation is weak. For example, the “Execution” ability is explicitly linked to the Processing Speed Index (PSI) in WPPSI-IV, which measures the speed and accuracy of completing simple tasks (Appendix A.2). This analogy seems more like a narrative framing rather than a strictly applied scientific principle.\n\nRef：\n[1]\tChollet F, Knoop M, Kamradt G, Landers B, Pinkard H. Arc-agi-2: A new challenge for frontier ai reasoning systems. arXiv preprint arXiv:2505.11831. 2025 May 17.\n[2]\tLi J, Zhang X, Zou H, Guo Y, Xu R, Liu Y, Zhu C, He Y, Cui P. COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts. InProceedings of the Computer Vision and Pattern Recognition Conference 2025 (pp. 9186-9198).\n[3]\tAmini-Naieni N, Han T, Zisserman A. Countgd: Multi-modal open-world counting. Advances in Neural Information Processing Systems. 2024 Dec 16;37:48810-37.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lwVldowMld", "reviewer": "Reviewer_nziT", "summary": "This paper introducedsa 2D MLLM benchmark KidGym, incorporating abilities based on the Wechsler Intelligence Scale. The abilities include execution, perception reasoning, learning, memory, planning.\n\nThe KidGym benchmark consists of 12 unique tasks (e.g., Classification, Maze, Puzzle, Counting) , each with three difficulty levels. These tasks are designed to assess the five capabilities either individually (6 tasks) or in combination (6 tasks). The environments feature diverse semantic scenes and randomized layouts to ensure robustness and prevent memorization. The benchmark is built on the Gym API, making it extensible.\n\nThe authors evaluates seven state-of-the-art MLLMs on KidGym including three closed-source models (o3, GPT-4o , Gemini-2.5-flash, Claude-3.7-sonnet) and four open-sourced models (DeepSeekVL-2, QwenVL-2.5, and InternVL-3).\n\nTheir experiments reveal significant limitations in current models, particularly in: reasoning over non-semantic visual information (e.g., the Puzzle task), identifying the quantity of items (e.g., the Counting task), dealing with composite tasks that require multiple capabilities (e.g., Maze* vs. Maze).", "questions": "See weaknesses for more details.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "The benchmark itself is robust. The inclusion of three difficulty levels (L1, L2, L3) effectively demonstrates performance scaling , and the use of randomized layouts prevents models from succeeding via memorization.\n\nKidGym is designed to be fully user-customizable and extensible, and it is built on the standard Gym API. This, along with its public release, makes it a valuable and practical tool for the MLLM community.", "weaknesses": "“Single-capability” claim unsubstantiated. The paper asserts six tasks isolate individual abilities but provides no validation (e.g., construct ablations/lesioning). In practice these tasks are compound: Maze (labeled Planning) also needs Perception; Classification (Execution) also needs Perception + Learning. Low scores may reflect hidden prerequisites, not the target skill.\n\nWechsler Intelligence Scales inspiration is superficial. Tasks don’t map cleanly to established Wechsler subtests/constructs, and no psychometric evidence (reliability/validity) is given.\n\nRelated work gaps. Important recent capability benchmarks (e.g., MaRs-VQA in COLM 2025) are missing.\n\nLimited headroom amid fast model progress. Frontier models such as GPT-5 and Gemini 2.5 Pro already approach saturation on several tasks, risking short shelf-life of your benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZAYbAJG0q8", "reviewer": "Reviewer_iozr", "summary": "The paper introduces KidGym, a 2D grid-based benchmark for multimodal LLMs (MLLMs), inspired by the Wechsler intelligence scales. It operationalizes five capabilities: execution, perceptual reasoning, learning, memory, and planning, via 12 tasks (6 single-capability and 6 composite), each with three difficulty levels and randomized layouts. Tasks include Classification, Filling, Puzzle, Selection, Decode, Maze, and composite variants like Maze, Decode, Sorting, Filling, Placement, Counting. The environment presents a scene map, hint bar, and backpack with labeled items and exposes high-level actions to avoid low-level control. The authors evaluate seven SOTA MLLMs (closed- and open-source) across 100 zero-shot rounds per task, plus CoT/ICL variants, and provide capability radar plots. Key findings: closed models outperform open models; tasks involving abstract visual patterns, quantities, and composite rules are consistently harder. The benchmark is positioned as customizable (Gym-based) and claims a public release.", "questions": "Please see the weaknesses. If the authors can address my concerns listed in the weaknesses,I'd like to raise my evaluation.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Clear capability decomposition tied to a widely known cognitive framework (Wechsler), with explicit mapping of tasks to abilities. This makes the evaluation scope transparent and multifaceted.\n2. Interactive, dynamic evaluation rather than purely static VQA-style prompts. The environment design (randomized layouts, diverse scenes, identification labels, hint/backpack) is thoughtful and motivated by known pain points (context consistency, low-level action brittleness).\n3. Breadth of models and tasks. Seven representative models, three difficulty levels, and 12 tasks give a reasonably comprehensive picture.\n4. Empirically grounded insights that echo community experience: weaknesses with non-semantic abstract vision (Puzzle), quantity sensitivity (Counting), and multi-rule composition (composites like Maze) are convincingly demonstrated and discussed.\n5. Extensibility claim (Gym API, configurable scenes/difficulty) is attractive for community uptake.", "weaknesses": "1. Metric and objective mismatch for “planning.” Maze/MA* are described as requiring “as few steps as possible” (§5.1, p. 6), but the reported results are success rates only (Table 3); there is no path-length/efficiency metric, nor time-to-solve, which weakens the planning claim. Without cost-sensitive metrics (or success under step budgets), planning is only partially measured.\n2. High-level actions may reduce ecological difficulty. By design, the agent executes macros like “pick up the basketball” rather than navigating. This meaningfully simplifies the control problem and can blur construct validity for execution and planning. An ablation showing how conclusions change under finer-grained actions would strengthen the case.\n3. Interpretation of Wechsler analogy may be overextended. The paper frames results as “children’s intelligence tests pose challenges” (title) and borrows the Wechsler taxonomy, but does not provide normative calibration or validated correspondences to psychometric constructs or score scaling. The framing risks being rhetorical rather than psychometrically grounded.\n4. Presentation quality / editing. There are numerous typos and grammatical issues (“capbilities,” “cusomization,” “commodate,” etc.), and a few imprecisions (e.g., “diamond is hidden among several treasure chests” but no precise termination rules).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "atO2aq7kzX", "reviewer": "Reviewer_fF2z", "summary": "The paper introduces KidGym, a benchmark that assesses five essential capabilities of MLLMs: execution, perception reasoning, learning, memory, and planning. The benchmark is cognitively inspired and uses games as benchmarking tools. The benchmark identifies key limitations of current MLLMs.", "questions": "1. In the abstract, the authors write that the progression “parallels the developmental trajectory from language acquisition to visual perception in children”. Is this true? Does visual perception occur after language acquisition, or do they develop concurrently?\n2. Table 1 is very helpful for illustrating the contributions of KidGym. However, the dimensions “Difficulty Level” and “Use Extensible” seem more trivial than aspects like “dynamic vs. static”, which seems to be the more important contribution of KidGym. I would suggest revisiting this table to better show what makes KidGym an important contribution and what distinguishes it from existing benchmarks for MLLMs.\n3. Based on the task description, the Sorting task sounds like a linguistic task. Why is it particularly helpful for evaluating MLLMs?\n4. Would be helpful to add the human baseline result to Table 3\n5. Table 3 can benefit from a clearer caption. For example, what does L stand for in the header? What’s 1, 2, and 3? What does the scale represent? Is the number accuracy?\n6. The observation that closed source models outperform open source model is less interesting than comparing general model capabilities, model size, and other factors. Why do you think these models perform better?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The benchmark captures current limitations of MLLMs: namely, in reasoning over nonsemantic, abstract visual information; insensitivity to item quantity; composite capacity tasks involving the interaction of multiple rules; and weak performance in perception reasoning and planning tasks.\n2. The tasks are well designed and include extensive ablations of the environment that allow for more robust evaluation of capabilities. The authors have also made the benchmark customizable to accommodate more tasks, which is a good contribution for future evaluations. The benchmark also varies the level of difficulty for each task type in a nicely controlled way. \n3. The authors provide human baselines to compare with MLLLM performance. \n4. The authors also tested CoT and ICL methods as additional ablations. It is interesting that CoT improves performance for some models.", "weaknesses": "1. Minor: Figure one shows the task types in KidGym. It would be helpful to also illustrate how each task type maps to the four capabilities that KidGym assesses\n2. It would be helpful to provide more context for the Wechsler Intelligence Scale. It is unclear where the tasks come from and how they are constructed. \n3. The amount of tasks (despite the ablations of environment and randomness) is quite small compared to other existing benchmarks\n4. The skills required for each task are largely based on intuitions. The mapping from task to skills would benefit from clearer explanations and justifications based on existing literature. \n5. The paper would benefit from more comprehensive error analysis of specific failure cases and whether any patterns can be surfaced from these cases (and providing concrete examples would help the reader get a better sense of the failure modes).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)