UrbanFeel：A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective
Authors: Jun He
Yi Lin
Zilong Huang
Jiacong Yin
Junyan Ye
Yuchuan Zhou
Weijia Li
Xiang Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Benchmark
Urban Change
Urban Perception
Multimodel Large Language Models
Primary area: datasets and benchmarks

Abstract: Urban development impacts over half of the global population, making human-centered understanding of its structural and perceptual changes essential for smart city planning. While Multimodal Large Language Models (MLLMs) have shown remarkable capabilities across various domains, existing benchmarks that explore their performance in urban environments remain limited, lacking systematic exploration of temporal evolution and subjective perception of urban environment that aligns with human perception. To address these limitations, we propose UrbanFeel, a comprehensive benchmark designed to evaluate the performance of MLLMs in urban development understanding and subjective environmental perception. UrbanFeel comprises 14.3K carefully constructed visual questions spanning three cognitively progressive dimensions: Static Scene Perception, Temporal Change Perception, and Subjective Environmental Perception. We collect multi-temporal single-view and panoramic street-view images from 11 representative cities worldwide, and generate high-quality question-answer pairs through a hybrid pipeline of spatial clustering, rule-based generation, model-assisted prompting, and manual annotation. Through extensive evaluation of 20 state-of-the-art MLLMs, we observe that Gemini-2.5 Pro achieves the best overall performance, with its accuracy approaching human expert levels and narrowing the average gap to just 1.5%. Most models perform well on tasks grounded in scene understanding. In particular, some models even surpass human annotators in pixel-level change detection. However, performance drops notably in tasks requiring temporal reasoning over urban development. Additionally, in the subjective perception dimension, several models reach human-level or even higher consistency in evaluating dimension such as beautiful and safety. Our results suggest that MLLMs are demonstrating rudimentary emotion understanding capabilities. The code and dataset of this work will
be released at https://github.com/Hejun0915/UrbanFeel .

Published as a conference paper at ICLR 2026
URBANFEEL: A COMPREHENSIVE BENCHMARK FOR TEMPORAL AND PERCEPTUAL UNDERSTANDING OF CITY SCENES THROUGH HUMAN PERSPECTIVE
Jun Hel, Yi Lin', Zilong Huang', Jiacong Junyan Ye', Yuchuan Zhou',
Weijia Li Xiang Zhanglt
Sun Yat-sen University SIGS, Tsinghua University
ABSTRACT
Urban development impacts half of the global population, making human- centered understanding of its structural and perceptual changes essential for sus- tainable development. While Multimodal Large Language Models (MLLMs) have shown remarkable capabilities across various domains, existing benchmarks that explore their performance in urban environments remain limited, lacking system- atic exploration of temporal evolution and subjective perception of urban envi- ronment that aligns with human perception. To address these limitations, we propose UrbanFeel, a comprehensive benchmark designed to evaluate the per- formance of MILMs urban development understanding and subjective envi- rommental perception. UrbanFeel comprises 14.3K carefully constructed visual questions spanning three cognitively progressive dimensions: Static Scene Per- ception, Temporal Change Understanding. and Subjective Environmental Per- ception. We collect multi-temporal single-view and panoramic street-view im- ages from 11 representative cities worldwide, and generate high-quality question- answer pairs through hybrid pipeline of spatial clustering, rule-based genera- tion, model-assisted prompting, and manual annotation. Through extensive eval- uation of 20 state-of-the-an art MLLMs. we observe that Gemini-2.5 Pro achieves the best overall performance, with its accuracy approaching human expert lev- els and narrowing the average gap to just 1.5%. Most models perform well on tasks grounded in scene understanding. In particular, some models even surpass human annotators in pixel-level change detection. However, performance drops notably in tasks requiring temporal reasoning over urban development. Addi- tionally, in the subjective perception dimension, several models reach human- level or even higher consistency in evaluating dimension such as beautiful and safety. Our results suggest that MLLMs demonstrating rudimentary emotion understanding capabilities. The code and dataset of this work will be released at https://githuut com fun //Ubba//raeeeee
INTRODUCTION
With over half of the global population now living in urban areas (World Bank 2024), understand- ing the dynamics of urban development has become inereasingly critical for designing sustainable governance strategies, guiding urban policy, and promoting human-centre smart cities (Yuan et al.)
2024 Van Etten al., 202T Zhang et al.) 2024b). Compared to satellite imagery, which provides macro-scale, top-down observations, street-view imagery offers fine-grained, street-level perspec- tives that are more aligned with human visual perception (Biljecki Ito 202T Naik et al. [201]] Wang et al. 2025). This unique characteristic enables it to capture subtle environmental changes within cities, making valuable data source for analyzing intra-urban transformation.
Recent research has explored the use of deep learning models in conjunction with street-view im- agery to assess urban development stages (Zhang et al. 2018 Alpherts et al., 2025). visual qual- ity (Ito et al. 2024; Benidir et al. 2025). ) and perceived livability (Dubey et al. 20166 Yang et al.
1 Correspondiig author(s). F-mail(s): {1 (eiwei ii9, 2hangx795) sysu.edu.co edu.cn

---

Published as a conference paper at ICLR 2026
2024 Li al. 2025). However, these approaches face challenges in terms of generalization across modalities and cities. More importantly, they struggle to effectively quantify and interpret human subjective perception
essential component of real-world urban understanding.
The advent Large Language Models (LLMs) and Multimodal LLMs (MLLMs) has introduced new possibilities for tackling these limitations (Zhang et al. 2024a Xuan al. 2025 Ye et al.,
2023). By leveraging massive amounts of multimodal pretraining data, MLLMs exhibit strong ca-
pabilities in spatial reasoning, visual-linguistic alignment, and commonsense inference. Initial
tempts have applied these models to urban imagery tasks, such as vehicle trajectory prediction (Liu al. 2025 Lai et al. 2025) or scene understanding (Yan et al. [024] Feng et al.) 2025b Huang et al. 2025). and several early benchmarks have emerged to evaluate their performance on objective
tasks such as image geolocalization (Zhou et al. 2025) and infrastructure inference (Feng et al. 2025c).
Prior work has largely been confined to static snapshots, focusing on objective recognition tasks such autonomous driving or urban planning, while overlooking the historical dynamics of cities and thus failing to capture trajectories of development, renewal, and transformation. At the same time, physical changes in the built environment perceptions of safety, beauty, and liveliness. However, existing benchmarks rarely examine how these perceptual shifts linked to temporal urban evolution, leaving critical gap in understanding the interaction between physical change and human experience.
renovation or dleca--otten reshape human
To bridge these gaps, present UrbanFeel, novel human-centric benchmark for evaluating MLLMs in the context of urban change perception. UrbanFeel defines 11 tasks across three di- mensions-static scene perception, temporal change understanding, and subjective environmental perception-to assess models' capabilities in recognition, reasoning, and alignment with human perception. Our benchmark emphasizes multi-view integration, temporal-spatin spatial consistency, and perceptual alignment, aiming to push the boundaries of MLLMs toward more human-alignnn ur- ban understanding, help MLLMs provide a reference for continuous monitoring and prediction capabilities in sustainable cities.
Our main contributions are summarized as follows:
We introduce UrbanFeel, multi multi- perspective, multi-dimensional benchmark designed to evaluate MILMs' performance on tasks related to urban development and human percep- tion. UrbanFeel carefully designs 11 subtasks, focusing on evaluating the model's percep- tion and understanding capabilities in three dimensions: Static Scene Perception, Temporal Change Understanding, and Subjective Environmental Perception.
We design scalable and interpretable task-querying framework, incorporating diverse range of evaluation formats including binary classification, multiplecchoie, sorting, and open-ended reasoning. To enhance explainability, we introduced manual annotation based on local visual evidence into the benchmark management process.
We conduct comprehensive evaluation of 20 state-of-the-art MLLMs on UrbanFeel, quan- tifying model differences across task categories and revealing that current models still fall short of human-leve performance in spatial reasoning and subjective perception within urban change scenarios.
RELATED WORK
2.1 URBAN TEMPORAL CHANGE ASSESSMENT FROM STREET- VIEW IMAGERY
With the accelerating pace of global urbanization, cities have undergone profound spatial and envi- ronmental transformations, prompting growing research interest urban evolution (Pandey & Seto 2015 Hatab et al. 2019: Follmann et al.) 2021). In recent years, street-view imagery has emerged as a valuable data source for urban change detection due to its close alignment with human per- spectives (Biljecki & Ito 2021 Ito et al. 2024). For instance, ChangeScore (Naik et al. (017) utilized deep networks to correlate visual changes with socioeconomic variables. Subsequent stud- ies focused quantifying the built environment's 's physical fabric: Street2 measured physical structural shifts via latent space embeddings (Stalder et al., 2024b), while CityPulse constructed semantic label sequences to detect binary environmental changes (Huang al. 2024b). Similarly,
2

---

Published as a conference paper at ICLR 2026
Existing Urban Scene Benchmark
Spatial Discontinuity
Short Time Span
Focus Spatial Understanding
The images from the same area are for sport
The time span betweon Images from the same area short.
Where picture possiblyy taken?
Kilaga
Vancouver
Cordoba
UrbanFeel (Ours)
Spatial Continuity
Long Time Span
The images from the same area show sequential changes
The time span between images from the same area long-
18
20099
2015
Image Number
Percoptual Domain
Please judge whetherthe appears di Beauty man perspective and some Vasal Fact contribuse
The two scenes show the same place, one from 2009 and one from 2032. Please deter mine whether the scene his besome
mealthy. less wealthy, stayed the
city beautiful, because: clean and bidy =#2>> The buildigs diverre-and and
2022
visually appesting..
WaP High degres of greanery..
Figure 1: Comparison with existing urban scene benchmarks. UrbanFeel introduces three key in- novations: (1) spatially continuous street-view data that includes both single-view and panoramic imagery, (2) long-term temporal coverage spanning over 15 years, and (3) novel evaluation di- mension focused on subjective human perception (e.g., safety, beauty), enabling human-centered assessment beyond conventional spatial understanding.
Stalder et al. (Stalder et al., 2024a) quantified urban decay dynamics using detected objects in dif- ferent years as proxy. While establishing street-view time series as high-resolution proxies for physical change, these methods predominantly frame urban evolution as binary classification. Re- cently, Visual Chronicles (Deng et al., (025) employed VLMMs to mine co-occurring visual trends from open-ended queries. However, existing research remains largely confined to describing ob- jective visual elements change within limited regions, rather than systematically evaluating model capabilitics in controlled temporal reasoning tasks across diverse urban contexts.
2.2 SUBJECTIVE PERCEPTION ASSESSMENT OF URBAN ENVIRONMENTS
Parallel to physical change detection, extensive research utilizes street-view imagery and deep learning to quantify subjective urban perceptions and their socioeconomic correlations (He et al. 2023 van Veghel al. 2024, Rui Xu 2024 He al. 20256 a). The foundational Place Pulse project (Dubey et al. 2016) established this paradigm by crowdsourcing pairwise comparisons to train deep learning models for large-scale prediction. Subsequent studies expanded this frontier: Yao et al. (Yao et al. 2019) improved alignment via adversarial learning, while Wei et al. (Wei al.) 2022) and Fan et al. (Fan et al. (023) linked perceptual attributes to planning metrics and socioe- conomic outcomes. Notably, Wang et al. Wang et al. (2025) applied interpretable machine learning to uncover functional zone-dependent nonlinear associations and threshold effects between envi- ronmental features and perception. Crucially, however, these approaches remain confined to static image snapshots and traditional discriminative models. They lack a comprehensive benchmarking framework to evaluate MILMs' capacity capture how human subjective perception evolves dy- namically alongside long-term physical transformations.
2.3 MULTIMODEL LARGE LANGUAGE MODELS
In recent years, Multimodal Large Language Models (MLLMs) such as Qwen (Bai et al. 2023 GPT-4o (OpenAl 2024), and Gemini-2.55 2.5-ii--..-pp-- 5-pro (Comanici et al. 2025) have achieved remarkable progress in image generation (Anonymous) 2025), visual reasoning (Zhang et al. 2025a), and cross- modal alignment (Wu et al. 2024a). Leveraging large-scale pretraining and instruction tuning, these models show strong generalization in open-domain visual understanding (Li et al. 2024b Wu et al.) 2024b), though their performance on Jomain-sspccfii applications remains limited. In urban texts, recent studies explored MLLMs' zero-shot spatial reasoning. UrbanCLIP (Yan et al., (024) aligns imagery with textual semanties via contrastive learning, while UrbanI.Lan La (Feng et al.
3

---

Published as a conference paper at ICLR 2026
2025b) integrates street-view, structured data, and trajectories, achieving strong generalization UBench. Despite these advances, a systematic framework for evaluating MLLMs on subjective ur- ban perception urban change assessment remains lacking. Prior efforts, such as (Zhang et al. 2025b), focus on isolated dimensions like safety, without addressing temporal coherence or per- ceptual consisteney. Overall, existing work emphasizes static reasoning functional classification, overlooking human-centric perceptual responses and their evolution over time.
2.4
MLLM BENCHMARKS IN URBAN SCENE
With Multimodal Large Language Models (MLLMs) advance in image understanding (Ma et al. 2024 and cross-modal reasoning (Huang al.) 2024a), benchmark datasets have evolved accord- ingly. Early benchmarks centered on basic Visual Question Answering (VQA), but such tasks no longer capture the full potential of modern MLLMs. To address this, several expert-level bench- marks have been introduced for domain-specine tasks with greater semantic and spatial complexity, especially in urban contexts. For example, V-IRL (Yang al. (024) focuses on street-view naviga- tion and recognition; CityBench (Feng et al., (025c) targets urban identity and navigation, though with limited task diversity, UrBench (Zhou et a incorporates multi-view imagery from street and remote sensing sources for spatial reasoning. CityLens (Liu et al. (0255 evaluates urban func- tion modeling using socio-economic indicators, and USTBench (Lai et al. 2025) assesses spatial planning via traffic and road network data. Despite these advances in modeling objective urban scenarios, most existing benchmarks are limited to static snapshots in time. They lack a compre- hensive evaluation of models' ability capture the spatiotemporal evolution of urban environments, particularly how physical transformations affect human subjective perception responses.
3 URBANFEEL
3.1 OVERVIEW
We present UrbanFeel, a comprehensive benchmark de- signed to evaluate the capabilities of Multimodal Large Language Models (MLLMs) in both physical understand- ing and subjective perception within the context of urban development. Built upon multi-view and multi-temporal street-view imagery collected from diverse global cities over the past 18 years, UrbanFeel simulates real-world urban evolution by capturing fine-grained visual and per- ceptual changes. As illustrated in Fig. [2] UrbanFeel includes four types of question format-binary judg- ment, multiple-choice, open-ended reasoning, and novel temporal sorting
Static Seana Percaption
Tamporal Change Undkrttading
Subjective Envinenmencl Perceptiun
Queetion
SSR
aEDEE
6UPER
2008
Ancwar Type
CR
MC
7561
THEE
2770
Juigamennt 6016
Singlowicw
5640
A
GP
issages
1020
RD
Imaaee
3100
3
View Type
aralz
2600
in over 14.300 high- quality QA samples, with approximately 11.0 for vali- dation and 3.3 K for testing. Detailed statistics and exam- ples can be found in Appendix A
paso
11420
Question Numher
Image Number
mathi
2V66
Figure 2: Data Statistics of UrbanFeel.
Unlike prior benchmarks that focus primarily object detection or scene classification within urban imagery, as shown in Fig [ UrbanFeel introduces several novel design dimensions. First, it system- atically incorporates both single-view and panoramic street images captured in a certain sequence to evaluate models' ability to capture spatial context across viewpoints. Second, it integrates long- grquences-spanneee
term urban development historical reasoning and temporal ordering. Third, UrbanFeel introduces human-centered affective perception tasks, covering four dimensions: beautiful, safety, wealthy, and lively. Each sample additionally annotated with localized visual evidence, enabling the evaluation of model explainabil- ity and alignment with human perceptual cues. Although real-world urban analysis often involves iterative workflows, UrbanFeel deliberately targets the fundamental atomic perception and reason- ing capabilities that serve as essential prerequisites for such complex decision decision-making. UrbanFeel thus offers a challenging and comprehensive evaluation framework for MLLMs in complex urban scenarios, laying a foundation for future studies on modeling the alignment and divergence between machine and human perception.
more than decade-to to support tasks that require
4

---

Published as a conference paper at ICLR 2026
2008
2017
Global Perception(GP)
007
Baxry
Roponing the
red righnn
Wealth
L.nig degine
Uetermine whhether are from The same location.
Temporal Co location RecognitionsTTn
ccerrigl..
T Serui Location
Lidb
3TM gound dean.
undiitruttdd
Panorama
Change Recognition(SCR) besc
Local Porc
Single-View
change betweer the two time porlods?
Comparative Perceptual Amalysis (CP)
Single to Pano Natchinng(PPN)
Road Change)
aefon the locole?
Qnd
Bu king Facicc Ourgg
Wogesttonn ehango
Piiel Change Recognition(PCR)
Whichurban element changed the most in terms city develpmennt
from
# +
200 significntt
Locatise
Venide
Sty
Roaut
Waslthy
Salley
Lindl
Co-loocation Recognition((R)
Budding Proseace Change
Vegesziin @ Bulding
Future Scone Identification(99)
о Wetter V and
Liely
drse?
Global Perception(
Reasoniag
Reauty
Dominant
DrDraction(DEE)
regie:
Weic rieeest tonglacche mom ana
Temporal- Sequenca Reasoning
The radd aftee
devilepe
Modlryy
Road
Vagetation
Buiding
Ski
thown the samre location atter ciy develemment.
Which
Image
C: Pleane rest dreetotee lans to most deweloped.
chronologial
Local Perception(LP)
2.Nodon prealant... rectoon
of ufter
Saleey
Divlyy
Static Scene Perception
Temporal Change Understanding
Subjective Environmental Perception
Figure 3: Overview of our UrbanFeel. UrbanFeel defines 11 sub-tasks spanning 3 cognitive di- mensions: static scene perception, temporal change understanding, and subjective environmental perception.
3.2 BENCHMARK TASK
Guided by a cognitively progressive evaluation framework, we design 11 diverse tasks and construct UrbanFeel, comprehensive benchmark for modeling urban development perception. As illus- trated in Fig. 3 these tasks span three levels of cognitive depth-Staatc Scene Perception, Temporal Change Understanding, and Subjective Environmental Perceptinn-enabling a dialiidiimesionnl assessment of MLLMs across recognition, reasoning, and perceptual alignment.
Static Scene Perception focuses on evaluating models' ability recognize salient visual elements and spatial consistency in single time frame. Tasks under this category include identifying dom- inant visual components in a given image and determining whether a pair of images single-view and panoramic depict the same geographic location. This dimension retains some classic scene perception tasks and aims to assess models' capacity for snapshot-level spatial understanding and contextual matching.
Temporal Change Understanding targets the model's ability to detect, differentiate, and reason about visual changes over time. Beyond identifying structural variations across temporally aligned images, models are required to classify the type of urban evolution (e.g., façade renovation, road maintenance, vegetation growth) and to perform temporal ordering of multiple images based on perceived development stages. These tasks simulate human-like reasoning about city progression and test the model's temporal-spatial integration abilities.
Subjective Environmental Perception emphasizes the alignment between MILMM and human sub- jective evaluation. We construct affective perception tasks across four dimensions beaeeniion-bbuuufffl, safe, wealthy, and lively-and require models not only to produce scalar judgments but also to provide localized visual justifications. In addition, we introduce before-after comparison tasks examine whether models can detect perceptual shifts in changing environments. This dimension moves be- yond objective recognition, probing whether MLLMs can simulate human affective responses in complex visual scenes.
3.3 BENCHMARK CURATION
Data Collection and Pre-processing, As shown in Figure the UrbanFeel benchmark collects over 4,000 street-view images from 11 cities across four continents via Mapillary and the Google Street View API, covering both single-view and panoramic formats. The selected cities include representative locations from the Global South (e.g., g., Kuala Lumpur, Tolyatti) and the Global North
5

---

Published as a conference paper at ICLR 2026
Data Collection & properscsing
Question Generation
MLLMs Inforence Quality Control
Download-source
Question Types
Input: Co
Image
Quastion
Googlo
Ceterris whstherimap
Judgement
Multiple
term city
charged
LLMs
Judgelt place appaara Realthy a1d
Detemiie the order the magee
Outtutt: @ Answer Stall Analysts
The theinageis weaktt Becanse
Sorting 2
?
Generate Question
Image Grouping
Reled Based
Question: Determine whethher two images
are location.
(A] Yes (8) No
Manusl Quality Inspection
Greand Trudie Error
43%
47%
50m
d5 Manual Annotation
category deseribes
18
City Number
LED ligly cill DM0 vndioc The
Negrrahin chauge
Metilly Standd Change
Ground Truthe
Correct
Figure 4: Benchmark construction process of UrbanFeel, including data collection and pre- processing, question generation, and MLLMs inference and quality control.
(e.g., Paris, Washington, D.C.), spanning temporal range from 2007 to 2024 and capturing diverse stages of urban development.
During the data pre- rre-processing stage, given the lack of precise spatial or temporal ordering in some images, we apply spatiotemporal clustering based on geolocation and timestamps to generate co- herent urban evolution sequences. To ensure data quality, we use pretrained segmentation model along with manual filtering to remove low-quality samples, such as indoor scenes, blurry captures, and heavily occluded images. Additional preprocessing details are provided in the Appendix C
Question Generation. UrbanFeel supports four question formats: binary judgment, multiple choice, sorting, and open-ended QA. To efficiently generate diverse question sets, adopt hy- brid strategy combining rule-based generation, model-assisted prompting, and manual authoring. For instance, tasks like same-place matching and future- prediction generated using tempo- ral and spatial metadata. For change-type recognition, we initialize annotations with outputs from general-purye purposs segmentation models, followed by manual verification and correction. Subjective perception tasks are written entirely by human annotators and span four dimensions: beautiful, safety, wealthy, and lively. Annotators also mark localized visual evidence to support reasoning and explainability evaluation.
MLLMs Inference and Quality Control. To ensure annotation accuracy and evaluation reliability, we introduce multi-stage validation pipeline. During model inference, strict output formatting constraints are enforced. Responses are automatically assessed using separate language model to compare with human- gunann-roovided ground truths. For ambiguous or illogical responses, manual review conducted to reclassify or remove problematic samples. This filtering ensures the final evaluation metrics robust and reproducible.
4 EXPERIMENTS
4.1 EVALUATED MODELS
We evaluate total of 20 multimodal large models under zero-shot setting using UrbanFeel, including 2 closed-source models and 18 open-source models. The closed-source models are GPT-4o (OpenAl 2024) and Gemini 2.5 Pro (Comanici et al., 2025), accessed via their official APIs. The open-source models cover diverse set of representative MLLM families, including DeepSeek- VL2 (Wu et al. 2024b). Intern 3 (Zhu et al. 2025). LLaVA (Guo & Huang 2025 Qwen2.5-VL Yu et al. 2025). Phi (Abdin al. 2024). Gemma-3 (Team et al.) 2025), and Idefics3-
8B (Laurençon et al. 2024). among others. A full list of model versions and configurations is
provided in Appendix

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative results for 2 closed-source and open-source MLLMs, well those for human and random guess across 11 tasks. The overall score is computed across all tasks. The maximum value and the second largest value of model performance in each task are indicated by the bold and underlined text, respectively. Task names are abbreviated for brevity.
Model
Scene Perception
Temporal Change Perception
Subjective Perception Consistency
Overall
DEE
SPM
TCR FSI PCR
ISR
SCR
GP
LP
CP
DeepScck-v12-tiny
44.5
18.0
44.5
48.2
25.5
31.5
24.1
59.6
53.8
43.6
36.1
DeepSeek-v12
99.3
29.0
43.7
94.8
53.8
37.1
8.2
38.9
65.8
40.9
33.3
45.9
MiniCPM-V2.666 2.6-8B
45.9
94.8
77.6
90.9
26.8
38.7
10.5
25.2
41.9
34,8
33.9
47.4
Owen2.S-v1-3B
76.2
43.2
28.9
17.1
35.6
28.3
Qwen2.5-vl-778
$2.4
98.2
75.8
85.1
43.2
33.4
10.0
43.9
56.6
33.8
38.6
51.9
Doven2.5-vl-72B
60.1
97.2
66.2
87.9
90.3
40.9
26.0
46.0
65.7
44,6
36.9
60.2
M.aVA-1.5-7B
1eoVV------mmisral-277
34,9
51.0
45.5
65.0
17.5
28.0
3.7
16.1
67.2
38.7
$1.0
38.
IntemVL13228
51.3
41.4
65.5
66.6
19.6
38.9
1.4
15.0
68.2
40.0
25.4
39.4
11.emV1.3-8B
31.8
Phi-3.5
46.0
57.2
51.4
75.4
56.9
24.8
7.8
37.4
34.0
374
36.2
44.1
Phi-4
37.7
26.2
69.7
82.9
57.4
32.1
2.3
39.9
71.0
42.2
48.1
46.3
Idefies-8BB
47.1
60.0
22.5
39.5
15.5
61.2
34.1
49.0
40,2
Mistral-Small-3.1-22
Sma21-3.1-24
19.6
91.6
67.9
86.6
64.3
28.9
16.0
46.7
63.3
42.4
39.9
$1.6
Aria
64.8
90.0
71.7
89.0
42.7
38.9
10.5
38.1
67.7
42.1
45.9
54.7
Aya-vision-Sb
38.6
51.5
26.2
24.6
33.3
69.6
43.8
39.5
G-mma-3-4h
42.5
81.6
64.1
59.5
25.9
39.0
8.2
36.5
53.1
46.1
26.7
43.9
Gemma-3-276
66.9
76.2
37.5
18.7
44.8
64.7
39.6
52.7
GPT-4o
50.8
79.2
74.2
40.5
49.9
60.2
37.3
36.4
59.4
Gemini-2 2.5-pro
64.8
96.3
78.2
95.4
97.6
36.3
52.1
56.3
67.7
49.0
30.3
Human
76.7
96.4
96.4
21.2
70.0
69.5
66.6
32.9
53.1
67.4
Random
21.4
48.6
50.3
47.6
26.3
19.5
3.9
18.8
51.0
18.2
35.2
31.5
4.2 EVALUATION PROTOCOL.
UrbanFeel includes four question types: binary judgment, multiple choice, sorting, and open- ended QA. Following the evaluation protocol of prior benchmarks such as MMMU (Yue al.) (024) and UrBench (Zhou et al., (0025), we adopt hybrid strategy combining exact matching, model verification, and semantic similarity evaluation.
For non-open-ended questions (i.e., judgment, multiple choice, and sorting), we first apply strict string matching answers are considered correct only if they match the reference label. However, since some models generate verbose responses without clearly selecting option, we employ an auxiliary language model to assess whether the prediction semantically aligns with the ground truth, ensuring fair evaluation of models that include rationale in their outputs. For open-ended questions, correctness is determined by measuring semantic similarity between the model-generated answer and reference answers.
For the human baseline, we recruited two independent groups of ten participants each, all with geography-relann related academic backgrounds (undergraduate, master's, or doctoral students). One group conducted the annotations, while the other performed the evaluations, ensuring no overlap between the More implementation details of evaluation protocol described in Appendix
4.3
MAIN RESULTS
Overall Challenge of UrbanFeel. Table D summarizes the overall quantitative performance of mainstream Multimodal Large Language Models (MLLMs) on UrbanFeel, revealing the significant challenges posed by our benchmark. While closed-source models such as GPT-4o and Gemini-2.5. Pro demonstrate impressive capabilities on selected tasks, their overall performance remains sub- stantially behind human-level sacuracy-particuuu and spatiotemporal understanding. This performance gap is more pronounced for open-source models, suggesting that current MLLMs still face major limitations in practical applications related to urban development, environmental perception, and city planning.
tasks that require compositional reasoning
Performance Across Task Dimensions. Table I further disaggregates model performance across the 11 sub-tasks in UrbanFeel. Most MLLMs exhibit strong capabilities in basic visual recogni- tion tasks; for instance, the majority models achieve over 60% accuracy on the Recognition (TCR) task, which requires only straightforward temporal identification.

---

Published as a conference paper at ICLR 2026
(b)
60
55
50
50
45
40
40
38
MuCCPM-N26
30
Idefictt-6D
Aria
PM-V PM-22 6-40
Mistral Suatla--Sall-33 Phi-1. 5
L1-00
1-248
Qwent LL-3B
Owand VL-7B
Phi-d
Qwem S-VI---77
DeenSeck Borkeek-VVLa VL2-ing 24iny
Ara-iiaion an
DeepBeok VL.2
Genmaz 2-48
Genmad 2.1
LLaVA-1 (er-s-----miitet 6-mistrall1
LLaVA--I. 5-70
Gemini 3.r.
GPT-4a
Human
Beautiful Lively Safe
Wealthy
Beautiful
Lively
Wealthy
Safe
Figure 5: Quantitative comparison of MLLM performance on subjective environment perception. (a) Accuracy across four dimensions. (b) Box plots show model variance, where the horizontal lines boxes indicate medians; box width indicates consistency.
However, model performance drops significantly when spatial and temporal reasoning must be in- tegrated. In the TSR task, for example, most models score below 10% accuracy. Even the best- performing
Seniini-2.5-PaP 2.5-Pro 1.5-Pro-still lags behind human performance by 17.9% in accuracy, revealing that existing models still struggle with Long-range temporal ordering and scene-level align- ment across time and space.
Interestingly, in the dimension of subjective environmental perception, many models show strong consistency with human judgment (CP, LP). Their visual justifications-sucee as cues used to in- fer aesthetic or safety-often align closely with those identified by human annotators, suggesting early potential for human-aligned perceptual reasoning. However, this alignment weakens substan- tially when temporal dynamics are introduced. On tasks involving perceptual comparison between before-afie scenes, most models exhibit heightened sensitivity to visual changes, often overempha- sizing fine-grained variations and diverging from human-level perceptual stability. More subjective analysis cases will be displayed in the Appendix D
To our surprise, in the PCR task with panoramic inputs, MILMs outperform human evaluators. This is because humans are less sensitive to pixel-level differences caused by panoramic distortions. While evaluators focus on salient foreground changes, mid- or long-range building variations occupy only small pixel proportion and may be less noticeable than background shifts in sky or road caused by slight camera movements, leading to frequent misjudgments.
DISCUSSION
5.1 MODEL PERFORMANCE ACROSS SUBJECTIVE PERCEPTION DIMENSIONS
To further investigate model behavior in subjective environmental perception, Figure 5 a) presents accuracy distributions across four key dimensions. The results show that MLLMs exhibit consider- able variation across dimensions. Most models achieve human-comee comparable even superior accu- racy in dimensions such as Safe, Beautiful, and Lively, suggesting promising potential for aligning with human perceptual judgments in urban scenes.
Among these, Safe is the dimension where most models perform best, reaching an average accuracy of 50.6%. However, this dimension also reveals the largest performance gap between models, indi- cating substantial inconsistency in safety-related judgments. In contrast, Lively displays more stable accuracy across models, despite having a slightly lower average performance, suggesting that mod- els more consistently capture liveliness, likely by relying broad visual cues such vehicles and crowds. However, in the Wealthy dimension, the models still underperform human evaluators by an average margin of 10.1%, implying that wealth perception may involve more nuanced or culturally specific visual cues that current models struggle to capture.
The box plot in Figure b)b) further supports these observations. Although the Safe dimension has the highest median accuracy (52.4%), also exhibits the widest interquartile range and largest over- all variance, confirming the inconsistent model behavior. Conversely, the Lively dimension has
8

---

Published as a conference paper at ICLR 2026
Beautiful
0.2
1,0
Cape Lumpur
City
Washington,
City
Wealthy
0.2
0.2
1.0
No
Cape Towa
Knala Lampue
Mexico City
Milan
Parts
Wastington, D.C.
No Pretend City
G1
Phi-4
LLaVA-L.6-mittr1-778
DeepSack-VL2
Sww-225-VVL-728
GPI-4o
GT
Figure 6: Quantitative comparison of different models under the assumed city identity setting. "Yes" indicates the proportion of positive evaluations made by MLLMs for the given perceptual dimension, while "No" represents the proportion of negative evaluations. The results show that LLaVA-1 6,6- mistral-7B and DeepSeek-v12 yield more positive evaluations across most cities, while Qwen2.5-Ve and GPT-4o show decline under assumed city identity.
the concentrated distribution, indicating higher inter-model agreement. This consistency sug- gests that current MLLMs may rely on more universal or easily detectable signals when evaluating liveliness, whereas dimensions like safety and wealth require finer-graiee uner-grained perceptual reasoning or socio-cultural understanding.
5.2
DOES CITY IDENTITY AFFECT MLLMs' SUBJECTIVE ENVIRONMENTAN JUDGMENTS?
To examine whether MLLMs exhibit geographic bias in subjective perception, we conducted "city identity intervention" experiment. We randomly selected 100 street-view images from the GP val- idation set and assigned each one of six hypothetical city identities (Cape Town, Kuala Lumpur, Mexico City, Milan, Paris, Washington, D.C.), comparing the results those without any assigned identity ("No Pretend City"). Figure 6 shows the distributions of positive (e.g.. "beautiful") and negative (e.g., "not beautiful") judgments under the Beautiful and Wealthy dimensions. Results for Lively and Safe are included in the Appendix
Overall, most models exhibit varying degrees of change in their subjective judgments when city identity introduced. LLaVA-1.6 and DeepSeck-v12 tend to produce more positive evaluations across most cities, suggesting a tendency to interpret identity labels favorably. In contrast, Phi-4 demonstrates high stability, indicating greater reliance image content and robustness added semantic labels. Notably, GPT-4o and Qwen2.5-VL. show a general decline in positive judgments when city identity is provided, implying a more "cautious" even "conservative" evaluation behav- ior, potentially triggered by the activation of learned stereotypes or expectations.
When comparing "Global North" and "Global South" city identities, observe that northern identity does not necessarily lead to more favorable evaluations. Although the average score for northern cities is slightly higher, GPT-4o's perception wealth for "Paris" and "Milan" drops cunificantly-someiiinn
even below that of cities like "Cape Town." This counterintuitive result may stem from a mismatch between the semantic label and the actual image content; for example, ordinary aged urban scenes labeled as "Paris" may result in greater expectation gaps, prompting the model to generate more negative evaluations. Conversely, GPT-4o's slightly increased positive judgments for "Mexico City" may be attributed positive visual signals such modern buildings, combined with the lack of strong negative priors associated with
clean streets, and bright the label "Mexico City" in the model's pretraining corpus.
5.3 Do MLLMS PERCHIVE SINGLE- VIEW AND PANORAMA DIFFERENTLY?
To evaluate whether the differences in viewpoint coverage and information organization between single-view and panorama introduce perceptual biases in MLLMs, we compare model performance across the two perspectives. As shown in Figure the majority of models consistently perform

---

Published as a conference paper at ICLR 2026
better single-view images than on their panorama counterparts. On average, single-view inputs yield an accuracy improvement of 1 1.7% over panoramic inputs. Notably, Gemini-2.5-PHr achieves the highest accuracy single-view images 69.4%, closely by Qwen2.5-VL-72B with 64.9%. In terms of panoramic images, Aria shows the best performance with 55.3%, while Gemini-2.5-Pro follows closely with with 54.9%.
This performance gap suggests that although panorama offer greater spatial coverage and denser visual information, their inherent geo- metric distortions and contextual blending may increase the "cogmitive burden" on MLLMs. It also shows that MILMS have perspective data imbalance and bias during training process.
Single View
Panorama
40
5.4 DOES EXPLICIT REASONING ENHANCE TEMPORAL UNDERSTANDING?
Ideffesk- S.
Aria
IntemV MlmmVL33.2) 3-
Min 6---8 6-88
Intenv 8-8
Mistral Mistral-Small- 1-248
Pli-..
Ownn 5-V1--177 TD
Owon2 S.VL -BB
Phi-J
Qwm2 5 728
viivviion--88 Deep Ve I-mmy
menseek-VV 1/12
Gemmai +I
Gemma3-22 278
Lia f-mistral-
LLaVe-l 7B
Gamin- 2.5ini-2.5-PPo P-o
GPT-40
To investigate whether guiding MILMS with explicit reasoning steps enhances performance in complex urban temporal tasks, conducted a controlled ablation study on the Temporal Se- quence Reasoning (TSR) task, comparing Di- rect Sorting, General CoT, and Re- Re-Thinking strategies. Contrary to the expectation that explicit reasoning consistently improves performance, our results reveal a counter-intuitive trend: for sfate-of-the-art models such as GPT-4o, Qwen2.5- VI.-2B,, and Gemini-2.5-PPo, the Direct Sorting strategy consistently yields the highest accu- racy. Introducing verbose reasoning steps often leads to performance degradation; for instance, GPT-4o's s accuracy drops from 46.1% to 37.0% when using the Re-Thinking strategy. Qualitative analysis suggests this stems from "over-reasoning" pitfall, where models hallucinate subtle de- tails or over-interpret transient noise to justify an incorrect linear progression. We provide detailed quantitative results and specific case studies of this phenomenon in Appendix C.4 and D.
Figure 7: Quantitative results of MLLMS perfor- mance from different perspectives.
CONCLUSION
In this study, we introduces UrbanFeel, a new benchmark for evaluating the capabilities of Multimodal Large Language Mod- els (MLLMs) in urban development understanding and subjec- tive perception. The benchmark includes over 14.3K questions across 11 tasks, covering static scene perception, temporal change understanding, and subjective environmental perception. It is
50
Accuracy (%)
30
constructed using single-view and panoramic street-view images
20
from 11 cities, spanning more than 15 years. We evaluate 20 MLLMs and identify key limitations. Current models underper- form in tasks requiring joint observe geographic bias in subjective perception tasks, where pre- dictions vary with city identity. In addition, models show per- ceptual inconsistencies across different viewpoints, particularly between single-view and panoramic inputs. We envision Urban- Feel advancing perception-awar urban intelligence. By bridg- ing temporal evolution and human perception, this work positions MLLMS as scalable tools for the continuous monitoring and as- sessment required achieve sustainable urban development goals.
reasoning. We also
Direct
General
Sorting
CoT
thinking
Phi-4
InterVLL-88B
GPT-40
Qwen2.5-VL--22
Gemini p.5
Figure 8: Quantitative results of different prompt input TSR.
ACKNOWLEDGEMENT
This research is funded by the National Natural Science Foundation of China (Grant No. 62571560), Natural Science Foundation of Guangdong Province (grant No. 2024A1515012083) and the Na- tional Key Research and Development Program of China (grant No. 2022YFB33034022)
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We developed UrbanFeel guided by principles of responsible and ethical Al research. We acknowl- edge that, despite efforts ensure coverage and diversity, our dataset and annotations may still carry biases from source imagery, annotator backgrounds, or cultural contexts-especialll in sub- jective perception tasks such as beauty, safety, wealth, and liveliness. Users should remain vigilant to these limitations. To mitigate bias, we provided annotators with standardized guidelines and training before labeling. The annotations are intended reference labels that reflect the consensus inclinations of the annotator group, not as absolute ground truth. Moreover, we recognize the possi- bility that models built on UrbanFeel might be misused to influence public perceptions or aesthetic judgments. Our intent is for positive applications urban analysis, perceptual model evaluation, and human-machine alignment research-ang we explicitly disavow any malicious uses. By releasing data, code, and evaluation tools, we hope to foster transparency, accountability, and further work toward fairer, cross-cultural, and ethically grounded Al urban contexts.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the full reproducibility of our work. The core concepts and method- ology of our benchmark design detailed in Section B] including the three task dimensions and the benchmark curation. Our experimental setup, including datasets, evaluation procedures, and baseline MILMs,, is descrihed in Section 4 and Appendix C Additional details the manual notation process, human evaluation settings, and operational guidelines for subjective perception tasks are provided in Appendix B To facilitate the reproduction of our results and support further research in urban multimodal perception, we will release the complete codebase, benchmark dataset (UrbanFeel), and evaluation scripts, along with detailed documentation and pretrained model pre- dictions.
REFERENCES
Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, al. Phi-4 techni- cal report. arXiv preprint w:Xiv:2412.08900. 2024.
Tim Alpherts, Sennay Ghebreab, and Nanne van Noord. Emplace: Self-supervised urban scene change detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 1737-1745, 2025.
Anonymous. ICG: Improving cover image generation via MLLM-based prompting and personalized preference alignment. In Submitted to ACL Rolling Review May 2025, 2025. URL https: under review.
//openzeview.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint S13iv::300.16000 2023.
Yanis Benidir, Nicolas Gonthier, and Clément Mallet. The change you want to detect: Semantic change detection in earth observation with hybrid data generationf. In Proceedings of the Com- puter Vision and Pattern Recognition Conference, pp.
2025.
Filip Biljecki and Koichi Ito. Street view imagery in urban analytics and gis: A review. Landscape and Urban Planning, 215:104217, 2021.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.0626. 2025.
Saurabh Dash, Yiyang Nan, John Dang, Arash Ahmadian. Shivalika Singh, Madeline Smith, Bharat Venkitesh, Vlad Shmyhlo, Viraat Aryabumi, Walter Beller-Morales, Jeremy Pekmez, Jason Ozuzu, Pierre Richemond, Acyr Locatelli, Nick Frosst, Phil Blunsom, Aidan Gomez, Ivan Zhang, Marzieh Fadace, Manoj Govindassamy, Sudip Roy, Matthias Gallé, Beyza Ermis, Ahmet Üstün,
11

---

Published as a conference paper at ICLR 2026
and Sara Hooker. Aya vision: Advancing the frontier of multilingual multimodality, 2025. URL https://arxiv.orgg http://wwww/b///...../o/
Boyang Deng, Songyou Peng. Kyle Genova, Gordon Wetzstein, Noah Snavely, Leonidas Guibas, and Thomas Funkhouser. Visual chronicles: Using multimodal Ilms to analyze massive collec- tions of images. arXiv preprint wrXiv:2504.0872) 2025.
Abhimanyu Dubey, Nikhil Naik, Devi Parikh, Ramesh Raskar, and César A Hidalgo. Deep learning the city: Quantifying urban perception at a global scale. In European conference on computer vision, pp. 196 212. Springer, 2016.
Zhuangyuan Fan, Fan Zhang, Becky PY Loo, and Carlo Ratti. Urban visual intelligence: Uncovering hidden city profiles with street view images. Proceedings of the National Academy of Sciences, x20077)e2220417100 2023.
Jie Feng. Tianhui Liu, Yuwei Du, Siqi Guo, Yuming Lin, and Yong Li. Citygpt: Empowering urban spatial cognition of large language models. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pp. 591-602, 2025a.
Jie Feng, Shengyuan Wang, Tianhui Liu, Yanxin Xi, and Yong Li. Urbanllava: A multi-modal large language model for urban intelligence with spatial reasoning and understanding. arXiv preprint arXiv:2506.2323 23321, 2025b.
Jie Feng. Jun Zhang, Tianhui Liu, Xin Zhang, Tianjian Ouyang, Junbo Yan, Yuwei Du, Siqi Guo, and Yong Li. Citybench: Evaluating the capabilities of large language models for urban tasks. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. pp. 5413-5424, 2025c.
Alexander Follmann, Maximilian Willkomm, and Peter Dannenberg. As the city grows, what do farmers do? systematic review of urban and peri-urban agriculture under rapid urban growth across the global south. Landscape and Urban Planning, 215:104186, 2021.
Yunfei Guo and Wu Huang. Llava-next-med:: Medical multimodal large language model. In 2025 Asia-Europe Conference on Cybersecurity, Internet of Things and Soft Computing (CITSC), pp. 474-477. IEEE, 2025.
Assem Abu Hatab, Maria Eduarda Rigo Cavinato, August Lindemer, and Carl-Johan Lagerkvist. Urban sprawl, food security and agricultural systems in developing countries: A systematic review of the literature. Cities, 149- 2019.
Jialyu He, Jinbao Zhang, Yao Yao, and Xia Li. Extracting human perceptions from street view images for better assessing urban renewal potential. Cities, 134: 134:104189, 2023.
Jun He, Yaopeng Zou, Zijun He, Qiliang Yi, and Xiang Zhang. Are there any benches around you? attribution of distribution and impact of perception differences of urban benches. In IGARSS 2025- 2025 IEEE International Geoscience and Remote Sensing Symposium, pp. 6103-6107. IEEE, 2025a.
Zongze He, Jun He, and Xiang Zhang. Can languae models see the world like you do? In IGARSS 2025-2025 IEEE International Geoscience and Remote Sensing Symposium, pp. 3194- 3198. IEEE, 2025b.
Jinghao Huang, Yaxiong Chen, Shengwu Xiong, and Xiaoqiang Lu. Visual contextual semantic reasoning for cross-moda drone image-text retrieval. IEEE Trans. Geosci. Remote. Sens., 62: 1-12, 2024a.
1109/TGRS.
Tianyuan Huang, Zejia Wu, Jiajun Wu, Jackelyn Hwang. and Ram Rajagopal. Citypulse: Fine- grained assessment of urban change with street view time series. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 22123-22131 2024b.
Zilong Huang. Jun He, Xiaobin Huang, Ziyi Xiong, Yang Luo, Junyan Ye, Weijia Li, Yiping Chen, and Ting Han. Majutsucity: Language-drin driven cesthetic-adaptii city generation with controllable 3d assets and layouts. arXiv preprint arXiv:251 1.20415, 2025.
12

---

Published as a conference paper at ICLR 2026
Koichi Ito, Yuhao Kang, Ye Zhang, Fan Zhang, and Filip Biljecki. Understanding urban perception with visual data: A systematic review. Cities, 152:105169, 2024.
Jitesh Jain, Jiachen Li, Mang Tik Chiu, Ali Hassani, Nikita Orlov, and Humphrey Shi. Oneformer:
One transformer to rule universal image segmentation. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 2989-29999 2023.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https:
Siqi Lai, Yansong Ning. Zirui Yuan, Zhixi Chen, and Hao Liu. Ustbench: Benchmarking and dissecting spatiotemporal reasoning of ilms as urban agents. arXiv preprint arXiv:2505. 17572, 2025.
Hugo Laurençon, Andrés Marafioti, Victor Sanh, and Léo Tronchon. Building and bet- ter understanding vision-language models: insights and future directions. arXiv preprint arXiv:2408. /2637, 2024.
Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Guoyin Wang, Bei Chen, and Junnan Li. Aria: An open multimodal native mixture-of-experts model. arXiv preprint arXiv:2410.05993, 2024a.
Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895. 2024b.
Weijia Jinhua Yu, Dairong Chen, Lin, Runmin Dong, Xiang Zhang, Conghui He, and Haohuan Fu. Fine-grained building function recognition with street-view images and gis map data via geometry-aware semi-supervised learning. International Journal of Applied Earth Observation and Geoinformation, 137:104386, 2025.
Tianhui Liu, Jie Feng, Hetian Pang, Xin Zhang, Tianjian Ouyang, Zhiyuan Zhang, and Yong Li. Citylens: Benchmarking large language-vision models for urban socioeconomic sensing, arXiv preprint arXiv:2506.005: 00530, 2025.
Chuofan Ma, Yi Jiang, Jiannan Wu, Zehuan Yuan, and Xiaojuan Qi. Groma: Localized visual tokenization for grounding multimodal large language models. ArXiv, abs/2404. 13013, 2024. semant icschola.. wtt.ccch........rrooo/oooooaaaaa
URL httphhtttt///ppp http:/pp..smaaiiiiiicimammm...
Nikhil Naik, Scott Duke Kominers, Ramesh Raskar, Edward Glaeser, and César A Hidalgo. Com- puter vision uncovers predictors of physical urban change. Proceedings of the National Academy of Sciences, 114(29)7571-7757, 2017.
OpenAl. Gpt-4o (may 13, 2024 version) [large language model]. httts::/ccat..ppee com/ 2024.
Bhartendu Pandey and Karen Seto. Urbanization and agricultural land loss in india: Comparing satellite estimates with census data. Journal of environmental management, 148:53 66, 2015.
Jin Rui and Yuhan Xu. Beyond built environment: Unveiling the interplay of streetscape perceptions and cycling behavior. Sustainable Cities and Society, 109:105522 2024.
Steven Stalder, Michele Volpi, Nicolas Büttner, Stephen Law, Kenneth Harttgen, and Esra Suel. Self-supervised learning unveils urban change from street-level images. Computers, Environment and Urban Systems, 12116,, 2024a.
Steven Stalder, Michele Volpi. Nicolas Büttner, Stephen Law, Kenneth Harttgen, and Esra Suel. Self-supervised learning unveils urban change from street-level images. Computers, Environment and Urban Systems, 12: 102156,, 2024b.
13

---

Published as a conference paper at ICLR 2026
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint 25X1:22503.19766, 2025.
Adam Van Etten, Daniel Hogan, Jesus Martinez Manso, Jacob Shermeyer, Nicholas Weir, and Ryan Lewis. The multi-temporal urban development spacenet dataset. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6398-6409 2021.
Joppe van Veghel, Gamze Dane, Giorgio Agugiaro, and Aloys Borgers. Human-centric computa- tional urban design: optimizing high-density urban areas to enhance human subjective well-being. Computational Urban Science, 4(1):13, 2024.
Zixuan Wang, Xiang Zhang, Yuchuan Zhou, Yiyi Jiang, and Haibin Xu. Exploring functional zone- dependent nonlinear associations between objective features and subjective perceptions: A case study in beijing. International Journal of Applied Earth Observation and Geoinformation, 142: 104682, 2025.
Jingxian Wei, Wenze Yue, Mengmeng Li, and Jiahin Gao. Mapping human perception of urban landscape from street-view images: A deep-learning approach. International Journal of Applied Earth Observation and Geoinformation, 12:1002886, 2022.
World Bank. Urban population phttps://datt www.ddaw..www.woo.oooooooooooooe wor www.lllannkk.rrooee.ee www..abb...rrr/.ddi.atoo... URB. TOTL. IN. 25 2024. Accessed: 2024-08-07.
Tao Wu, Mengze Li, Jingyuan Chen, Wei Ji, Wang Lin, Jinyang Gao, Kun Kuang, Zhou Zhao, and Fei Wu. Semantic alignment for multimodal large language models. Proceedings of the ACM International Conference on Multimedia, 2024a. UtLpttps: //atps sww.nt cschhoarr. org/CorpusID::2115.2
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et Deepseek-v12: -feeeerss vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412. 10302, 2024b.
Weihao Xuan, Junjue Wang, Heli Qi. Zihang Chen, Zhuo Zheng, Yanfei Zhong, Junshi Xia, and Naoto Yokoya. Dynamicvl: Benchmarking multimodal large language models for dynamic city understanding. ArXiv, 0tbp//555.21066.6 2025. URL https: www.pe/sstaaatnnccilaaaamo.
org/CorpusID: 22822166928
Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmer- mann, and Yuxuan Liang. Urbanclip: Learning text-enhanced urban region profiling with con- trastive language-image pretraining from the web. In Proceedings of the ACM Web Conference 2024, pp. 4006 4017, 2024.
Jihan Yang, Runyu Ding, Ellis Brown, Xiaojuan Qi, and Saining Xie. V-irl: Grounding virtual intelligence in real life. In European conference on computer vision, pp. 36-55. Springer, 2024.
Yao Yao, Zhaotang Liang, Zehao Yuan, Penghua Liu, Yongpan Bie, Jinbao Zhang, Ruoyu Wang. Jiale Wang, and Qingfeng Guan. A human-machine adversarial scoring framework for urban per- ception assessment using street-view images. International Journal of Geographical Information Science, 33(12):2363-233) 2019.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang. Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint a4Xiv:2408. 01 800, 2024.
Junyan Ye, Jun He, Xiang Zhang, Yi Lin, Honglin Lin, Conghui He, and Weijia Li. Satellite image synthesis from street view with fine-grained spatial textual guidance: A novel framework. IEEE Geoscience and Remote Sensing Magazine, 2025.
Yueyang Yu, Chuanwei Shi, Jiuqi Tang, and Sicheng Zheng. Qwen-vl2 model with neftune tech- nique for medical report generation. In 2025 4th International Symposium on Computer Applica- fions and Information Technology (ISCAIT), pp. 165-168. IEFE, 2025.
14

---

Published as a conference paper at ICLR 2026
Shuai Yuan, Guancong Lin, Lixian Zhang, Runmin Dong, Jinxiao Zhang, Shuang Chen, Juepeng Zheng, Jie Wang, and Haohuan Fu. Fusu: A multi-tempora--suuee land use change segmen- tation dataset for fine- fine-grained urban semantic understanding. Advances in Neural Information Processing Systems, 37:132417-132499 2024.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9556-9567, 2024.
Congzhi Zhang, Jiawei Peng, Zhenglin Wang, Yilong Lai, Haowen Sun, Heng Chang, Fei Ma, and Weijiang Yu. Vrest: Enhancing reasoning in large vision-language models through tree search and self-reward mechanism. In AnLC (1), pp. 3922-3941, 2025a. URL httptttts///aaatttttt org acl
long. 1997
Fan Zhang, Bolei Zhou, Liu Liu, Yu Liu, Helene H Fung, Hui Lin, and Carlo Ratti. Measuring human perceptions of a large-scale urban region using machine learning, Landscape and Urban Planning. 1880:18-1660 160, 2018.
Jiaxin Zhang, Yunqin Li, Tomohiro Fukuda, and Bowen Wang. Urban safety perception assessments via integrating multimodal large language models with street view images. Cilies, 2024a. URL
https://appe www.steieinnnnoeeeeeeeee
Jiaxin Zhang, Yunqin Li, Tomohiro Fukuda, and Bowen Wang. Urban safety perception assessments via integrating multimodal large language models with street view images. Cities, 165:106122,
2025b.
Maomao Zhang, Shukui Tan, Jinshui Liang, Cheng Zhang, and Enqing Chen. Predicting the im- pacts of urban development on urban thermal environment using machine learning algorithms in nanjing, china. Journal of Environmental Management, 356: 120560, 2024b.
Baichuan Zhou, Haote Yang, Dairong Chen, Junyan Ye, Tianyi Bai, Jinhua Yu, Songyang Zhang, Dahua Lin, Conghui He, and Weijia Li. Urbench: A comprehensive benchmark for evaluating large multimodal models in multi-view urban scenarios. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 10707-10715, 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint 2544:25544414799 2025.
15

---

Published as a conference paper at ICLR 2026
UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective
Supplementary Material
In this appendix, we present supplementary materials that could not be included in the main paper due space constraints. These materials offer extended details on the benchmark construction, evaluation protocols, and additional experimental results to support reproducibility and inspire future research. Specifically, we provide:
Additional Benchmark Statistics: Including detailed question distributions and compar- ative analysis with existing urban perception benchmarks.
Benchmark Construction Details: Covering the entire pipeline from data collection and pre proceeseesngg to manual annotation and evaluation protocols.
Listing all evaluated baseline models, pre- senting extended experimental results, and analyzing whether reasoning-augment mod- els outperform standard baselines on UrbanFeel. We also provide detailed results on how assigned city identities influence model perception across different dimensions.
Experiment Details and Additional Results:
Case Studies: Illustrative examples showcasing model responses and human annotations across UrbanFeel's s task types.
Discussing current constraints of UrbanFeel and outlining directions for further expansion and refinement.
Limitations and Future Work:
Use Large Language Models: Describe the main uses of LLM the writing of this manuscript.
A ADDITIONAL BENCHMARK STATISTICS
A.1 QUESTION STATISTICS
Table 2 summarizes the number of test and validation instances across different subtasks. Urban- Feel comprises carefully curated set of 14.3K visual questions, with 11K the test set and 3.3K in the validation set. Our benchmark provides comprehensive suite of representative questions under three evaluation dimensions, enabling a holistic assessment of MILMs' capabilities in un- derstanding spatiotemporal urban dynamics and aligning with human perception in complex urban development scenarios.
Table 2: Question Distribution in Test and Val Sets
Evaluation Dimension
Task
Test
DEE
1600
400
Static Scene Perception
CR
400
100
SPM
238
60
TCR
1760
440
FSI
Temporal Change Perception
PCR
880
220
TSR
160
SCR
880
220
GP
1358
560
Subjective Perception Consistency
L.P
1638
280
1241
Total
11035
3260
16

---

Published as a conference paper at ICLR 2026
Table 3: Comparison of Existing Urban Scene Benchmarks. Our UrbanFeel designs a variety of question types over a longer time span, comprehensively evaluating the perception ability of differ- MLLMs on the physical space and human subjective dimensions of the environment in urban development scenarios.
Feature
CityBench
CityGPT
USTBench
Urbanch
UrbanFeel (Ours)
Judgements
x
X
X
X
Multi-Choice
V
Open-Ended
X
X
X
Sorting
Static Scene
V
Historical Data
X
X
X
X
Subjective Perception
X
x
X
Temporal Resolution
Hourly or Event-based
Hourly or event-based
Hourly with planning loops
Event-bused
Yearly
sequence
A.2 BENCHMARK COMPARISON
Table 3 presents a systematic comparison between UrbanFeel and existing urban perception bench- marks. Current benchmarks such as CityBench (Feng et al., (025c), CityGPT (Feng et al. (025a), and USTBench (Lai et al. 2025) primarily focus on single-timestamp static images tasks at the hourly or event level, aiming to assess models' understanding of urban infrastructure or real-time dynamics. These tasks often constructed using short-term trajectories, single-frame imagery, or synthetic datasets, and lack modeling or reasoning over real-world urban evolution.
In contrast, UrbanFeel emphasizes evaluating MLLMs' multimodal perception capabilities within the long-term context of urban development. introduces street-view image sequences spanning 17 years, capturing visual transformations across phases of urban planning, expansion, and renovation. More importantly, UrbanFeel goes beyond physical spatial changes by systematically incorporating subjective environmental perception, designing tasks that assess perceptions of beautiful, lively, safe, and wealthy- thus exploring how human perceptions of environmental quality shift across different urban contexts, and how well models align with such perceptions.
Additionally, UrbanFeel supports diverse task formats including multiple choice, open-ended rea- soning, and ranking-and and covers both static scene understanding and dynamic urban transforma- tions. It fills critical gaps in spatiotemporal reasoning and human-centric perception evaluation, establishing the first comprehensive multimodal benchmark framed from the perspective of hu- man man-city interaction.
B BENCHMARK CONSTRUCTION DETAILS
B. DATA COLLECTION DETAILS
This section outlines the data collection pipeline for street-view imagery used in UrbanFeel. The dataset comprises two main components: single-view images and panoramic images. We first obtained the vector boundaries of 11 representative cities using OpenStreetMup.
For sing imagery, we utilized the Mapillary API provided by the Global Streetscapes dataset to download images. Each image was renamed using standard- ized format based the provided timestamp, geographic coordinates, and image ID: {lat,
jpg. to facilitate streamlined preprocessing and file management. We strictly adhere the CC BY-SA license for Mapillary data, using official tools for acquisition and ensuring that all released content satisfies the attribution and share-alike requirements.
For panoramic imagery, we employed the Google Street View API to retrieve data and aligned the naming convention with that of the single-view images. This ensures consistency across spatial
17

---

Published as a conference paper at ICLR 2026
Conaication
Figure 9: An example of the user interface of LabelU.
and temporal dimensions, enabling cross- view and temporal comparisons in subsequent benchmark tasks. To ensure full compliance with Google's Terms of Service and ropyrighteegulation regulations, the open- source version of UrbanFeel does not distribute raw Google Street View imagery. Instead, we re- lease only the unique Panorama IDs and metadata, accompanied by retrieval script that allows researchers to legally fetch the images via the official API using their own credentials.
B.2 DATA PRE-PROCESSING DETAILS
During data preprocessing, we first standardized the orientation of all street-view images by rotating them to face true north based on camera heading metadata, minimizing the impact of viewpoint variation in panoramic images on downstream MILM perception. Since the images lack explicit spatial relationships, we performed spatial clustering by calculating pairwise geodesic distances using image coordinates, with a 50-meter threshold to identify multi-view, multi-temppral image sequences from the same urban location.
We then applied the OneFormer et al. 2023) semantic segmentation model to preprocess the images, discarding those with less than 5% sky coverage, which were likely captured indoors. Fi- nally, we manually filtered out low-quality images affected by motion blur, overexposure, or other visual defects.
B.3 MANUAL ANNOTATION AND EVALUATION DETAILS
During the annotation phase, acknowledging the inherent subjectivity of perception-related tasks, recruited group of 10 undergraduate and master's students with geography-related academic backgrounds. Annotators were provided with standardized written guidelines (as shown in Table and short training session with representative examples prior to the formal labeling process. They were instructed to identify localized visual evidence from the images that supported their global perceptual judgments. The resulting annotations are treated as reference labels rather than absolute ground truth, reflecting the consensus tendencies of this annotator group. All annotation work was conducted using the LabelU platform. Fig. 9 illustrates an annotation case the Local Perception task of the Beauty dimension.
For the evaluation phase, we recruited an independent group of volunteers, entirely distinct from the annotators, comprising undergraduate, master's, and doctoral students in geography-relate fields. This separation ensured that annotation and evaluation were performed by different popula- tions, thereby reducing potential bias introduced by overlapping roles. All human assessments were conducted on the LabelLLM platform, which provided standardized interface for task interaction and response collection.
is important to note that subjective concepts such as beauty, safety, or wealth are inevitably in- fluenced by cultural and personal perspectives. While the provision of operational guidelines and
18

---

Published as a conference paper at ICLR 2026
Table 4: Annotation guidelines for subjective perception dimensions. These guidelines were pro- vided to annotators as operational references rather than absolute criteria.
Dimension
Description (Instruction for Annotators)
Annotators were instructed to focus the overall aesthetic impression of the scene, considering whether the environment appears visually harmonious, orderly, and pleasant. They were asked pay attention greenery, landscaping. architectural style and maintenance, cleanliness of streets, balance of colors, and whether the layout looks uncluttered. Example: a tree-lined avenue with well-maintained modern buildings should labeled graffiti and broken infrastructure.
Beauty
beautiful than cluttered street with
Annotators were instructed to evaluate whether the environment gives sense of security. especially from a pedestrian's perspective. They were asked to check for cues such as adequate street lighting, visible sidewalks, orderly traftic. the presence of surveillance cameras or other visible security measures, and the absence of disorder (e.g.. litter, vandalism). Example: well-lit commercial street with open shops and visible security cameras should be laheled safer than dark, narrow alley with poor visibility and signs of decay.
Safety
Annotators were instructed to judge the degree of pereeived cconomic prosperity in the environment. They were asked to consider quality modernity of buildings, presence commercial activity (e.g. s., branded shops), the maintenance of infrastructure, and visible indicators aflluence (e-g., luxury cars). Example: a district glass office towers and upseale retail should be labeled as wealthier than neighborhood with dilapidated housing and cracked pavements.
Wealth
Annotators were instructed to assess the vibrancy and human activity in the scene. They were asked to pay attention pedestrians, cyclists, vehicles, open businesses, street vendors, public events, as well as infrastructure supporting activity (benches, playgrounds). Example: a busy marketplace with crowds and open shops should be labeled more lively than empty with little visible activity.
Liveliness
training sessions was intended to minimize ambiguity and promote consistency across participants, these annotations should be regarded as references produced by specific annotator population, rather than universal ground truth. Future work will extend this framework through cross-cultural annotation campaigns and inter-annotator agreement analyses to further address cultural bias and subjective variability.
B.4 EVALUATION PROTOCOLS
Given the varying structures and formats of different question types in UrbanFeel, we adopt tailored evaluation strategies for each to ensure fairness and reproducibility.
To provide rigorous quantitative assessment, we adopt Accuracy (ACC) as the primary evaluation metric across all tasks. The accuracy is computed as:
ACC .Cooreet Predictions
NTotal Predictions
where Total Predictions denotes the total number of evaluated instances, and Correct Predictions denotes the number of instances for which the model output satisfies the task-specific correctness criterion.
1. Exact Matching for Objective Tasks (MCQ, Binary, Sorting). For multiple-choice, binary judgment, and sorting tasks, the correctness criterion corresponds to strictly matching the objec- tive ground truth label sequence. To handle verbose model outputs (c.g., when model outputs reasoning alongside the option), we employ a two-step normalization process: first, we attempt rule- based parsing to extract the option label; if this fails, we lightweight LLM call solely to extract the predicted label (e.g., "Option A") without altering the semantic content. The extracted label is then compared against the ground truth using exact string matching.
Semantic Similarity for Open-Ended Tasks. For open-ended questions where exact string matching is too rigid, we employ semantic similarity metric. We consider prediction correct
19

---

Published as a conference paper at ICLR 2026
if the cosine similarity between the Sentence-BEi BERT embedding of the predicted answer and the ground text Y/ exceeds a pre-defined threshold т. This is formally expressed as:
Correct(O)) (sim vz) 7)
(2)
where 1(-) is the indicator function returning if the condition is satisfied and 0 otherwise, and sim(...) denotes the cosine similarity computed using a pre-trained Sentence-BEil model. In our experiments, the threshold 0.6 ensuring that correct but phrased-differently answers accepted while irrelevant hallucinations are rejected.
C EXPERIMENT DETAILS & ADDITIONAL RESULTS C.I BASELINE MODELS
We evaluate total of 20 state-of-the-art Multimodal Large Language Models (MLLMss), encom- passing both open-source and closed-source models with diverse model sizes and capabilities. All models capable of processing visual inputs and assessed under a unified evaluation pipeline. The list of baseline models used in UrbanFeel includes:
GPT (OpenAl 2024): adopt the latest version of GPT-4o as the representative model from the GPT series.
2. Gemini Comanici et al. 2025 Gemini-2.5-Pro is selected as the representative of the Gemini family.
3. Qwen Yuu et al. 2025 We evaluate three variants of Qwen2.5-VL, including the 3B, 7B, and 72B checkpoints.
4. InternVL (Zhu et al. language expert models.
2025):
IntemVL3-2B and Intern VL3-8B are included as vision-
5. MiniCPM (Yao et al.
2024):
MiniCPM-V-2.6 is used to represent lightweight MLLMs.
6. DeepSeek (Wu et al.
2024b): Both DeepSeek- VL2-tiny and DeepSeek-VLL V1.2 are included
explore performance scaling trends.
7. LLaVA (Li et al.
2024b): We include LLaVA-1.5 (7B) and LLLVA-1.66mmista as repre- sentatives of this popular open-source family.
8. Mistral (Jiang al.) 2023): -ittral-Smmll33-22------
evaluated a strong
language-focused bascline.
9. Gemma (Tea et al.
2025):
Both Gemma-3-4B and Gemma-3-27B checkpoints are as-
sessed.
10. Aya (Dash et al.
2025
The Aya-vision-8B checkpoint is included.
11. Phi (Abdin et al.
2024 We evaluate multimodal variants of Phi-3.5 and Phi-4.
Aria et al.
2024a):
We also include the open-source Aria model.
All models accessed via their official APIs released checkpoints, and evaluated using stan- dardized prompt structure and visual input protocol to ensure faireess and consistency. To ensure reproducibility, we the temperature to 0 and perform greedy decoding.
C.2 Do REASONING-AUGMENT MODELS OUTPERFORM BASELINES ON URBANFEEL?
To systematically assess the benefits of reasoning capabilities for urban perception tasks, we conduct a comparative analysis of various base MI.LMs and their aassoning-augmented counterparts across five sub-tasks of the UrbanFeel (SDM TCR, TSR, SCR, LP). As shown in Table 5 these tasks span a wide spectrum, from static scene understanding and cross-view matching to temporal semantic consistency reasoning and fine-grained local perception.
Experimental results indicate that reasoning augmented models (e.g., QVQ, GPT-o3, Gemini-2.5- Pro-thinking) generally perform better tasks emphasizing spatial understanding. For instance, in SPM and TCR- tasks that require scene consistency and localized judgment GPT-o3 achieves ac- euracy rates of 86.2% and 93.8%, respectively. In the subjective perception task (LP), QVQ brought
20

---

Published as a conference paper at ICLR 2026
Table 5: Quantitative comparison results of massoning-augmentn model and the non-reasoning- augmented model. The maximum value and the second largest value of model performance in each task are indicated by the bold and underlined text, respectively. Task names are abbreviated for brevity.
Model
SPM
TCR
TSR
SCR
LP
Qwen2.5VL-77
66.2
87.9
26.0
46.0
36.9
QVQ
68.3
76.4
17.5
24.5
48.7
GPT-4o
81.4
89.2
38.9
49.9
37.3
03
86.2
93.8
37.0
25.5
37.0
Gemini-2.55 Pro
80.3
95.4
52.1
56.5
49.0
Gemini-2.5-Prun 2.5-Pro-
75.9
96.3
39.5
24.4
42.6
Human
76.7
96.4
70.0
69.5
32.9
an 11.8% increase in accuracy compared to Qwen2.5-VL--72B, suggesting stronger alignment with human perception.
However, reasoning does not consistently lead to performance gains across all tasks. In the tem- porally ordered TSR task, the reasoning-augmmetted models exhibit varying degrees of performance degradation, which may be attributed to the extended reasoning span required when processing multiple images, thereby limiting the models' ability to effectively capture and model the relation- ships among these images. In the scene change- sensitive SCR task, several reasoning-an reasoning-augmenteed augmented models even show significant declines, with an average accuracy drop of 51.2% compared their non-reasoning counterparts. This suggests that reasoning models may overemphasize fine- tne-grained differences when facing abrupt scene transitions, thereby overlooking global semantic coherence and resulting in perceptual misjudgments.
C.3 Do MLLMS POSSESS ROBUST GENERALIZATION ACROSS DIVERSE CITIES?
Due to space limitations, this appendix provides the quantitative results MLLMs on different perception dimensions across the six cities mentioned in the main paper, under the Global Perception (GP) task without city identity intervention, as shown in Table 10 to Table 13
To assess model robustness across diverse urban environments. we analyzed performance variations at the city level, revealing two distinct generalization patterns. beautiful dimension (table 10) of global perception (GP task), leading closed-soures source models exhibit "inverse geographic bias") for instance, Gemini-2.5-Pa aligns more closely with human perception in Global South cities (aver- aging 67.1% than in the Global North (52.4%), suggesting reduced tendency to idealize Western aesthetics. Conversely, open-source models demonstrate severe city-specific overfitting. LLaVA- 1.5-7B achieves near-perfect accuracy in Washington (98.1%) 18) but drops drastically to 38.9% in Kuala Lumpur, indicating a reliance on US-centric training data rather than true perceptual general- ization.
This geographic variance extends to objective recognition tasks (PCR and DEE), distinguishing ro- bust generalists from brittle systems. As shown table and lixel-level detection (PCR), while models like Aria maintain consistency (e.g., 53% in Washington vs. 47% in Tolyatti), others suffer catastrophic collapse; notably, Aya-vision-88 plummets from
to 6%. Furthermore, the Dom- inant Element Extraction (DEE) task reinforces the "inverse bias" phenomenon even in objective settings: GPT-4o surprisingly achieves 70% accuracy in Cape Town versus just 35% in Washington. These sharp contrasts underscore the critical value of UrbanFeel's multi-city framework, aggre- gate metrics frequently mask significant regional failures that only granular geographic evaluation can reveal.
C.4 ADDITIONAL CITY IDENTITY INTERVENTION RESULTS
In Discussion section the main paper, we analyzed how different models' perceptions of the Beautiful and Wealthy dimensions shift under hypothetical city identity interventions. Due to space limitations, this appendix provides the quantitative results of MLLMs on different perception di- mensions across the six cities mentioned in the main paper, under the Global Perception (GP) task
21

---

Published as a conference paper at ICLR 2026
Yes
Beautiful
0.2
0.2
0.6
Cape Town Kuala Lumpur Mexico City
Milan
Paris
Washington No Pretend City
1.0
Yes
0.6
Lively
0.2
0.6
1.0
No.
Cape Town Kuala Lumpur
Mexico City
Milan
Paris
Washington Pretend City
Yes
0.6
Safe
0.2
0.2
Cape Town Kuala Lumpur
Mexico City
Milan
Paris
Washington No Pretend City
Yes
Wealthy
0.2
0.2
0.6
No
Cape Town
Kuala Lumpur Mexico City
Milan
Paris
Washington No Pretend City
Phi-4
LLaVa-1.-mmistra-77
DeepSeck-VL22
Qwen2.5-VL-72B
GPT-4o
Figure 10: Quantitative comparison of different models under the assumed city identity setting, "Yes" indicates the proportion of positive evaluations made by the model for the given perceptual dimen- sion, while "No" represents the proportion of negative evaluations.
without city identity intervention, as shown in Table I0 Table 13 Fig. 10 supplements the exper- imental results of City Identity Intervention under the Lively and Safe dimensions.
Our findings reveal significant diter-dimeniional differences in model sensitivity identity prompts. For the Lively dimension, most models exhibit a consistent tendency toward negative judgments. GPT-4o, Qwen2.5-VL-722 and DeepSeek- VL2 remain largely stable before and after city identity assignment, suggesting minimal perceptual bias. In contrast, Phi-4 shows notable decline in posi- tive evaluations dropping from around 60% below 30% after identity intervention. Interestingly, VLaV6- demonstrates the opposite trend, labeling almost all images "Lively," indicating high susceptibility to identity cues.
For the Safe dimension, most models maintain a high rate of positive judgments regardless of city identity, suggesting more robust safety perception. The only exception LLaVA-1.6, which shows marked decrease when the identity "Mexico City" is assigned-poteneeee related stereotypes learned from training data.
reflecting latent safety-
C.5 ADDITIONAL. DETAILS ON REASONING ABLATION STUDY
This section provides the experimental details and supplementary data for the reasoning ablation study discussed in Section 5.4 As noted in the main text, compared three prompting strategies to evaluate the efficacy of explicit reasoning in the Temporal Sequence Reasoning (TSR) task.
The specific contents of the prompts used for each strategy illustrated in Figure 11 are designed as follows:
The strategies
Direct Sorting (PO): A concise prompt asking only for the final chronological sequence
(e.g., "[Image A Image B > ...!"") without intermediate reasoning steps.
22

---

Published as a conference paper at ICLR 2026
Direct Sorting Prompt
Question: You are given four street-view images (Image A, C, and D), each captured at a different point time from the some location. These images reflect different stages of urbon development. chronological order of the images, from the eari lest captured (oldest)
Your task determine the
latest captured (most recent).
Please respond only with the image arder in this format:
[Image X -.. Image y
Image Z Image W]
General CoT Prompt
Question: are given streat-view images (Image
b), each at e different point
time from the same location. These reflect different stages of urban development. Your task of the images from the earliast captured (oldest) to the latest
determine the correct chronological order
captured (most recent).
To complete this task, first analyze each image based on visual cues such as building construction, road quality, greenery. public infrastructure, and signs of madervization,
Please follow this format in your response:
1. Answer: List the image order from the earliest captured (oldest) to the latest captured (most recent).
using this format:
[Tmmage X Image
Image Image W]
2. Recsoning: Briefly explain why you chose this order, referring to the key urbon development features
you observed the images.
think step by step.
Re- Re-thinking Prompt
You are given four street-view images (Image A, C, and D) from the same location, each coptured of a different time. They show different stoges of urbon development. Your task determine the correct chronological order of the images, from the earllest captured (oldest) latest coptured (most recent).
Questions
Step 1
Initial order:
Propese initial order of the images from earliest to latest.
your decision on permanent features as building construction, infrastracture, long-torm
changes in greenery.
Ignore transient factors such as cars, pedestrions, on weather.
Step Consistency check:
Check whether any image in your order appears to reverse development time (e g. partly
demalished building after a fully completed one, or neully buift structure appearing before empty lct). If such cases adjust the order and briefly exploin why.
Finol response format:
Initial order: [Image X Image
Z Image W]
Checked order: [Image Image Image Image W]
Reasoning: short expleration referring to permonent features and any non-monotonic changes you considered when deciding the final earliest-to-lates* order.
Figure 11: Different prompt type of TSR ablation study.
. General CoT (P2): A structured prompt requiring the model to first list "permanent fea- tures" (buildings, infrastructure) and explicitly ignore "transient factors" (cars, weather) before deriving the order.
Re-Thinking (P3): A multi-step prompt where the model first proposes an initial order, then performs a "consistency check" for non-monotonic changes (e.g., reverse develop- ment), and finally outputs the corrected sequence.
Table lists the detailed accuracy scores for all seven evaluated models across the three strate- gies. The results confirm that while reasoning strategies (P2, P3) offer marginal gains for smaller models like Phi-4 (improving from 5.5% to 7.3%), they consistently degrade the performance of high-capacity models (GPT-4o, 1wee2.55-VL7225 compared to the Direct Sorting baseline.
To further illustrate the "over-reasoning" phenomenon, Figure I2 visualizes a representative fail- ure case. Despite correctly identifying some features, the General CoT prompt leads the model to hallucinate a "linear growth" narrative- narrative-specffccllly claiming that vegetation in Image A is "vis- ibly larger" than in Image D to justify later timestamp whille ignoring the definitive structural evidence of a new blue building in Image D. This confirms that verbose reasoning can induce con- firmation bias, overriding visual evidence with plausible-sounding but incorrect logical chains.
23

---

Published as a conference paper at ICLR 2026
Table 6: Full ablation results on the TSR task. Consistent with the discussion in Section 5.4 the Direct Sorting strategy yields the highest accuracy for all SOTA models.
Model
Direct Sorting
General CoT
Re-Thinking
Phi-4
5.5
4.1
7.3
InterVLL--83
4.6
3.2
5.0
Qwen2.5-VL-722
32.0
24.7
29.7
GPT-440
46.1
38.4
37.0
Gemini-2.5-Pro
52.5
52.1
52.1
03
60.3
59.8
60.7
Gemini-2.5 2.5-Pro- S3-Pr--Thinking
54.8
49.8
51.1
C BB
A
GT: [Image Image Image A Image DJ
Direct Sorting Result
[Image Image Image D Image A]
General CoT Result
Answer:
[Image
Image
Image D Image A]
Reasoning:
Image image represents earliest stage. area lorgely undeveloped, older,
buildings on the left and what appears to be on open or industrial lot, construction crane in the background is the first sign the major development to come. The overall infrastructure minimal.
Image image
complete, the large building the right still a concrete skeleton under active construction, with crone on top. The area where the future medion and landscaping will be is still just dirt and construction ground.
significant progress.
red
building
Image In this stage, the construction finished, All modern buildings complete, and the public londscaped
infrestructure, including
paved rood, sidewalks, this stage is the newly planted trees in the medion, which are still relatively small and have autumn foliage.
place. key indicator
Image This image shows the
later time after completion. The buildings and infrastructure are the are visibly larger, fuller, leaves This indicates that time has passed, allowing the vegetation to mature, representing the most developed and settled state of the four images.
Image
landscaped
green summer
Figure 12: faliure case of TSR tasks using different type of prompt. The red text indicates incorrect reasoning results from Gemini-2.5-pro.
C.6 STATISTICAL SIGNIFICANCE ANALYSIS
To validate the robustness of our findings regarding model performance gaps and human-level con- sistency, we conducted a formal statistical analysis based on the sample sizes reported Table 2. We calculated 95% Confidence Intervals (CIs) for accuracy using the Wilson score interval and performed two-propottoon z-tests to assess the significance of performance differences.
Table 7 summarizes the statistical comparison between Human evaluators and the best-performing model (Gemini-2.5-PP)) across two representative tasks: Temporal Sequence Reasoning (TSR) and Global Perception (GP).
24

---

Published as a conference paper at ICLR 2026
Table 7: Statistical significance analysis of performance gaps between Human evaluators and the best-performing MLLMs. The analysis reveals three distinct distinct capability regimes: Signifi- cant Inferiority in temporal reasoning (TSR), Statistical Parity in subjective perception (GP), and Significant Superiority in pixel-level detection (PCR).
Task
Sample Size (N)
Subject
Accuracy (%)
95% CT
Significance
Human Gemini-2.5-pro
70.0
(64.0, 76.0)
TSR
219
p < 0.01
52.1
45.5, 58.7
Human
66.6
[64.5, 68.7
GP
1,918
Not Significant
Phi-4
67.7
[65.6, 69.8
PCR
1,100
Human
21.2
[18.8, 23.6
p < 0.001
Qwen2.5-v1-72B
40.9
[38.0, 43.8)
Temporal Reasoning Gap (TSR): For the TSR task (N 219), the 95% CIs for Humans and Gemini-2..5Prr are clearly separated (Human: 76.0] vs. Gemini: (45.5,58.77] The non- overlapping intervals and a z-test (p < 0.01) indicate that the 17.9% performance difference is statistically significant, statistically supporting conclusion that current MLLMs achieve substan- tially lower accuracy than humans on long-range temporal ordering.
2. Human-Level Parity in Perception (GP): For the GP task (NV 1, 918), the large sample size yields narrow error margins (+s +2.1%) The overlapping CIs indicate that the accuracies of Gemini- 2.5-Pro and human annotators are statistically indistinguishable in this setting. This validates statement that state-of-the-art MILLMs have reached a level of consistency comparable to human annotators for general scene perception.
3. Surpassing Humans in detection (PCR): Interestingly, for the PCR task (N = 1, 100), models significantly outperform humans (40.9% 21.2%). The distinct separation of CIs ([8..0, 43.8) vs. [18.8, 23.6)) and z-test (p < 0.001) provide robust statistical evidence of this advantage. As discussed, this is likely because human evaluators struggle to identify subtle pixel-level changes obscured by the geometric distortions inherent in panoramic imagery, whereas MILLMs maintain high sensitivity to such fine-grained variations.
D CASE STUDY
In this section, we present illustrative examples of model responses and corresponding ground-truth labels across the distinct sub-tasks designed in UrbanFeel (Figure 13 to Figure (T). The exam- ples span wide range of perception categories -inlludin static scene understanding, temporal change understanding, and subjective enviromental consistency reveallnn both the strengths and limitations of current models in handling real-world urban dynamics.
antegories-incluuinng
D.1 QUALITATIVE ERROR ANALYSIS
Based on the qualitative breakdown of model reasoning chains across these tasks, identify four primary categories of failure modes that limit the performance of current MLLMs in urban contexts:
Over-reliance surface semantics over spatial invariance (Temporal Context), tasks requir- ing temporal context understanding, models often prioritize salient surface-level semantic features while neglecting spatial geometric invariance. For instance, in the Temporal Co-location Recogni- tion task (Figure [9), when an empty lot evolved into a developed residential block, models such as Mistral-Small and Aria incorrectly classified the pair as "Different Locations" solely due to the emergence of new buildings. This indicates deficiency in utilizing invariant cues-such road layout and curvature- recognize that the images depict the same geographic location despite drastic semantic shifts over time.
"Linear development" assumption in urban evolution and neglect of urban decay (Temporal Reasoning). Regarding temporal reasoning, models frequently exhibit "linear development" bias, often operating under the heuristic that newer or cleaner infrastructure always corresponds to a later timestamp. This leads to failures in accounting for urban decay complex maintenance cycles.
25

---

Published as a conference paper at ICLR 2026
In the sorting example shown in Figure 28 GPT-4o chronologically misplaced an image showing worn road markings before a pristine one, ignoring critical alignment cues like speed bumps. This limitation is further illustrated in the urban renewal scenario in Figure 29] Here, models consistently identified the intermediate demollition/constuutiinn phase as the earliest stage, placing it chronolog- ically before the original standing building. This reveals rigid "Tabula Rasa" heuristic assuming that any construction site represents the genesis of development-theree monotonic processes where established neighborhoods undergo decline renewal.
failing to recognize non-
Spattal-perspective misalignment (Static Spatial Perception). In static spatial perception, mod- els demonstrate significant difficulties in cross-view mapping, particularly between single-view and panoramic imagery. In the Single-to- Pogge-to-Paano Matching task (Figure 13). models like Aria failed to match a single-view crop to corresponding panorama. This failure suggests that models treat the geometric distortions inherent in panoramic projections as semantic differences rather than perspec- tival variations, revealing limitations performing spatial transformations on 360 imagery.
Hallucinated visual evidence in subjectíve reasoning (Subjective Perception). Finally, in sub- jective perception tasks, models occasionally hallucinate negative visual cues to justify conservative or biased classifications. For example, in the Local Perception task (Figure GPT-4o categorized a scene as "not wealthy" by citing non-existent "shuttered storefronts" " and "lack of investment," despite visual evidence of well well-maintained infrastructure. This reveals disconnect between the reasoning chain and the actual pixel data, where models generate plausible-sounding but factually incorrect evidence support high-level prior impression.
E LIMITATIONS & FUTURE WORK
While UrbanFeel provides a comprehensive benchmark for evaluating MLLMs in urban develop- ment understanding and human-centered perception, several limitations remain. First, although the dataset spans 11 cities across different continents, it suffers from geographic imbalance, with under- representation of regions such as Africa and South America. This may affect the generalizability of models in culturally sensitive subjective perception tasks. Second, the annotations for affective per- ception tasks are statically defined, potentially failing to capture the temporal diversity and evolving nature of human opinions. Moreover, despite covering 18 years of visual urban change, Urban- Feel lacks explicit causal labels or structured socio-enving entrronnniion metadata, limiting its capacity to support deeper reasoning about the underlying drivers of urban transformation.
In future work, we plan to address these limitations through a concrete roadmap focused on geo- graphic inclusivity and causal depth:
Geographic Expansion and Cultural Grounding, We are actively expanding UrbanFeel to include cities with emerging temporal coverage, focusing on urban development tasks to mitigate regional representation bias. Furthermore, to address cultural bias in subjective perception, future iterations will aim diversify the annotator pool to include local residents and refine annotation guidelines with region-specific examples. We plan to report perception scores in stratified manner, ensuring that labels like "safety" or "beauty" reflect locally grounded interpretations rather than a single cultural perspective.
Towards Causal Spatiotemporal Reasoning. While UrbanFeel currently emphasizes visually veri- fiable changes, we aim to bridge the gap between pixel-level observation and socio-cconomic causal- ity. Inspired by recent trends in urban analytics, we plan to align strect-view segments with external urban datasets-sun
as land-use layers, POIs, and census statistics- enable models to contextu- alize physical changes. For cities with accessible records, we will introduce coarse- coarse-grained -rained causal event annotations (e.g., new transit line openings, policy-driven redevelopment projects) support reasoning about the drivers of urban evolution. Building on this, we will also explore how Urban- Feel tasks can be composed into multi-turn, scenario- based evaluations that more closely resemble real planning and policy workflows.
F USE OF LARGE LANGUAGE MODELS
During the preparation of this manuscript, Large Language Models (LLMs) were used solely for grammar checking and language polishing. The use of L.LMs was strictly limited improving
26

---

Published as a conference paper at ICLR 2026
the clarity, readability, and overall presentation quality of the text. All aspects related to research idea, experimental code development, and result analysis were strictly conceived and completed independently by the authors.
27

---

Published as a conference paper at ICLR 2026
Table 8: Quantitative comparison of different MLLMs in the Pixel-level Change Recognition (PCR) task across cities. City names are replaced with abbreviations.
Model
Cam.
Cap.
Edi.
K.L.
Lis.
Mex.
Mil.
Par.
Tol.
Tyl.
Was.
DeepSeck-vi2-tiny
31
29
33
35
26
30
26
39
21
29
48
DeepSeek-vi2
33
35
25
44
27
54
MiniCPM-V 2.6-8B
38
49
35
40
34
34
44
30
42
53
37
Qwen2.5-v1-3B
35
35
31
41
22
Qwen2.5-x1-7B
39
30
31
33
25
41
37
47
26
33
50
Qwen2.5-v1-72B
36
45
48
42
L.LVA-115--B
22
20
21
31
21
38
16
38
17
41
ILaVA-v1..-mistrall-777
19
28
33
21
43
19
Intem
41
44
29
36
28
42
32
45
27
53
51
InternVL3-8B
34
33
27
34
24
40
19
47
23
43
48
Phi-3.5
23
18
21
31
21
40
15
40
17
41
Phi-4
30
27
27
34
25
40
19
43
25
40
43
Idefies3-8B
39
40
35
35
34
44
26
51
59
Mistral-Small-3311222
25
22
29
30
23
40
19
45
20
21
43
Aria
45
42
36
28
39
30
44
30
47
53
Aya-vision-8b
22
18
21
31
21
40
15
39
17
41
Gemma-3-4b
40
46
28
Gemma-3-27b
37
41
32
35
28
26
41
42
34
47
49
GPT-4o
30
41
30
40
29
46
43
49
43
48
47
Gemini-2.5-pro
30
35
25
33
35
38
34
54
34
42
41
Table 9: Quantitative comparison of different MILMs in the dominant element extraction (DEE) task across cities. City names are replaced with abbreviations.
Model
Cam.
Cap.
Edi.
K.L.
Lis.
Mex.
Mil.
Par.
Tol.
Tyl.
Was.
DeepSeek-v12-tnny
42
32
40
49
35
66
41
45
31
45
59
DeepSeek-v12
57
51
56
48
79
67
62
49
51
68
MiniCPM-V 2.6-8B
38
55
52
45
34
58
37
42
33
38
54
Qwen2.5-vl-3B
56
70
59
45
Qwen2.5-v1-7B
38
65
61
60
43
62
47
43
48
45
45
Qwen2.5-v1-72B
53
64
46
64
54
LLLVA-1.5-7B
23
21
29
29
18
33
23
27
20
29
33
ILaVA-vi.6-mistral-77
35
26
42
40
23
41
IntemVVL3-2B
35
39
60
41
48
71
50
51
46
41
70
VL3-MB
37
55
44
27
45
27
36
41
Phi-3.5
46
31
43
46
39
70
40
47
33
40
66
Phi-4
37
31
35
41
32
46
37
36
26
43
47
Idefics3-8B
41
60
47
37
49
34
46
56
42
Mistral-Small-3.1-2448
6
27
30
24
13
11
18
22
10
32
11
Aria
58
65
62
57
57
75
72
68
63
59
66
Aya-vision-Bb
8
29
25
24
13
12
16
21
10
26
9
Gemma-3-4h
40
54
33
28
48
26
48
41
Gemma-3-27b
42
65
66
41
41
50
45
41
50
58
41
GPT-4o
42
70
72
43
42
52
36
36
53
54
35
Gemini-2.5-ppm
52
76
72
66
55
71
59
62
70
61
53
28

---

Published as a conference paper at ICLR 2026
Table 10: Quantitative comparison results of different MILMs in the Beautiful dimension under GP tasks. The maximum value and the second largest value of model performance in each city are indicated by the bold and underlined text, respectively.
Model
Cape Town
Kuala Lumpur
Mexico City
Milan
Paris
Washington
PropSeek-vl2-tingg
57.5
44.4
62.9
69.5
69.7
94.3
DeepSeck-v12
55.0
72.2
74.2
48.8
63.6
71.7
MiniCPM-V 2.6-8B
42.5
66.7
54.8
39.0
39.4
20.8
Qwen2.5-v1-3B
57.5
88.9
71.0
53.7
65.2
81.1
Qwen2.5-vl-7B
72.5
66.7
69.4
64.6
63.6
81.1
Qwen2.5-v1-72B
75.0
50.0
69.4
64.6
65.2
84.9
LLaVA-1.5-7B
62.5
38.9
59.7
67.1
66.7
98.1
LLaVA-v1.6-mistrll
47.5
88.9
66.1
64.6
62.1
64.2
IntemnVL33-BB
60.0
72.2
72.6
69.5
68.2
96.2
InterVVL3-8B
70.0
88.9
79.0
56.1
59.1
75.5
Phi-3.5
47.5
83.3
66.1
25.6
50.0
41.5
Phi-4
57.5
83.3
62.9
68.3
62.1
88.7
Idefics3-8B
60.0
77.8
74.2
50.0
50.0
58.5
Mistral-Small-3.1122
70.0
88.9
79.0
56.1
59.1
75.5
Aria
57.5
66.7
64.5
72.0
63.6
88.7
Aya-vision-8h
57.5
72.2
64.5
64.6
68.2
83.0
Gemma-3-4b
57.5
72.2
61.3
35.4
51.5
54.7
Gemma-3-22 3-27b
60.0
55.6
71.0
41.5
56.1
66.0
GPT-4o
60.0
72.2
72.6
42.7
53.0
47.2
Gemini-2.5-pm
55.0
72.2
74.2
41.5
59.1
56.6
Quantitative comparison results of different MLLMs in the Lively dimension under GP tasks. The maximum value and the second largest value of model performance in each city are indicated by the bold and underlined text, respectively.
Table 11:
Model
Cape Town
Kuala Lumpur
Mexico City
Milan
Paris
Washington
DeepSeck-vi2-ting
70.0
83.3
80.0
34.1
56.7
55.6
DeepSeek-v12
75.0
55.6
68.3
29.3
55.2
35.2
MiniCPM-V 2.6-8B
67.5
38.9
66.7
41.5
46.3
46.3
Qwen2.5-vl-3B
77.5
77.8
63.3
42.7
68.7
38.9
Qwen2.5-vl-7B
77.5
66.7
55.0
36.6
56.7
40.7
Qwen2.5-v1-72B
77.5
66.7
53.3
28.0
49.3
37.0
LLaVA-1.5-7B
75.0
33.3
75.0
26.8
50.7
46.3
LLaVA-v1.6-mistrae
70.0
72.2
63.3
81.7
70.1
63.0
InternVL332BB
77.5
66.7
73.3
$2.4
73.1
InternVL3-8B
80.0
66.7
83.3
39.0
65.7
63.0
Phi-3.5
72.5
61.1
68.3
41.5
56.7
40.7
Phi-4
57.5
66.7
70.0
78.0
76.1
63.0
Idefics3-8B
80.0
72.2
68.3
40.2
53.7
37.0
72.5
72.2
70.0
25.6
47.8
46.3
Aria
75.0
77.8
80.0
50.0
67.2
48.1
Aya-vision-Sh
70.0
72.2
75.0
$7.3
80.6
57.4
Gemma-3-48
65.0
55.6
81.7
56.1
76.1
57.4
Gemma-3-271
77.5
55.6
73.3
31.7
59.7
44.4
GPT-4o
66.7
76.7
25.6
61.2
42.6
Ge5ini-2..5pro
82.5
61.1
75.0
37.8
68.7
46.3
29

---

Published as a conference paper at ICLR 2026
Quantitative comparison results of different MILMs in the Safe dimension under GP tasks. The maximum value and the second largest value of model performance in each city are indicated by the bold and underlined text, respectively.
Table 12:
Model
Cape Town
Kuala Lumpur
Mexico City
Milan
Paris
Washington
DeepSeek-Vo -in-tiiy
32.5
27.8
24.2
19.5
25.4
7.7
DeepSeck-v12
70.0
61.1
68.1
82.9
74.6
87.2
MiniCPM-V 2.6-8B
25.0
33.3
20.9
22.0
23.9
15.4
Qwen2.5-v1-3B
65.0
66.7
68.1
81.7
71.6
85.9
Qwen2.5-vl-7B
70.0
66.7
70.3
84.1
73.1
85.9
Qwen2.5-v1-72B
70.0
66.7
69.2
84.1
74.6
87.2
LLaVA-1.5-7B
38,9
30.8
32.9
43.3
42.3
LLaVA-v1.6-mistrll
70.0
55.6
69.2
79.3
73.1
84.6
IntemnVL33-BB
67.5
61.1
65.9
76.8
70.1
82.1
InterVVL3-8B
67.5
66.7
65.9
82.9
73.1
84.6
Phi-3.5
65.0
66.7
53.8
47.6
64.2
61.5
Phi-4
67.5
66.7
70.3
80.5
74.6
85.9
Idefics3-8B
75.0
72.2
60.4
78.0
67.2
85.9
Mistral-Small-3.1122
72.5
66.7
61.5
80.5
68.7
87.2
Aria
70.0
72.2
67.6
77.4
77.8
84.6
Aya-vision-8h
60.0
61.1
67.0
73.2
68.7
79.5
Gemma-3-4b
30.0
27.8
20.9
18.3
16.4
10.3
Gemma-3-22 3-27b
70.0
61.1
58.2
79.3
68.7
84.6
GPT-4o
65.0
66.7
59.3
74.4
68.7
76.9
Gemini-2.5-pm
75.0
66.7
64.8
79.3
73.1
80.8
Table 13: Quantitative comparison results of different MLLMs the Wealthy dimension under GP tasks. The maximum value and the second largest value of model performance in each city are indicated by the bold and underlined text, respectively.
Model
Cape Town
Kuala Lumpur
Mexico City
Milan
Paris
Washington
DeepSeck-vi2-ting
67.5
55.6
65.0
36.6
50.7
48.1 1
DeepSeek-v12
80.0
72.2
70.0
63.4
77.6
77.8
MiniCPM-V 2.6-8B
50.0
22.2
45.0
22.0
17.9
25.9
Qwen2.5-vl-3B
77.5
66.7
70.0
57.3
70.1
79.6
Qwen2.5-vl-7B
75.0
72.2
73.3
53.7
70.1
77.8
Qwen2.5-v1-72B
80.0
66.7
63.3
52.4
70.1
79.6
LLaVA-1.5-7B
77.5
61.1
71.7
67.1
71.6
64.8
LLaVA-v1.6-mistrae
70.0
72.2
61.7
52.4
64.2
61.1
InternVL332BB
75.0
72.2
58.3
47.6
67.2
74.1
InternVL3-8B
77.5
66.7
73.3
75.6
74,6
70.4
Phi-3.5
62.5
38.9
51.7
26.8
44.8
42.6
Phi-4
70.0
55.6
58.3
87.8
80.6
74.1
Idefics3-8B
67.5
50.0
50.0
40.2
32.8
68.5
70.0
66.7
73.3
41.5
62.7
75.9
Aria
60.0
61.1
66.7
52.4
71.6
72.2
Aya-vision-8h
82.5
72.2
76.7
68.3
79.1
68.5
Gemma-3-48
70.0
61.1
53.3
42.7
70.1
74.1
Gemma-3-271
66.7
64.7
56.8
57.9
81.2
86.5
GPT-4o
60.0
44.4
45.0
45.1
55.2
72.2
Gemini-2..5pro
77.5
72.2
71.7
54.9
74.6
79.6
30

---

Published as a conference paper at ICLR 2026
SSP:
SPM [Judgement]
Question
Determine whether panorama and same location or different location.
single view image are from the
Option: (A) Same Location
Different Location
R
(Aria) Answer: (B)
Different Location
Sistral-Small3.1-24B-nntstt Answer:
(B)
Different Location
(DeepSeek-VL2) Answer:
(A) e Same Location
Ground Truth:
(A)
Same Location
Figure 13: A question case of the Single-to-Pano Matching(SPM) task in UrbanFeel responses
from Aria, Mistral-Small-33 1...aalll-.1------- DeepSeek-V122
31

---

Published as a conference paper at ICLR 2026
SSP:
SPM [Judgement]
Question
Determine whether panorama and same location or different location.
single view image are from the
Option: (A)
Same Location
B
Different Location
(MiniCPM-V-2_6) Answer:
(A) Same Location
(InternVL3-8BB) Answer:
(B)
Different Location
(QvVn2.5-VL-77--nnsrrrcc) Answer:
B Different Location
Ground Truth: (B)
Different Location
Figure 14: A question case of the Single-to-Pano MStch)ng(SPMD task in UrbanFeel responses
from MiniCPM-V-22.6, IntemVLL3-8B, Qwven2.5-VL-77-Inniiin
32

---

Published as a conference paper at ICLR 2026
SSP:
CR [Judgement]
Question
Determine whether single view image and single view image 2 are from the same street different street.
Option: (A)
Same Street
(B)
Different Street
(Idenfics3-BB-Llamma] Answer:
(A)
e Same Street
(Phi.3.5vvision-insttrct) Answer:
(B) #
Different Street
(Gemma-3-27B-it) Answer:
(B)
Different Street
Ground Truth: (B) Different Street
Figure 15: A question case of the Co-location Recognition(CR) task in UrbanFeel responses from -an--lla3,,- Phi-3.5-visse -iiioni-iiion-iistruuc,, Gemma-3-27B-it
33

---

Published as a conference paper at ICLR 2026
SSP:
CR [Judgement]
Question
Determine whether single view image and single view image 2 are from the same street different street.
Option: (A)
Same Street
(B)
Different Street
(Aya-vision-8B) Answer: (A) Same Street
(Llava-1.5-7B-HF) Answer: (A) Same Street
(Qven225-VL-77B-nnsttccc) Answer: (B) Different Street
Ground Truth: (B) Different Street
Figure 16: A question case of the Co-location Recognition(CR) task in UrbanFeel responses from Ava-vision-SB, 1lava--.5-7B-HF:: Diven2.-vVL-22B-Immiii IIsstrccc
34

---

Published as a conference paper at ICLR 2026
SSP:
DEE [MC]
Question
Which element occupies the most visible area in the
panorama
Option: (A)
Road (B)
Vehicle (C)
Sky
Vegetation
Building
(GPT-4o) Answer:
(D) Vegetation
(Gemini-2.5ppro) Answer:
(A)
Road
(Human) Answer:
(A) Road
Ground Truth:
Road
Figure 17: A question case of the Dominant Element Extraction(DEE) task UrbanFeel re- sponses from GPT-4o, Gemini-2.5-pro, Human
35

---

Published as a conference paper at ICLR 2026
SSP:
DEE [MC]
Question
Which element occupies the most visible area in the
single view image
Option: (A)
Road (B)
Vehicle (C)
Sky
Vegetation
Building
(InternVL-3-2B) Answer:
(C) Sky
(Qwenn2.-VL--BB-Insttrct))) Answer:
(C)
Sky
(DeepSeek-VL2) Answer:
(E) Building
Ground Truth:
(E)
Building
Figure 18:
A question case of the Dominant Element Extraction(DEE) task UrbanFeel re-
sponses from IntemVVL-3-2B, 1i--n--5-VV--B--ssrrrr -e.
36

---

Published as a conference paper at ICLR 2026
TCU:
TCR [Judgement]
Question
Determine whether panorama and
panorama are from the same
location different location.
Option: (A)
Same Location
B
Different Location
2012
2021
R
Answer: (B)
Different Location
Sisstal-Smaall-31-44-----tttttt Answer:
(B) Different Location
(Qven2..W..-77--nnssrrttt)) Answer:
(A) Same Location
Ground Truth: (A)
Same Location
Figure 19: A question case of the Temporal Co-location Recognition(TCR) task in UrbanFeel responses from Aria_cut, -.a.ea...-ll---------a-.. Dwweb2.5-VL-72B-nnntit
37

---

Published as a conference paper at ICLR 2026
TCU:
TCR [Judgement]
Question
Determine whether panorama and location different location.
panorama are from the same
Option: (A)
Same Location
(B)
Different Location
2023
(DeepSeek-VL2-Tiny) Answer:
BB)
Different Location
(InternVL-BB) Answer:: (B)
Different Location
Misstrr-SSmalll3322224--ittttttd Answer:
(A)
e Same Location
Ground Truth: (A)
Same Location
Figure 20: A question case of the Temporal Co-location Recognition(TCR, task in UrbanFeel responses from V1.2-Tiny, Intern InternVL-8BB, VL-8B, 3aaiau--malll311-22 1-24B-Instruch
38

---

Published as a conference paper at ICLR 2026
TCU:
PCR [MC]
Question
Based on the changes between and
which urban element has undergone
the most significant transformation in terms of city development?
Option: (A)
Road (B)
+
Vehicle
(C)
Sky
Vegetation
(E)
Building
2011
2023
(Aya-vision-8B) Answer:
Building
(Gemini-2.5-pro) Answer:
(A)
Road
(QvBnn.5 Answer:
B
Vehicle
Ground Truth: (B)
Vehicle
Figure 21: A question case of the Pixel-level Change Recognition(PCR) task in UrbanFeel sponses from Aya-vision-8B, Gemini-2..5pro, Qwven.55-VL-72B-nnsrrr
39

---

Published as a conference paper at ICLR 2026
TCU:
PCR [MC]
Question
Based on the changes between and
which urban element has undergone
the most significant transformation in terms of city development?
Option: (A)
Road (B) +
Vehicle
(C)
Sky
Vegetation
Building
2009
2019
(Human) Answer:
EE)
Building
(DeepSeek-VL2) Answer:
(A)
Road
(Gemini-2.5-pro) Answer:
Road
Ground Truth:
(A)
Road
Figure 22: A question case of the Pixel-level Change Recognition(PCR) task in UrbanFeel sponses from Human, DeepSeek-VL2, Gemini-2.5-pro
40

---

Published as a conference paper at ICLR 2026
TCU:
SCR [MC]
Question
2
Based on the changes between and change between the two time periods?
Which category best describes the
Option:
(A)
Road Change
(B) Vegetation Change
(C) Building Facade Change
(D)
Mobility-Related Change
EE)
Building Presence Change
2007
(Arial) Answer:
(B)
Vegetation Change
(Gemma-3-27BB) B-it) Answer:
(B)
Vegetation Change
(GPT-40) Answer:
Building Presence Change
Ground Truth:
(E)
Building Presence Change
Figure 23: A question case of the Scene-level Change Recognition(SCR) task in UrbanFeel sponses from Arial, Gemma-3-278-it, GPT-40
41

---

Published as a conference paper at ICLR 2026
TCU:
SCR [MC]
Question
2
Based on the changes between and change between the two time periods?
Which category best describes the
Option:
(A)
Road Change
(B) Vegetation Change
(C) Building Facade Change
(D)
Mobility-Related Change
E Building Presence Change
2023
(MiniCPM-V-2_66 Answer:
(A)
Road Change
(Phi-3..vision-nnsstuut))) Answer:
Building Presence Change
Miistral-Smalll33-444---nstuu Answer:
Building Presence Change
Ground Truth: (C)
Building Facade Change
Figure 24: A question case of the Scene-level Change Recognition(SCC) task in UrbanFeel sponses from MiniCPM-V-2 6, snl.:35-viiioo-nnsteee Mistral-Small-3. -itrrl--Salll33.-244--ttt-t I-2tt--nstruuc
42

---

Published as a conference paper at ICLR 2026
TCU:
FSI [MC]
Question
Given an image as reference, which the following images most likely shows the same location after city development?
B
2008
(Llava-1.5-7B-HH) Answer: Image
A
(Ideiiss3-8B-Llamaa] Answer:
Image
C
(Qven2.--)L772-nnnstucc) Answer:
Image
Ground Truth: Image
D
Figure 25: A question case of the Future Scene Identification(FSD) task in UrbanFeel responses
from Llava-1.5 1.5--- Catfesss--B-Lamaad Swnen..5vLL72B--nsteee
43

---

Published as a conference paper at ICLR 2026
TCU:
FSI [MC]
Question
Given an image as reference, which the following images most likely shows the same location after city development?
2008
(DeepSeek-VL2-Tiny) Answer: Image
(InternVL3-8BB) Answer:
Image
A
(Gemma-3-27B-it) Answer:
Image
B
Ground Truth:
Image
B
Figure 26: A question case of the Future Scene Identification(FS)) task in UrbanFeel responses
from DeepSeek VL2-Timy. Intern VL3-8B, 3-27-it
44

---

Published as a conference paper at ICLR 2026
TCU:
TSR [Sorting]
Question
Please sort the four street-view images in chronological order from least to most developed based on visual cues like buildings, roads, greenery, and modern infrastructure.
A
B
D
(Aya-vision-8BB Answer:
C
D
(Gemma-3-27B-it) Answer:
D
B
(Qven22.WWL-778-nnsttuc)))) Answer:
B
Ground Truth: C
B
Figure 27: A question case of the Temporal-Sequence Reasoning(TSRR) task UrbanFeel re- sponses from Aya-vision-8B, Gemmu-3-27B-it, Qwwnn.5-VLL-72B-hime
45

---

Published as a conference paper at ICLR 2026
TCU:
TSR [Sorting]
Question
Please sort the four street-view images in chronological order from least to most developed based on visual cues like buildings, roads, greenery, and modern infrastructure.
B
D
(DeepSeek-VL2) Answer:
A
B
D
(GPT-40) Answer:
D
C
B
A
(Gemini-2.5-pro) Answer:
C
Ground Truth:
C
D
B
Figure 28: A question case of the Temporal- SSeuueee Reasoning(TSR) task UrbanFeel re- sponses from V1.2, GPT-4o, Gemini-2.5-pro
46

---

Published as a conference paper at ICLR 2026
TCU:
TSR [Sorting]
Question
Please sort the four street-view images in chronological order from least to most developed based on visual cues like buildings, roads, greenery, and modern infrastructure.
B
CINERA
2015-07
2009-02
C
D
2018-05
2022-05
(DeepSeek-VL2, Answer:
A
B
(GPT-40) Answer:
A
C
D
(Gemini-2.5-pro) Answer:
A
B
D
Ground Truth:
B
D
Figure 29: A question case of the Temporal Sequeee Reasoning(TSR) task UrbanFeel re- sponses from V1.22, GPT-4o, Gemini-2.5-pro
47

---

Published as a conference paper at ICLR 2026
TCU:
GP [Judgement]
Question
Based on the image, please judge whether the city appears from a human perspective.
beautiful or not
Option: (A)
Yes, it is beautiful
(B) No, it is not beautiful
(Arial Arial_cct) Answer:
(A)
Yes, is beautiful
(Gemma-3-4B-it) Answer:
(
No, it is not beautiful
((defics3-BB-Llamaa] Answer:
(A) Yes, it is beautiful
Ground Truth: (A)
Yes, it is beautiful
Figure 30: A question case of the Global Perception(GPP) task in UrbanFeel responses from Ar-
cut, Gemma-3-44 3-4B-ir Idenfics3-8B Berfffsss----LLAe 3l-maa
48

---

Published as a conference paper at ICLR 2026
TCU:
GP [Judgement]
Question
Based on the image, please judge whether the city appears from a human perspective.
wealthy or not
Option: (A)
Yes, it is wealthy
(B) No, it is not wealthy
(Aya-vision-8B) Answer:
(A)
Yes, it is wealthy
(MiniCPM-V-2_6) Answer:
(B)
No, it is not wealthy
(Phi:3..viision-insttuct))) Answer:
(
No, it is not wealthy
Ground Truth: (B)
No, it is not wealthy
Figure 31: A question case of the Global PG)) task in UrbanFeel responses from Aya-
vision-8B, MiniCPM --22--66,. Phi-3.5-vision-insttiee
49

---

Published as a conference paper at ICLR 2026
TCU:
GP [Judgement]
Question
Based on the image, please judge whether the city appears human perspective.
safe or not from a
Option: (A) Yes, it is safe
(B) No, it is not safe
(DeepSeek-VL2-Tiny) Answer:
(B) No, it is not safe
(Qwen2.5vVL-BB-nnstrcc)) Answer:
(B)
No, it is not safe
(Llava-1.5-7B-HFFA Answer: (A)
Yes, it is safe
Ground Truth: (A) Yes, it is safe
Figure 32: A question case of the Global PG)))ption(GP) task UrbanFeel responses from
DeepSeek VL2 VL2-Timy, Qwen2.5-VL-7B -nsstrcct Alavv--..5-7B---
50

---

Published as a conference paper at ICLR 2026
TCU:
GP [Judgement]
Question
Based on the image, please judge whether the city appears a human perspective.
lively or not from
Option: (A)
Yes, it is lively
(B) No, it is not lively
(InternVL3-8B)) Answer:
(A) Yes, lively
(Gemini-2.5-pro) Answer:
(A) Yes, it is lively
GGTT-4) Answer:
(A) Yes, is lively
Ground Truth:
(B) No, it is not lively
Figure 33: A question case of the Global Perception(GP) task UrbanFeel responses from VLternVL3-8B, 2emini-2.5-prr, GPT-4o
51

---

Published as a conference paper at ICLR 2026
TCU:
LP [Open]
Question
Based on the image, please judge whether the city appears beautiful not from human perspective and find some visual factors that contribute to it.
(Gemini-2.5-pro) Answer:
1. Harmonious Blend of Nature and Urbanism: The street is lined with rows of healthy, lush green trees. This greenery provides a pleasant, organic contrast to the hard lines the buildings and asphalt, making the urban environment feel more alive, breathable, and visually appealing.
Pleasant Atmosphere and Lighting: The bright sunlight and clear, vibrant blue sky create a cheerful and positive atmosphere. This type of weather often enhances the perception of place, making it seem cleaner, more energetic, and more inviting.
3. Interesting Architectural Mix: The image showcases visually engaging contrast between different architectural styles. On the left, there is classic red- brick building with arched windows, while the right side features more modern, large-scale structures. This juxtaposition adds character and depth to the streetscape.
Ground Truth:
tidy and aesthetically pleasing building facade
2. Good sky visibility
3. Neat and dense greenery
Evaluation:
Correct (0.723)
Figure 34: A question case of the Local Perception(LP) task in UrbanFeel responses from Gemini- 2.5-pro
52

---

Published as a conference paper at ICLR 2026
TCU:
LP [Open]
Question
Based the image, please judge whether the city appears wealthy or not from human perspective and find some visual factors that contribute to it.
(GPT-40) Answer:
1. Building Condition: The buildings appear older and somewhat worn, suggesting a lack of recent investment.
2. Storefronts: Closed shuttered storefronts indicate limited commercial activity or economic challenges.
The road and public area appear to be relatively well- maintained, but lack of luxury features such as premium sidewalks, exclusive shops, vibrant landscaping.
3. Street Maintenance:
Ground Truth:
Tall and densely packed buildings indicate a high level of urbanization
2. Well-maintained pedestrian walkways
3. Comprehensive road signage and infrastructure development
Evaluation:
x
False (0.523)
Figure 35: A question case of the Local Perception(LP) task in UrbanFeel responses from GPT-4o
53

---

Published as a conference paper at ICLR 2026
TCU:
LP [Open]
Question
Based on the image, please judge whether the city appears safe or not from a human perspective and find some visual factors that contribute to it.
(DeepSeek-VL2)) Answer:
The street appears narrow and congested, which can make it difficult for pedestrians to navigate safely
2. There are multiple cars parked along the street, which could obstruct visibility and create potential hazards for both drivers and pedestrians.
3. The road surface shows wear, lacks lane markings, and the road signs unclear, creating potential driving hazards
Ground Truth:
1. The high walls on both sides of the road obstruct visibility, posing safety hazards
2. The streetlight poles appear somewhat outdated, and the distribution of electrical wires is chaotic
3. The road surface shows some wear, lacks lane markings, and the road signs are unclear, creating potential driving hazards
Evaluation:
Correct (0.691)
Figure A question case of the Local Perception(I.P) task in UrbanFeel responses from DeepSeek-Viz VL2
54

---

Published as a conference paper at ICLR 2026
TCU:
LP [Open]
Question
Based the image, please judge whether the city appears lively not from a human perspective and find some visual factors that contribute to it.
(Qwen2..-VL-72B-Instuu) Answer:
1. Presence of People and Activity: There are several people visible in the image, walking across the crosswalk and along the sidewalks. This indicates active pedestrian movement, contributing to lively atmosphere.
The image shows market stalls with vendors and customers, suggesting a bustling market scene. The presence these stalls adds to the vibrancy and activity of the area.
2. Market Stalls and Vendors:
3. Traffic and Vehicles: There are multiple vehicles, including cars, vans, and trucks, parked and moving along the street. This indicates a level of traffic activity, which is a sign of a busy and lively urban environment.
Ground Truth:
1. There many pedestrians
2. There are many vehicles
There are many shops, and there are also many parked vehicles around, creating lively atmosphere
4. The buildings consist of multiple residential blocks
Evaluation:
Correct (0.735)
Figure 37: A question case of the Local Perception(LP) task in UrbanFeel responses from Qwwen.5-vVL-22--nnnnt
55

---

Published as a conference paper at ICLR 2026
TCU:
CP [MC]
Question
and show the same place, one from 2010 and one from 2020. Please determine whether the scene has become more beautiful, less beautiful, stayed the same visually over time.
Option: (A)
More beautiful
(B) Less beautiful
(C)
No change
2010
(Qwen2.5-VL-3B-nnsttrcl) Answer:
(A)
More beautiful
(Arial) Answer: (A)
More beautiful
(DeepSeek-VL2) Answer:
(C)
No change
Ground Truth:
(C)
No change
Figure 38:
A question case of the Comparative Perceptual analysis(CP) task in UrbanFeel re-
sponses from Qween.5--VL-BB-nnsrrrre Arial, DeepSeek-Vis VL2
56

---

Published as a conference paper at ICLR 2026
TCU:
CP [MC]
Question
and show the same place, one from 2007 and one from 2018. Please determine whether the scene has become more wealthy, less stayed the same visually over time.
wealthy,
Option: (A)
More wealthy
(B)
Less wealthy
(C)
No change
2007
2018
(Aya-vision-8B) Answer:
(C)
No change
(Gemma-3-27BBi) Answer:
(A)
More wealthy
(GPT-40) Answer:
(A)
More wealthy
Ground Truth:
(A)
More wealthy
Figure 39; A question case of the Comparative Perceptual analysis(CP) task in UrbanFeel re- sponses from Aya-vision-8B, Gemmm-3-27B-it, GPT-4o
57

---

Published as a conference paper at ICLR 2026
TCU:
CP [MC]
Question
and show the same place, one from 2008 and one from 2023. Please safe, stayed the
determine whether the scene has become safer, less same visually over time.
Option: (A)
Safer
(B)
Less safe
(C)
No change
2008
2023
(MiniCPM-V-2_6) Answer: (A)
Safer
Answer:
(C)
No change
(Human) Answer:
(B)
Less safe
Ground Truth:
(B)
Less safe
Figure 40: A question case of the Comparative Perceptual analysis(CP) task in UrbanFeel re- sponses from MiniCPM-V-2.6, 1awa--/..-miistrl-777 Human
58

---

Published as a conference paper at ICLR 2026
TCU:
CP [MC]
Question
and show the same place, one from 2009 and one from 2024. Please determine whether the scene has become more lively, less stayed the same visually over time.
lively,
Option: (A)
More lively
Less lively
(C)
No change
2009
2024
(DeepSeek-VL2-Tiny) Answer:
(B) Less lively
((defiss3-BB-Llama)) Answer:
(B)
Less lively
(Gemini-2.5-pro) Answer:
(A)
More lively
Ground Truth:
(A)
More lively
Figure 41: A question case of the Comparative Perceptual analysis(CP) task in UrbanFeel re- sponses from DeepSeek-VLL--1 Lddenfiss3-B--Llmmee Gemini-2.5-pro
59


Reviews:
[{"review_id": "1eGV6lkt6o", "reviewer": "Reviewer_tZ4v", "summary": "The paper proposes UrbanFeel, a novel and comprehensive benchmark designed to evaluate MLLMs on the crucial but underexplored tasks of temporal evolution and subjective perceptual understanding of urban scenes from a human perspective. By focusing on time-variant analysis, the work addresses a key limitation of existing urban environment benchmarks. The authors present a structured task set and an initial evaluation. The focus on time-variance and human-centric perception makes this a unique and potentially valuable contribution to the MLLM communities.", "questions": "1. Given that the temporal aspect is the paper's core innovation, please significantly elaborate on the comparison with existing dynamic understanding and subjective perceptual understanding of street view.  A dedicated sub-section in Related Work contrasting task types is highly recommended.\n\n2. Please provide a more detailed analysis of the model performance, e.g.,  a qualitative breakdown of typical successful and unsuccessful reasoning chains of thinking models, and a dedicated, in-depth error analysis section categorized by the type of failure. This is critical for the paper's utility to the MLLM community.\n\n3. Please clarify the specifics of data acquisition. If the data is derived from commercial sources, please outline the measures taken to address potential licensing/copyright issues (the \"potential risks\") for open-sourcing the dataset. Furthermore, what additional detailed information will be provided alongside the release to fully document the benchmark construction process?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The core problem of analyzing time-variant street-level data for human-centric perceptual understanding is interesting and currently under-addressed in the MLLM community. This focus on \"change over time\" is timely and relevant.\n\n2. The paper is very well-structured and clearly written. The task design and evaluation methodology are presented in a logical manner, making the work easy to follow.", "weaknesses": "1. Although the paper repeatedly frames the benchmark goal around assessing the models' generalization capability, the experimental results and subsequent discussion dedicated to proving or analyzing this specific aspect are surprisingly sparse. More focused experiments or discussion are needed here.\n\n2. The discussion on highly related existing work needs substantial improvement. Specifically, the authors should include and contrast their work with recent efforts like Visual Chronicles at ICCV 2025 and relevant research from the urban planning domain that studies temporal changes and subjective perceptual understanding of street view data. This is essential to fully justify the novelty.\n\n3. Key illustrative elements, specifically Figures 2 and 3, lack sufficient detail and are difficult to read at standard resolution. Improving the clarity of these figures is necessary for the reader to fully understand the task design and examples.\n\n4. More comprehensive details are required regarding the data collection process, particularly the source (e.g., if commercial street view data like Google Streetview data was used). Potential licensing/ethical/legal risks must be explicitly addressed, and more thorough documentation of the benchmark construction process is necessary for credibility and reproducibility.\n\n5. The analysis section, particularly the error analysis and the investigation into the reasoning process of the tested models, should be significantly deepened to provide greater insight into MLLM capabilities and shortcomings on this benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WW28lS5Erz", "reviewer": "Reviewer_2E4Z", "summary": "The paper presents UrbanFeel, a comprehensive benchmark designed to assess the performance of Multimodal Large Language Models in urban development and subjective perception tasks. It spans three key dimensions: Static Scene Perception, Temporal Change Understanding, and Subjective Environmental Perception, leveraging over 14,300 questions derived from multi-view and multi-temporal street-view imagery from 11 global cities. The evaluation of 20 MLLMs reveals significant gaps in spatial-temporal reasoning and the alignment of models with human perception, particularly in tasks requiring long-term urban evolution understanding and subjective environmental evaluations.", "questions": "1. How do the authors plan to address the geographic imbalance in future versions of UrbanFeel, and how will they ensure better cultural representation in subjective perception tasks?\n2. Would it be possible to include external metadata or causal labels in future releases to help improve the interpretability and reasoning capabilities of models trained on UrbanFeel?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. UrbanFeel introduces of a comprehensive benchmark, going beyond traditional benchmarks by integrating spatial, temporal, and subjective perception tasks, providing a holistic evaluation framework for urban scene understanding.\n2. UrbanFeel defines 11 tasks across three dimension, Static Scene Perception, Temporal Change Understanding, and Subjective Environmental Perception. This framework allows for an in-depth assessment of MLLMs, examining not only their ability to recognize static features in urban scenes but also their capacity to reason about urban changes over time.\n3. High-Quality Data and Methodology: The dataset consists of 14.3K carefully curated questions, covering a broad range of urban environments and spanning over 15 years.\n4. The paper comprehensively evaluates 20 state-of-the-art MLLMs, providing detailed insights into their strengths and weaknesses across the benchmark’s tasks.", "weaknesses": "1. Despite the paper claims to include cities from diverse regions and covers underrepresented regions like Africa and South America, the cities selected from these two continents are Cape Town and Mexico City, which are far more developed comparing to other cities around. This might make this benchmark \"only for the developed ones\". And this also limits the generalizability of models trained on UrbanFeel, especially for culturally sensitive subjective perception tasks.\n2. While the benchmark covers urban changes over a long period, it lacks causal annotations that could provide deeper insights into the reasons behind urban transformations, limiting the ability to conduct more advanced spatiotemporal reasoning.\n3. The judgement VQAs from Fig 25-28 seems to be over subjective, and lack of proper context. Visual comparison would make more sense in these cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rjvUtM5FDl", "reviewer": "Reviewer_7xMU", "summary": "The paper introduces UrbanFeel, a benchmark for evaluating multimodal LLMs on urban visual tasks, focusing on three dimensions: (1) Static Scene Perception (e.g., co-location, dominant element extraction), (2) Temporal Change Understanding (e.g., pixel-level change, temporal sequence reasoning), and (3) Subjective Environmental Perception (beauty, safety, wealth, liveliness). The data come from ~4,000 multi-temporal street-view images from 11 cities (2007–2024), expanded into 14.3K QAs using a mixed pipeline (rule-based → model-assisted → manual). The authors evaluate 20 closed/open MLLMs (GPT-4o, Gemini-2.5-Pro, Qwen-VL, InternVL, LLaVA, etc.) and claim: (i) models are decent on static tasks, (ii) models drop sharply on temporal reasoning (especially TSR), (iii) some models reach or exceed human performance on pixel-level change with panoramic inputs, and (iv) MLLMs show early signs of “human-aligned” subjective perception.", "questions": "None", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Timely topic: urban, multi-temporal, human-centered evaluation is indeed underexplored.\n\n2. Breadth of models: evaluating 20 MLLMs gives the community a rough leaderboard.\n\n3. Task diversity: binary, MCQ, sorting, open-ended — this mirrors what MLLMs actually output.\n\n4. Clear structure of 3 dimensions: static → temporal → subjective is easy for readers to follow.", "weaknesses": "1. Lack of Novelty: The paper primarily aggregates existing datasets and tasks (e.g., Place Pulse, CityPulse, and ChangeScore) without presenting significant novel contributions. There is no new model architecture, learning objective, or groundbreaking evaluation framework introduced. The novelty is largely confined to combining pre-existing ideas into a single benchmark rather than offering a new approach or insight.\n\n2. Questionable Evaluation Methodology:\n\n- The paper uses GPT-4o as both a model being evaluated and the evaluator of other models, which introduces bias and conflicts of interest. The evaluation is thus not independent, and the scores could be inflated due to the evaluator’s own performance.\n\n- Inter-annotator agreement for the subjective tasks (beauty, safety, wealth, etc.) is not provided, which is essential for verifying the reliability of the subjective labels. Additionally, the paper does not address the cultural bias inherent in such subjective dimensions, as they are highly culture-dependent and may not generalize across regions.\n\n- There is no clear statistical analysis to validate the significance of the results. Performance differences across models (e.g., human-level performance claims) are not accompanied by statistical tests or confidence intervals, undermining the robustness of the claims.\n\n3. Dataset Issues:\n\n- The paper relies on Google Street View and Mapillary imagery, but there is no clear mention of how the dataset complies with legal and ethical guidelines for reuse. The potential issues with data licensing and privacy concerns (e.g., faces, license plates) are not adequately addressed.\n\n- The paper does not disclose whether the data used for subjective perception tasks might have been contaminated by biases from prior LLM training, as many subjective labels (e.g., beauty and safety) are culture-specific.\n\n4. The tasks in your benchmark involve single-shot answers (e.g., “Is this beautiful?”). In contrast, real-world urban analysis often requires ongoing, multi-step reasoning where feedback loops are important. How do you reconcile this simplification, and do you believe that such a limited approach can reflect the long-term reasoning and decision-making required in actual urban environments?\n\n5. The proposed benchmark includes multiple question types (binary, multiple-choice, sorting, open-ended). How do these abstract and highly controlled question formats relate to real-world urban decision-making? In real-world scenarios, urban planning, safety assessments, or aesthetic evaluations are not performed in this rigid, isolated format — how do you justify their relevance?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7Bdl905ZRS", "reviewer": "Reviewer_XNMk", "summary": "This paper constructs a comprehensive evaluation benchmark named UrbanFeel with a rich dataset containing 14.3k question-answer pairs to assess MLLM's understanding of temporal changes in urban environments and human subjective perceptions of urban settings. This benchmark comprises three major categories of tasks: Static Scene Perception, Temporal Change Perception, and Subjective Environmental Perception. This paper confirms through extensive experimentation the performance of current mainstream MLLMs on this benchmark and their gap relative to human experts.", "questions": "1. Can the benchmark or design approach proposed in this paper be applied to research beyond smart cities, such as topics related to sustainable development? \n2. Cities may undergo a process of decline alongside development. Can MLLMs effectively distinguish such scenarios? Or do MLLMs always assume that more prosperous neighborhoods are newer?\n3. What is your open-source plan?\n4. Would incorporating reasoning processes or using simple prompts to guide MLLMs in performing basic reasoning before answering questions enhance their performance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Organizing urban imagery on a temporal scale offers a novel perspective for observation. On one hand, it extends beyond conventional benchmark tasks to further test the comprehensive temporal and visual understanding and reasoning capabilities of MLLMs. On the other hand, it provides datasets and fresh insights for research on urban development.\n2. Based on temporal image data, this paper proposes targeted evaluation methods such as comparison and temporal sorting, effectively supporting the benchmark process.\n3. The text and figures are easy to follow.", "weaknesses": "I appreciate the research perspective of this work, but it still has some specific issues. If all the following issues are properly addressed, I will adjust the rating to \"Accept\".\n1. This paper does not sufficiently explore the impact and value of the proposed benchmark, which may limit its future popularity.\n2. Some figure descriptions are unclear and confusing. For example, the meanings of the numbers in Figure 2 (a) are very confusing. What is the relationship between 1100 and 2200 in the SCR subtask? And what are the meanings of yellow and grey colors? And what is the meaning of the abbr \"str\" in the \"View Type\" part? I hope the author will carefully examine every detail of these large and complex images to ensure they are self-descriptive.\n3. The details of the experimental evaluation require specific clarification, particularly regarding the accuracy metrics or calculation formulas used for each subtask, along with necessary parameters such as thresholds. Currently, I do not appear to have seen relevant descriptions in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)