Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents
Authors: Peilin Feng
Zhutao Lv
Junyan Ye
Xiaolei Wang
Xinjie Huo
Jinhua Yu
Wanghan Xu
Wenlong Zhang
LEI BAI
Conghui He
Weijia Li
Venue: ICLR 2026 Poster (2026)
Keywords: Earth observation
Earth-Agent
Earth-Bench
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Too Long; Didn't Read

Abstract: Earth observation (EO) is essential for understanding the evolving states of the Earth system.  Although recent MLLMs have advanced EO research, they still lack the capability to tackle complex tasks that require multi-step reasoning and the use of domain-specific tools. Agent-based methods offer a promising direction, but current attempts remain in their infancy, confined to RGB perception, shallow reasoning, and lacking systematic evaluation protocols.
To overcome these limitations, we introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem, enabling cross-modal, multi-step, and quantitative spatiotemporal reasoning beyond pretrained MLLMs. Earth-Agent supports complex scientific tasks such as geophysical parameter retrieval and quantitative spatiotemporal analysis by dynamically invoking expert tools and models across modalities. To support comprehensive evaluation, we further propose Earth-Bench, a benchmark of 248 expert-curated tasks with 13,729 images, spanning spectrum, products and RGB modalities, and equipped with a dual-level evaluation protocol that assesses both reasoning trajectories and final outcomes. We conduct comprehensive experiments varying different LLM backbones, comparisons with general agent frameworks, and comparisons with MLLMs on remote sensing benchmarks, demonstrating both the effectiveness and potential of Earth-Agent. Earth-Agent establishes a new paradigm for EO analysis, moving the field toward scientifically grounded, next-generation applications of LLMs in Earth observation. More information about Earth-Agent can be found at https://github.com/opendatalab/Earth-Agent

Published as a conference paper at ICLR 2026
EARTH-AGENT: UNLOCKING THE FULL LAND- SCAPE OF EARTH OBSERVATION WITH AGENTS
Peilin Fengl Zhutao Lv Junyan Ye Xiaolei Wang
Xinjie Huo²
Jinhua Yu² Wanghan Xu!
Wenlong Zhang' Lei Bai
Conghui He
Weijia L
Shanghai Artificial Intelligence Laboratory Sun Yat-sen University
Tsinghua Shenzhen International Graduate School, Tsinghua University
Github:
Dataset:
https///hnggnnngeecoo///tsttssss
Bench
Project Page:
sttpss//gperrdttalbbbgitttbbbbb io..loo//oooooo
MLLM- EO Research
Existing Agent-Basen EO Research
Earth-Agent (Ours)
(Classify image-lmageo
Sedan?
Calculate trend building volume
S
ChangeAgent
RSAgent
ThinkGeo
Toal 1
Pretrained-
MLLM
Model
Final Answer
T 2
Final Answer
Answer
RGB
RGB
Model
Tool
Drawbacks
rawbbcks
Advantage
1. Only suppoct processing RGB images
Still limited remote semsing datasets (RGl)
1. Able process Multi-Spetral and RGB mages
Able process numerous images
Only single-step
Tool remains shallow limited
Complex
interactive reasoning
rensoning
predefined ecosystem
Interact with external tools/ models, extensible
Can't Integrate extemnal wools or expert models
Evaluztion overlcoking reasoning trajectory
Systemic evaluation: trajectory result
Earth-Bench
Earth Earth-Agent
Evaluation
expert-curated 13729 Images
LLM
Toolkit
Arawer
Annotations
Questions
Quey
Step
Spee: trum
Respoine
Final
Tbea ay ande
Products
Folder
Steps
Referenee steps
Trais mact
GEE, EarthData
RGB
ReAct
Call
output porams
input porms
Rad Fnd
E Accaaay
Pricasay
Figure 1: Overview of our work:
The top panel contrasts prior paradigms: MLLM-based EO
research (left), Existing agent-based EO research (middle), and our Earth-Agent (right). The bottom panel illustrates contributions, including Berth-BBnchh construction, Earth-Agent ReAct with the predefined toolkit, and dual-level evaluation of both reasoning trajectories and final results.
ABSTRACT
Earth observation (EO) is essential for understanding the evolving states of the Earth system. Although recent MILMs have advanced EO research, they still lack the capability to tackle complex tasks that require multi-step reasoning and the use of domain-specifie tools. Agent-based methods offer a promising direction, but current attempts remain in their infancy, confined to RGB perception, shallow reasoning, and lacking systematic evaluation protocols. To overcome these lim- itations, we introduce Earth-Agent the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem, enabling cross-modal multi-step, and quantitative spatiotemporal reasoning beyond pretrained MLLMs. Earth-Agent supports complex scientific tasks such as geophysical parameter re- trieval and quantitative spatiotemporal analysis by dynamically invoking expert
"qqual contribution.
This work was done during their internship at OpenDatal ab of Shanghai Al Lab. "oorrssonniigg author. E-mail: wewi a@sz. nggua edu.cn

---

Published as a conference paper at ICLR 2026
tools and models across modalities. To support comprehensive evaluation, we fur- ther propose
a benchmark of 248 expert-curated tasks with 13,729 images, spanning spectrum, products and RGB modalities, and equipped with a dual-level evaluation protocol that assesses both reasoning trajectories and fi- nal outcomes. We conduct comprehensive experiments varying different LLM backbones, comparisons with general agent frramworks, and comparisons with MILMS on remote sensing benchmarks, demonstrating both the effectiveness and potential of Earth-Agent. Earth-Agent establishes a new paradigm for EO analy- sis, moving the field toward scientifically grounded, next next-generation applications of LLMs in Earth observation. More information about Earth- Sarth-Ageet can be found at htttps:://gtttii
opwndatt ab/ Saaarrhh-ggeee
INTRODUCTION
Earth observation (EO) Transon et al. 2018 Kokkoris et al., 2024 Li et al. 2023a) plays a critical role in understanding the evolving states of the Earth system in spatial and temporal dimensions (An- derson et al. 2017 Li et al. 2024a Brown et al. 2025), and has been successfully applied to urban planning (Shaker al. 2219). agriculture (Wojtowicz al. 2016), resources management (Li et al. 2020 Wang et al. (025a)). building extraction (Li et al. 2023b 2024c), disaster monitoring (Joyce al. 2009; Van Westen 2000), etc. Typically, EO data is categorized into two types (Samadzade- gaa et al. (022)): Perceptual data, such as RGB Imagery (RGB) aligned with human vision, and Raw Observational Data, including Raw Spectral Data (Spectrum) and Processed Earth Prod- uets (Products) stored in geodatabases such as Google Earth Engine (GEE and NASA Earth- d Both types of data are indispensable for EO research: perceptual data provides intuitive and human-interpretable insights, while raw observational data offers rich spectral and spatiotemporal information that enables quantitative analysis (Valipour et al. 2025 Xiong et al. 2022).
In recent years, multimodal large language models (MLLMs) have achieved excellent performance on classical remote sensing perceptual tasks such as VQA
Kuckrejja et al.) 2024 Muhtar et al.,
2024). scene classification (Kuckreja et al. 2024] Muhtar et al.
2024]]Liu et al. [024c]] Wang et al. (Zhang et al.) (024b), and semantic segmentation (Mall et al. 2023] Guo et al.] 2024a). However, despite their promising results, exist- ing MLLM-based EO research still faces several fundamental drawbacks: (1) they cannot process diverse EO modalities beyond RGB, such as thermal infrared (TIR), synthetic aperture radar (SAR), or hyperspectral imagery (Zhang et al., 2024b); (2) they typically operate on only one or few im- ages at a time (Li et al. 2024b). making it difficult to scale to large EO corpora; (3) they are limited to executing only single-step or shallow reasoning like VQA and classification, struggling with com- plex multi-hop analytical tasks; and (4) their reasoning is bounded by the static knowledge encoded in pretrained parameters, without the ability to integrate external scientific tools or expert models, making it difficult to extend beyond the generic capabilities of the foundation model; This naturally raises the question: how can we move beyond basic RGB perception and single-step reasoning to de- sign models that integrate diverse EO modalities and support complex multi-step scientific analysis?
2024e] Hu et al., 2025bb Zhan et al., 2025). D, object detection
Tool-augmented LLM agents represent a promising trajectory beyond MLLMs (Xi al. 2025 Sun
et al. 2025 Si et al. 2024 Tian et al. 2024 Tang et al. 2025). Unlike MLLMS that are restricted to RGB inputs, simple reasoning, and limited image contexts, agents are not inherently constrained by input modality or data volume (Xie et al. 2024 Gao et al. 2024). By leveraging the reasoning capabilities of LLMs and dynamically interacting with external tools [u al. (025). they can flex- ibly process diverse EO modalities, perform multi-step analytical reasoning, and integrate domain- specific tools and expert models that go beyond the scope of the pretrained MI.LM model (Ding et al. 22255] Wang et al.) 2024c). This mechanism directly tackles the core weaknesses of MLLMs, extending beyond RGB to diverse modalities, scaling from single-image inputs to tasks involving hundreds of images, advancing from shallow perception to multi-step reasoning, and bridging LLMs with external scientific tools for domain domain-specific analysis.
However, existing agent-based research in Earth science is still at an early stage (Pantiukhin et al. 2025), with existing attempts largely confined to perceptual tasks such as change detection (Liu
https://search.eat nawa. gev
2

---

Published as a conference paper at ICLR 2026
et al. 2024b; 2025) and classification (Xu et al. 2024a Hu et al. 2025a), often emphasizing caption ability rather than scientific analysis. Efforts on Raw Observational Data are even more limited. Uni- vEarth (Kao et al. 2025) considers EO data from GEE but operates essentially
code generation agent, without implem implementing genuine tool calling, making it difficult to handle complex and realistic geoscientific analysis tasks that require professional tool use. These efforts reveal several key limita- tions: (1) current EO agents support only limited data modalities, with most efforts still centered on conventional remote sensing datasets dominated by RGB imagery (Xu et al., 2 (2) their tool usage remains shallow, limited to few expert models and reasoning steps, even some agents lack a predefined tool ecosystem. making them insufficient for complex analytical workflows (Shabbir et al. 2025); and (3) their evaluation remains unsystematic, with emphasis only final answers while overlooking reasoning trajectory. This raises another question: how can we design an EO agent with a structured tool ecosystem and systemalic evaluation, capable of bridging perceptual and spectral data like Earth scientists?
Spectrum
Products
RGB
Besed temperature and vegetotion indiostors (LST and NDVI) River Basin region from J June to September 2023, calculate the number of spikes in the drought indek, which may correspond severe drought events.
Analyze the righttime light intersity trend in Leon (2013-2024) using linear regression.
To International maritime pkmiingg calculate difference
Yellow
between the harbor areas of the two images (GSD: 0.50m/ppx 8 O. 0.30m/px).
A. Slope: 0.10: Intercept: 1.85
B. Slope: 0.12: Intercept: 1.96
Slope: 0.14; Intercapt: 2.05
A. 1000000
15000 2.200 30000
b. Slope: 0.16; Intercept: 2.12
get_filelist
=
SM3Det
get filellst todi tvdi
get filelistt c>
cale_ cale_batch batch_..
mmge__ mean
count__spikes from values
cale_ batch image_meon
calculate_areaa +
SAM2
Final Answer:
compute__. linear frend
Final Answer:
difference
Final Answer
Earth-Agent solving tasks across Spectrum, Products, and RGB data through multi-step reasoning with expert tool calls.
Figure 2:
To address these questions and unlock the full landscape of EO, we propose Earth-Agent, an agen- tic framework that unifies perceptual and spectral EO data within single architecture in section By coupling LLM reasoning with a structured toolkit in 3.2 Earth-Agent supports diverse modali- ties and complex multi-step analysis, enabling agents to tackle real-world geoscientific tasks beyond the limits of existing MLLMs and EO agents. Concretely, Earth-Agent integrates 104 specialized tools, built upon the Model Context Protocol (MCP) (Hou et al. 2025 Rayl 2025) for interop- erability, and grouped into five domain-specifee tool kits: Index, Inversion, Perception, Analysis, and Statistics. This structured design not only enables the agent go beyond classical percep- tual tasks toward quantitative analysis and spatiotemporal reasoning, but also makes the framework easily extensible with additional comainn-speciin tools. To systematically evaluate its effectiveness, we further introduce Earth-Agent Benchmark (Earth-Bench) in section 4 which reflects real- istic EO workflows and supports both Auto-Planning and mnstruttionFFolowinn query regimes, together with dual-level evaluation protocol that measures reasoning trajectories well final outcomes. We comprehensively evaluate Earth-Agent by varying LLM backbones on Earth- Berch, comparing with general agents, and benchmarking against MLLMs remote sensing datasets.
To sum up, our main contributions are summarized as follows:
We propose Earth-Agent the first agentic framework for EO, built upon the MCP and a ReAct (Yao et al., 2023) reasoning, integrating 104 specialized tools and expert mod- els within predefined tool ecosystem, while remaining easily extensible with additional domain-specific tools and models.
We construct Earth-Bench, a benchmark of 248 expert-curated questions with 13,729 im- ages, spanning perceptual and spectral modalities beyond RGB. Each question requires multi-step reasoning with explicit tool use, and the benchmark supports dual-level evalu- ation protocol that assesses both reasoming trajectories and final answers.
Through comprehensive evaluation, we show that Earth-Agent substantially outperforms general agents such as Operator (OpenAl 2025b) and Manus (Shen et al.) 2025) on Earth- specific tasks in Earth-Bende and also surpasses remote sensing MLLMS on remote sens- ing benchmarks, demonstrating both its effectiveness and potential for advancing EO re- search.
3

---

Published as a conference paper at ICLR 2026
2 RELETED WORK
MLLM-based Earth Observation Research The rise of multimodal large language models
(MLLMs) has stimulated growing interest in their use for Earth observation (EO) (Aleissace et al.
2023 Lu et al. 2025 Li et al. 2024b). Early studies mainly explored captioning (Hu et al. (025b)
and question answering Kuuckreja et al. 2024) for single remote sensing images (Shi & Zou 2017
Wang et al., 2020), aiming to align visual features with natural language. With the avai availability biyy of
larger datasets (Xiong et al., 2022 Zhou et al., 2025) and stronger backbones (Team] 2024
al. 2024d), subsequent works extended this paradigm to broader perception tasks: for instance, GeoChat (Kuckreja et al. 2024) enabled interactive scene understanding, while RS-GPT (Hu et al.,
2025b) combined captioning with visual question answering, More recently, simple temporal rea- soning has been introduced, with ChangeCLIP (Dong et al. 2024) addressing bi-tem bi-temporal change captioning and SkyEye-GPT (Zhan et al., 2025) extending to rideo-based analysis. However, the scope of MLLM-bar -ased EO research remains narrow: existing approaches are still centered on RGB imagery and struggle with complex multi-step reasoning without domain-specific tool integration.
Agent-based Earth Observation Research. Tool-augmented agents have gained traction gen- eral AI, achieving remarkable progress in domains such as code generation (Qian & Cong 2023
/Zhang et al.] 2024a), web search (Xu et al.) (024b), and video understanding (Ren et al. 2025
Wang et al. 2024d), but their application to Earth observation (EO) is still at early stage (ao
ct al. 2025). Early systems such as Change-Agent (Liu et al. 2024b) focus on bi-temporal change detection, while RS- RS-ChatGPT (Guo et al., (0224) and RS-Agent (Xu et al.) 2024a) combine LLMs with pretrained detectors or tool suites for scene classification, detection, and segmentation. More recently, ThinkGeo (Shabbir et al.) 2025) introduces agentic workflows for simple geospatial calcu- lations on perceptual data, and Unrth (Kao et al., 2025) requires LLMs to generate GEE code for spectral analysis, with high execution failure rates. Despite these advances, existing EO agents remain constrained: they operate mainly on RGB perception tasks, rely on remote sensing models for simple reasoning that does not extend to multi-step analysis, and lack a predefined tool ecosys- tem, making them insufficient for complex real-world geoscientific workflows. Moreover, current benchmarks cover limited task types and annotations, lacking systematic evaluation protocols that assess both the correctness of outcomes and the quality of reasoning trajectories. As result, cur- rent frameworks remain limited in modality coverage, constrained to shallow reasoning with remote sensing models, and hindered by the absence of predefined tool ecosystem, highlighting the neces- sity for EO agents and benchmarks that support diverse data, multi-step analytical workflows, and systematic evaluation.
EARTH-AGENT FRAMEWORK
In this section, we detail the operation mechanisms of Earth-Agent. We first formulate its opera- tion pipeline as ReAct-style (Yao et al. 2023) Partially Observable Markov Decision Processes (POMDP) formulation (Huang et al. 2024; Chala et al. (025) in section 3.1 including the ob- servation process, policy reasoning and memory update, as shown in Figure Then we introduce the functionality of the specialized tool kits that enable EO analysis across perceptual and spectral data in section 3.2 Finally, we define the dual-level evaluation protocol, which assesses EO agents in both end-to-end and step-by-step modes to evaluate not only final accuracy but also reasoning trajectories in section 3.3
3.1 OPERATION MECHANISMS
Earth-Agent receives a task goal g. interprets user queries and intermediate tool outputs, and se- lects actions from modular toolkit to progressively solve the task. This process formulated as a POMDP, defined by the tuple (g. S, O. T), where is the task goal, S is the state space (un- observable environment status such as geospatial data files or raster values), AA is the action space (tool calls in the kit), O is the observation space (outputs returned by tools, including text, numerical values, and images), and T : S x A + S is the state transition function.
At each time step t. given a policy x, the agent selects an action conditioned on the goal g and its interaction history, which records past observations and actions:
m
(oo, Go 01, (1,.-..;0)) Ot);
4

---

Published as a conference paper at ICLR 2026
K
Things happen before step t+1
Query
Folder
Things happen after step t+1
Think
Dual-level Evaluation
Memory
Og
01
O2
O
Tool rajectory
Go
Updato
Updata
④ Action
"hame"(t)
"input"(t)
"output"(t)
Annotations
a2
0. 2
R efference steps name
Step-by- Step Expected Tools: (A,B,C)
e ools order I..D.C.BVV
A.D,
H Nat include tools
Toot
Tool Calling
DT Tools in, order
(
(AD. B.C
Ot
Calling
oc+1
0t+2
output params
input params
OT Tools exact, m
AAD,B,
Not call order
A/inpul autcut =ST
nuut.uupuu GT
End-to-End
Annotations
Efficiancy
mum (Tool Trajectory)
(Refeeenoo Stops)
Index 12
Inversion Perception 15
Analysis
Statistics
Accuracy
Jsnn (i(y
18
10
49
(qucecione)
The left part illustrates the ReAct-style workflow, where Earth-Agent iteratively performs tool calling, memory update, thinking, and action using domain- specific toolkits. The right panel presents the dual-level evaluation protocol, assessing both step-by- step reasoning trajectories and end-to-end outcomes.
Figure 3: Earth-Agent Framework:
The action distribution modeled as:
~ π Ta; g.).
The full agent trajectory r [sp, Oo, Co, $1, 01, a1....... ...,. or is determined jointly by the policy and the environment dynamics:
Tool calling
T-1
(T( = p(su) (ou so)
II E=
Taa g, m²) Z(o (441) T(s+11 st, at)
Initial
Think
where denotes the observation distribution induced by tool outputs.
In this formulation, the LLM controller functions as the policy reasoning over the history and task goal g to decide the next tool calling. while the Toolkit provides executable atomic actions categorized into Index, Inversion, Perception, Analysis, and Statistics. Concretely, as illustrated in Figure 3 each loop proceeds as follows: @ Tool calling At step 1, the agent invokes the most suitable tool conditioned the current memory my and task goal g. which yields the tool response for observation 0,+1- @ Memory update At step f, the agent appends the pair (o,, a,) together with the resulting observation 0t+1 into the memory stack, ensuring that the complete interaction history is preserved for subsequent reasoning. @ Think At step (+1, the LLM controller reasons over the updated memory m, together with the task goal g to plan the next action, determining which tool to invoke and how to configure its inputs. + Action Selecting and executing the subsequent tool call a++1 that produces 0 2- The ReAct loop continues until the stopping condition is satisfied, yielding both the final answer and reproducible sequence of tool calling trajectory.
3.2 TOOL KIT
To enable comprehensive EO analysis, Earth-Agem integrates 104 specialized tools organized into five functional kits. The Index kit provides implementations of widely used EO indices (e.g.. NDVI, NDWI, NBR) (Montero et al.) (202)) for rapid environmental characterization. The Inversion kit fo- cuses geophysical parameter retrieval, including land surface temperature (L.ST) (Li ei al. 2013)
5

---

Published as a conference paper at ICLR 2026
precipitable water vapor (PWV) (He & Liu| 2020), vegetation water content (Ceccato et al. 2001 sea ice concentration DiGirolamo et al. 2022). and others. The Perception kit supports vision- oriented tasks such scene classification (Ma et al. 2025), object detection (Li et al. 2024e). and segmentation (Ravi et al. 2024). The Analysis kü targets spatiotemporal reasoning. offering trend detection, seasonality decomposition, change point analysis, and spatial autocorrelation. Finally, the Statisties kif provides large-scale data preprocessing and statistical computation (c.g.. variance, skewness, batch operations, cloud masking). Together, these tool kits cover the diverse types of EO tasks from perceptual to spectral, and from descriptive to quantitative analysis. The detailed list and description of tools can be found in Appendix
3.3 EVALUATION PROTOCOL
Prior benchmarks have primarily emphasized final accuracy, overlooking the reasoning trajectory that leads to the final output (Mialon et al. 2023 Jimenez et al,, 2024; Chen et al. 2025). To address this, we adopt dual-level evaluation protocol: agents assessed in step-by-step mode to capture the quality of their reasoning trajectories, and in an end-to-end mode to measure final task performance. This dual perspective enables fine-grained diagnostics of both reasoning and outcomes. The detailed calculation formulas can be found in Appendix B.2
End-to-End evaluation measures task-level performance, including Accuracy of the final answer and Efficiency of the trajectory relative to expert solutions.
Step-by-Step evaluation assesses the quality of intermediate reasoning. We consider four com- plementary aspects: Tool-Any-Order, which checks whether all necessary tools are used in LLM planning: Tool-In-Order, which evaluates whether tools are invoked in the correct sequence; Tool- Exact-Match, which evaluates the exact prefix-level accuracy between the predicted and expert tra- jectories; and Parameter Accuracy, which verifies whether both tool identifiers and their arguments are correctly matched.
EARTH-AGENT BENCHMARK
4.1 OVERVIEW OF EARTH-AGERT AGENT BENCHMARK
We introduce Earth-Agent Benchmark (Farth-Bench), a dataset designed to evaluate tool- augmented EO agents in realistic Earth science analysis scenarios. The benchmark integrates three major types of Earth observation data: RGB Imagery (RGB), Raw Spectral Data (Spectrum). and Processed Earth Products (Products). It supports 14 representative tasks, including classification, detection, temperature monitoring, weather forecasting, etc., with particular emphasis scientific analysis that requires quantitative reasoning rather than qualitative description.
Datsset
Cross Modality
Image Sample
Regaining Tool
Ressoning Stops
Quantitative Analgai
Teajectory Eval.
Statement
RSVGA HR
X
2.7M
X
X
X
Questiers
240
13729
FartiVIQA
X
AK
X
X
X
Spectrum
Avengee per Quection
VRSSench
X
208K
X
X
X
Masrmur
670
Products
X
X
x
Average per Soas
Madisue
PEACE
X
0.22K
X
X
X
Apstcreppen
Thinageo
X
34
X
X
Specturn
Products
Unimeorth
X
X
(Ours)
6F1.6
Figure 4: Dataset Comparison and Overview: The left panel compares Earth-Bench with prior MLLM and agentbased EO benchmarks. The right panel presents the statistics of Barth-Bence and its evaluation with SOTA LLMs using Earth-Agent, highlighting the difficulty of Earth-Bench.
As shown in Figure 4 MLLM-based benchmarks including RSVQA-HR (Lobry al. 2020 VQA /Wang et al. 2024b), VRSBench (Li et al.) 2024d) and Geo-Bench (Lacoste et al. 2023
mainly limited to single-step perceptual for RGB data using pretrained ML.LMs (Liu

---

Published as a conference paper at ICLR 2026
et al. 2024d Team 2024| OpenAl 2024), without requiring external tool use for scientific quantita- tive analysis (e.g., spatiotemporal trend estimation), not to mention reasoning trajectory evaluation. On the other hand, Earth-Bench advances beyond prior Agent-based EO benchmarks, such as PEACE (Huang et al. 2025), Thinkgeo (Shabbir et al. 2025) and UnivEarth (Kao et al. (025). by incorporating 13K+ images across spectrum, product and RGB modalities, while requiring inter- action with 104 domain tools. It consists of 248 expert-curated questions, requiring an average of 5.4 reasoning steps of quantitative analysis. Even with the state-of-the-ar (SOTA) LLM backbones, performance remains limited, which underscores not only the benchmark 's difficulty and diagnostic value but also the necessity of reasoning trajectory evaluation. Therefore, we need to annotate on both trajectories and final answers in section 4.2
4.2 DATA ANNOTATION PIPELINE
To construct Earth-Bench, we collected raw data from platforms such as Google Earth Engine, NASA EarthData, and public remote sensing datasets (Xia al. 2017 Zhan et al. 2023 Xia al. 2018 Su et al. 2019). From these data sources, a team of domain experts curated 248 problems that require multi-step quantitative reasoning. The annotation team was composed of computer science experts, 7 remote sensing specialists, and 3 Earth science specialists. Each problem is accompanied by a step-by-step Python solution and is designed to reflect the complexity of real-world Earth science workflows, which demand the coordinated use of multiple tool kits.
In prior benchmarks, queries have been designed either as step-implicit, where no intermediate step guidance is provided (Mialon et al., 2023 Wang et al.] (2024a). or step-explicit, where the query itself contains step guidance (Guo et al. 2024b Ma et al., 2024). Motivated by the com- plexity of EO workflows, which often require multi-step processing, Earth-Bench incorporates both regimes: Auto-Planning corresponds to the ate-iimpicit setting and evaluates the agent's 's ability to autonomously plan its solution trajectory, while Instruction-Follvvii explicit setting and evaluates the agent's 's ability to follow and translate human instructions into ex- ecutable actions. Both regimes contain 248 complete questions for evaluation. Together, these two regimes provide a comprehensive assessment of both autonomous reasoning and guided execution in EO contexts. Details can refer Appendix A.4
corresponds to the step-
Question Construction
Data Annotation
Google Earth Engine
Generation NASA Earth
Python Solution
Python Result
Armatation
Data
Punc
Pencee
<teol
Reriste Serding Image
Data
Punec
Input
def:
Cutpar
JSDI
DIOR RSVELS
DOTA
NMPU- VHRRO
Func 3
Bunc
realt
Armwer
Exomple Query
Trpart
Outpon/ieet
aattiiae rnd anttitiao nottt
Nut
'out tit Tee 3.p
Viing Landon 6 Bard 10 MD. evey dis meen 25%.4
Step
Step
Step
netropolttnn oraa scceec
daya
Chicago Lendset
Calculate
Answen: 12 days
1h Days
Daye
Figure
Choice
Figure 5: Construction and Annotation of Earth-Bench. The left shows question generation from EO data, the right illustrates the data annotation pipeline that simulates ReAct-style trajectories, and the bottom provides an example explaining the multi-step annotation process.
To enable dual-level evaluation, we explicitly annotated both the final answers and the full reason- ing trajectories. As illustrated in Fig. 5 the annotation process was designed to mimic the ReAct loop of agents: Python Solution. Annotators first identify the domain tools such as compute _nww and spldt_window) required to solve problem and then assemble them into step-by-step main() program. Each tool is represented as a Python function, and the functions are planned in the correct order form executable workflow that mirors the agent's reasoning trajectory. Python Result.

---

Published as a conference paper at ICLR 2026
When executed, the program produces explicit input and output arguments for each function call, as well as the final output of the main() function. JSON Annotation. Each function call is then trans- lated into structured JSON record to align with the ReAct-style trajectory annotation. The function name corresponds to the action tool name, the function input arguments corresponds to the action passed by the agent, and the function output arguments corresponds to the tool responses returned to the agent. The final output of the main() function is recorded as the ground-truth answer for the problem. This provides a complete record of both the reasoning trajectory and the final answer.
5 EXPERIMENTS
5.1 EXPERIMENTAL. SETUP
Evaluated Models. We evaluate closed-source and 10 leading open-source L.Mss. For closed- source models, we consider GPT-5 (OpenAl 2025a). GPT-4o (OpenAl 2024), and Gemini- 2.5
(Comanici al.] 22025). For open-source models, including Deepseck-V3. V3.1 (Liu et al., 2024a), Kimik2 (Team et al. 2025). (reen3-max-Prevvee
Swen3-32B (Yang et al., (025), and
InternVL33. (Wang et al. (025b), which represent the smartest open LLMs available to date.
5.2 EARTH-AGENT WITH DIFFERENT LLM BACKBONES
As shown in Table [ we evaluate the impact of different LLM backbones on Barth-Bench. Results are reported under both step-by-step and end-to-end evaluation protocols, allowing us to jointly assess the quality reasoning trajectories and final outcomes. The following observations can be made:
Table 1: Performance of different LLM backbones Earth-Bench under both Auto-Planning (AP) and Instruction- Following (IF) regimes. Results are reported with dual-level evaluation, covering Accuracy, Efficiency for final outcomes and Tool-Any-Order, Tool-In-Ordes Tool-Exact-Mand Pa-
rameters for trajectory quality. We bold the best results and underline the runner-ups.
Accuracy
Efficiency
Toul-Any-Ordee
Tool-In-Order Tool-Exact-Matee
Parameters
Model
AP
IF
AP
IF
AP
IF
AP
IF
AP
IF
AP
IF
GPT-5
65.99
62.35
2.3560
2.9093
68.74
71.41;
57.71
61.06.
44.97
46.01;
26.11
25.91
Gemini-2.5
52.23
53.04
2,9958
2.4595
58.04
61.63;
45.31
50.72:
31.32
41.04;
17.26
23.43
GPT-4o
43.72
44.94
2.1211
2.6007
65.65
67.02;
50.76
53.04
46,26
47.41
26.55
27.92;
Kimik2
50.61
56.68
1.8542
2.1793
71.03
78.86
57.57
68.83
42.11
51.15,
25.90
30.45,
DeepSeek-VS
V3.1
51.42
52.23
2.6116
2.6303
78.31
78. 78.66
62.73
64,50
48.54
58
34.81
31.36
Qwen3-Max
50,20
47.37
1.8110
1.9511
69.56
70.14
53.28
56.02
37.02
42.74
21.83
26.27
Seed-1.6
52.48
59.51
1,3110
1.3408
55.43
59. 44,
40.67
46.79.
28.39
35.47
18.32
23.13
LLaMA-4
44.94
38.46
0,2886
0,3211
16.51
22.41;
2.45
12.05
1.70
9.05
1,30
6.46.
Qwen-Plus Plen-Plus
42.51
38.46
1.5119
1.5854
52.04
55.96:
30.75
39.77
11.69
25.51;
9.12
16.95;
GLM-4.5v
32,86
35.25
1.7123
2.0129
42.48
46.69;
28.57
35.24
14.12
19.95;
11.02
15.37,
Mistral
29.96
22.67
0.8802;
27.73
29.64;
11.78
20.90
9.13
18.13;
7.24
11.66
Qwen3-32B
20.65
24.80-
2.7274
1.9010;
39.76
42.39
21.56
33.79.
9.51
26.10;
8.17
17.73
IntemV1..3.5
26.72
26.72
0.1206
0.1750
8.83
16.62;
3.87
10.59.
2.02
9.32
1.46
5.32,
Obs.1 LLM models pretrained with tool calling, such as GPT-5, Gemini-2.5, DeepSeek- 13.11, Kimik2, and Qwen3, demonstrate strong performance across both step-by---s and end-to-end eval- uations. Further, closed-source models like GPT-5 typically achieve higher final accuracy, while open-source models, particularly DeepSeek-Val V3.1 and Kimik2, outperform GPT-5 in tool-use ac- curacy, demonstrating superior performance in reasoning trajectory alignment. We have provided a detailed analysis of the I.MMs' performance across the Spectrum, Products, and RGB modalities, which can be found in Appendix C
regimes enhance tool calling by providing explicit step guidance in the query, leading to improved tool calling accuracy across nearly all models. Interestingly, despite the improved reasoning trajectories, this does not necessarily lead to higher final accuracy. In fact, for some advanced models, this added complexity may even result in a decrease in final accuracy. We have conducted detailed error analysis of Earth-Agent' performance in the Earth-Bench bench- mark, focusing on representative models such as GPT-5, DeepSeek V3.1, Kimik2, and Qwen3-man max. This can be found in Appendix D
Obs.2. Instruction-followee
8

---

Published as a conference paper at ICLR 2026
Obs.3. Across nearly all models, the ability to identify the correct set of tools, reflected in Tool-Any-Order and Tool-In-Order metrics, remains consistently strong. However, models often introduce irrelevant steps during reasoning. which undermines their accuracy Tool-Ehact-Match and parameter execution. Crucially, these two fine-grained capabilities are indispensable real EO analysis workflows. For example, if additional transformations data process, it becomes extremely difficult to obtain correct results in the subsequent steps, Their weakness therefore exposes a key bottleneck that prevents EO Agents from achieving higher final accuracy in EO data processing.
mistakenly applied to the EO
5.3
COMPARISON WITH GENERAL AGENTS
Since many Earth-Bench tasks involve processing hundreds images, existing open-source agent frameworks cannot handle these questions due to input size constraints. To enable fair comparison, we construct Barth-Bench-LAnn
reduced yet representative subset that preserves modality diver- sity while remaining within the capacity of general-purposee agents. It consists of 60 questions evenly distributed across the three EO modalities: Spectrum, Products, and RGB. As shown Table 2 we report results across three modalities: Spectrum, Products, and RGB.
Table 2: Comparison with general agents on Earth-Bench- Bench-Lite. Results are reported across Spec- trum, Products, RGB modalities. We bold the best results and underline the runner-ups.
Method
Spectrum
Products
RGB
Avg.
Latency
GPT-Agent
45.00
31.60
45.26
40.12
300 min
MGX
40.00
15.80
0.00
18.60
2 min
Manus
15.00
15.80
47.62
26.14
150 min
Coze
35.00
10.50
0.00
15.30
2 120 min
Farth-AgentGPPTS)
65.00
36.84
65.71
55.83
158 min
Earth-Agent Deepseek V3.11)
50.00
42.11
51.43
47.84
79 min
Earth-Agent( Selll-Agget((Kiiiikk Kimik2)
36.84
50.00
50.00
45.95
131
By comparison, general agents show limited modality coverage. They can handle relatively simple Spectrum tasks by writing ad-hoc code, but perform poorly on Products tasks due to the lack of domain-specific spatiotemporal analysis tools. For the RGB modality, MGX and Coze even fail to complete any tasks. In contrast, by interacting with 104 predefined geoscience tools, Earth-Agemt consistently achieves superior performance across all three modalities, whether driven by the closed- source GPT-5 or the open-source DeepSeck-V33 V3.1. We also compared the latency our Earth-Agent framework with that of the baseline agents. The latency remains within reasonable range when compared to existing general agents. The substantial performance improvements in task completion more than justify the additional computational cost. A detailed discussion can be found in the Appendix
5.4 COMPARISON WITH ALLM-BASEE EO МETHODS
We further compare Earth-Agent with remote sensing large models on classification, detection, and segmentation tasks. The results are summarized in Table
Table 3: Comparison with MLLMs Remote sense benchmarks. Results are reported on classifi- cation, detection, and segmentation tasks. We bold the best results and underline the runner-ups.
Model
Classification
Detection
Grounding DIOR-RSVG
AID
WHU-RS19
DOTA
HRSC2016
MiniGPT-v2 Chen
2023)
32.96
64.80
14.8
24.8
29.892
LLaVA-1.5 Liu et al. 2024d)
51.00
74.52
17.5
22.1
12.085
Sphinx Lin et
2023
58.20
15.1
25.7
0.939
Geochat Kuckreja et al. 2024
72.03
86.47
16.5
24.0
10.024
VHM (Pang et al. 2025)
91.70
95.80
LHRS- Bot (Muhtar et al.
2024]
91.26
93.17
17.1
24.4
11.826
Earth-Agent (ours)
93.42
96.12
60.88
65.60
60.46
Earth Agent demonstrates clear superiority over existing MLLMs across classification, detection, and segmentation benchmarks (Table Prior ML..M-based approaches often lack generalization

---

Published as a conference paper at ICLR 2026
across diverse EO tasks: for example, LHRS-Bot achieves strong results in classification but strug- gles on detection and grounding, while VHM attains high classification accuracy but cannot even handle detection segmentation tasks. In contrast, Earth-Agent interacts with a predefined toolkit of 104 geoscience functions and expert models, allowing it to adaptively invoke specialized tools or models for each task type. This modular design enables Earth-Agent maintain robust performance across modalities, overcoming the limited extensibility of previous MLLM-based EO systems.
6 CONCLUSION
Earth-Agent marks significant advancement in applying (M)LLMs to BO analysis, extending RGB perception to Spectrum, Products and RGB modalities. By shifting from single-step inference with pretrained MLLMs to multi-step reasoning through external tool/model integration, it overcomes key limitations of prior MLLM-based approaches, such as handling numerous image inputs and quantitative spatiotemporal analysis. To support rigorous evaluation, introduced Earth-Bench which requires multi-step quantitative reasoning and dual-level evaluation, which evaluate both rea- soning trajectories and final outcomes. Our experiments further reveal the current bottlenecks of Earth-Agent in EO applications and provide detailed diagnostics. Finally, by comparing with both general agents and domain MLLMs, we highlight the transformative potential of Earth-Agent foundation for the revolutio of LLM applications in Earth observation.
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. this study, no human subjects or animal ex- perimentation was involved. All datasets used, including GEE, NASA EarthData, AID, DOTA, HRSC2016, DIOR-RSVG, were sourced in compliance with relevant usage guidelines, ensuring no violation of privacy. We have taken care avoid any biases discriminatory outcomes in our research process. No personally identifiable information was used, and no experiments were con- ducted that could raise privacy or security concerns. We are committed to maintaining transparency and integrity throughout the research process.
REPRODICIBILITY STATEMENT
We are committed to ensuring the full reproducibility our work. The proposed Agent framework described in Section 3 while the evaluated models and metrics detailed in Appendix B To further support transparency, we will release our codebase, which includes the Agent framework, evaluation scripts, and other relevant components, and will also provide the evaluation logs from the agent experiments, enabling the community to replicate our results and build upon them.
ACKNOWLEDGEMENTS
This project funded by National Natural Science Foundation of China (Grant No. 62571560) and Shanghai Al Laboratory.
REFERENCES
Abdulaziz Amer Aleissaee, Amandeep Kumar, Rao Muhammad Anwer, Salman Khan, Hisham Cholakkal, Gui-Song Xia, and Fahad Shahbaz Khan. Transformers in remote sensing: A survey. Remote Sensing, 18(7):1860, 2023.
Katherine Anderson, Barbara Ryan, William Sonntag, Argyro Kavvada, and Lawrence Friedl. Earth observation in service of the 2030 agenda for sustainable development. Geo-spatial Information Science, 20(2):77-95 2017.
Christopher Brown, Michal Kazmierski, Valerie J Pasquarella, William Rucklidge, Masha Samsikova, Chenhui Zhang, Evan Shelhamer, Estefania Lahera, Olivia Wiles, Simon Ilyuhchenko,, et al. Alphacarth foundations: An embedding field model for accurate and effi- cient global mapping from sparse label data, arXiv preprint arXiv:2507. 22291, 2025.
10

---

Published as a conference paper at ICLR 2026
Pietro Ceccato, Stéphane Flasse, Stefano Tarantola, Stéphane Jacquemoud, and Jean-Marie Grégoire. Detecting vegetation leaf water content using reflectance the optical domain. Re- mote Sensing of Environment, 77(1):22-33 2001. ISSN 0034-4257. doi: https://oi.org// com/sct scisccess ence/
001618902244433570010001111 URL https: //wp:///www.. sci enceedecddiie
att.ccle//11/500344551100.0
Olena Chala, Vladyslav Yevsieiev, Svitlana Maksymova, and Amer Abu-Jassar. Mathematical model based on multi-agent reinforcement learning (marl) and partially observable markov deci- sion process (pomdp) for modeling cargo movement for a mobile robots group. Multidisciplinary Journal of Science and Technology, 5(4)) 489, 2025.
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Yuefeng Huang, et al. Acebench: Who wins the match point tool usage?
arXiv preprint 2rXiv:2501.12851 2025.
Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as unified interface for vision-language multi-task learning. arXiv preprint 10.04:2310.09478 2023.
Gheorghe Comanici, Eric Bieber, Mike Schackermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Oni Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06266 2025.
N. DiGirolamo, C. L. Parkinson, J. J. Cavalieri, P. Gloersen, and H. J. Zwally. Sea ice concentrations from n smmr and dmsp ssm/i-ssmis passive microwave data, 2022. URL https:// doi. org/ 10. 5067 /MP YG1 5WAA4WX NSIDC-0051. [Data Set]. Accessed: 2025-09-23.
Keyan Ding, Jing Yu, Junjie Huang, Yuchen Yang, Qiang Zhang, and Huajun Chen. Scitoolagent: knowledge-graphh graph-driven scientific agent for multitool integration. Nature Computational Science, pp. 1-11, 2025.
Sijun Dong, Libo Wang. Bo Du, and Xiaoliang Meng. Changeclip: Remote sensing change detection with multimodal vision-language representation learning. ISPRS Journal of Photogrummetry and Remote Sensing, 208:53-69, 2024.
Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1)11-24, 2024.
Haonan Guo, Xin Su, Chen Wu, Bo Du, Liangpei Zhang, and Deren Li. Remote sensing chat- gpt: Solving remote sensing tasks with chatgpt and visual models. In IGARSS 2024-2024 IEEE International Geoscience and Remote Sensing Symposium, pp. 1147-111478 IEEE, 2024a.
Zhicheng Guo, Sijie Cheng, Hao Wang, Shihao Liang, Yujia Qin, Peng Li, Zhiyuan Liu, Maosong Sun, and Yang Liu. Stabletoolbench: Towards stable large-scall scale benchmarking on tool learning of large language models. arXiv preprint arXiv:2403.07711 2024b.
Jia He and Zhizhao Liu. Water vapor retrieval from modis nir channels using ground-based gps data. IEEE Transactions on Geoscience and Remote Sensing, 58(5):3726-3777, 2020. doi: 10./
TGRS.20019299 29.22077777
Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. Model context protocol (mep): Land- scape, security threats, and future research directions. arXiv preprint arXiv:2503.23278 2025.
Huiyang Hu, Peijin Wang, Yingchao Feng, Kaiwen Wei, Wenxin Yin, Wenhui Diao, Mengyu Wang, Hanbo Bi, Kaiyue Kang, Tong Ling, al. Ringmo-agent: A unified remote sensing foundation model for multi-platform and multi-modal reasoning. arXiv preprint arXiv:2507.2077 2025a.
Yuan Hu, Jianlong Yuan, Congcong Wen, Xiaonan Lu, Yu Liu, and Xiang Li. Rsgpt: A remote sensing vision language model and benchmark. ISPRS Journal of Photogrammetry and Remote Sensing, 224:272-286, 2025b.
11

---

Published as a conference paper at ICLR 2026
Yangyu Huang, Tianyi Gao, Haoran Xu, Qihao Zhao, Yang Song, Zhipeng Gui, Tengchao Lv, Hao Chen, Lei Cui, Scarlett Li, et al. Peace: Empowering geologie map holistic understanding with mllms. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3899- 3908, 2025.
Zhiyu Huang, Chen Tang, Chen Lv, Masayoshi Tomizuka, and Wei Zhan. Learning online belief prediction for efficient pomdp planning in autonomous driving. arXiv preprint arXiv:2401.15115, 2024.
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL
net/ htte:/oorummiddd/i 6yNOM66
Karen E Joyce, Kim C Wright, Sergey V Samsonov, and Vincent G Ambrosia. Remote sensing and the disaster management eycle. Advances in geoscience and remote sensing, 48(7):317-346, 2009.
Chia Hsiang Kao, Wenting Zhao, Shreelekha Revankar, Samuel Speas, Snehal Bhagat, Rajeev Datta, Cheng Perng Phoo, Utkarsh Mall, Carl Vondrick, Kavita Bala, et al. Towards Ilm agents for earth observation. arXiv preprint arXiv:2504.12110 2025.
Ioannis P. Kokkoris, Bruno Smets, Lars Hein, Giorgos Mallinis, Marcel Buchhorn, Stefano Balbi, Ján Černecký, Mare Paganini, and Panayotis Dimopoulos. The role of earth ob- servation in ecosystem accounting: A review of advances, challenges and future directions. Ecosystem Services, 70:101659, 2024. ISSN 2212-0416. doi:
2024.101659. URL https: http:p:://wwwwcee www.scienceiirctt com www.//iienc//////ccccccccoccceea http:///eeeaiiccpep.....
52212041 600006664
Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. Geochat: Grounded large vision-language model for remote sensing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 27831-27840, 2024.
Alexandre Lacoste, Nils Lehmann, Pau Rodriguez, Evan Sherwin, Hannah Kerner, Björn Lütjens, Jeremy Irvin, David Dao, Hamed Alemohammad, Alexandre Drouin, et al. Geo-bench: Toward foundation models for earth monitoring. Advances in Neural Information Processing Systems, 36:51080-5111 51093, 2023.
Jinpeng Li, Jun He, Weijia Li, Jiabin Chen, and Jinhua Yu. Roadcorrector: A structure-aware road extraction method for road connectivity and topology correction. IEEE Transactions on Geoscience and Remote Sensing, 62:1-18, 2024a.
Ming Li, Keyu Chen, Ziqian Bi, Ming Liu, Benji Peng, Qian Niu, Junyu Liu, Jinlang Wang, Sen Zhang, Xuanhe Pan, et al. Surveying the milm landscape: A meta-review of current surveys. arXiv preprint 2rXiv:2409.18999 2024b.
Weijia Li, Runmin Dong, Haohuan Fu, Jie Wang, Le Yu, and Peng Gong. Integrating google earth imagery with landsat data to improve 30-m resolution land cover mapping. Remote Sensing of Environment, 237: 237:111565 11563, 2020.
Weijia Li, Yawen Lai, Linning Xu, Yuanbo Xiangli, Jinhua Yu, Conghui He, Gui-Song Xia, and Dahua Lin. Omnicity: Omnipotent city understanding with multi-level and multi-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 17397-17407, 2023a.
Weijia Li, Wenqian Zhao, Jinhua Yu, Juepeng Zheng, Conghui He, Haohuan Fu, and Dahua Lin. Joint cnraaticc-comettie learning for polygonal building segmentation from high-resolution re- mote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 201:26-333 2023b.
Weijia Li, Haote Yang. Zhenghao Hu, Juepeng Zheng, Gui-Song Xia, and Conghui He. 3d building reconstruction from monocular remote sensing images with multi-level supervisions. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27728- 27737, 2024c.
12

---

Published as a conference paper at ICLR 2026
Xiang Li, Jian Ding, and Mohamed Elhoseiny. Visbench: A versatile vision-language benchmark dataset for remote sensing image understanding. Advances in Neural Information Processing Systems, 37:3229- 3242, 2024d.
Yuxuan Li, Xiang Li, Yunheng Li, Yicheng Zhang, Yimian Dai, Qibin Hou, Ming-Ming Cheng, and Jian Yang. Sm3det: A unified model for multi-modal remote sensing object detection. arXiv preprint arXiv:2412.20665 2024c.
Zhao-Liang Li, Bo-Hui Tang, Hua Wu, Huazhong Ren, Guangjian Yan, Zhengming Wan, Isabel F. Trigo, and José Sobrino. Satellite-derived land surface temperature: Current status and per- spectives. Remote Sensing of Enviromment, 131:14-37, 37, 2013, ISSN 0034-4257. doi: https://doi.
htg/100..01o//..c2211222..0.00 atticle/ppt//0034422333
URL https: www. www.sseiieeencceeeeeeeeeeeeeee.e com/ science//////// 749
Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arX15:2311.0757,, 2023.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 2rXiv:2412.19437, 2024a.
Chenyang Liu, Keyan Chen, Haotian Zhang. Zipeng Qi, Zhengxia Zou, and Zhenwei Shi. Change- agent: Toward interactive comprehensive remote sensing change interpretation and analysis. IEEE Transactions on Geoscience and Remote Sensing, 62:1-16, 2024b. 3425815.
doi: 10. 109//00SSSS202244
Fan Liu, Delong Chen, Zhangqingyun Guan, Xiaocong Zhou, Jiale Zhu, Qiaolin Ye, Liyong Fu, and Jun Zhou. Remoteclip: A vision language foundation model for remote sensing. 1EEE Transactions on Geoscience and Remote Sensing, 2024c.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 26296-26306, 2024d.
Zhuoran Liu, Danpei Zhao, Bo Yuan, and Zhiguo Jiang. Rescueadi: adaptive disaster interpretation in remote sensing images with autonomous agents. IEEE Transactions on Geoscience and Remote Sensing, 2025.
Sylvain Lobry, Diego Marcos, Jesse Murray, and Devis Tuia. Rsvqa: Visual question answering for remote sensing data, IEEE Transactions on Geoscience and Remote Sensing. 58(12):8555-855) 2020.
Siqi Lu, Junlin Guo, James Zimmer-Dauphinee, Jordan M Nieusma, Xiao Wang, Steven A Wernke, Yuankai Huo, et al. Vision foundation models in remote sensing: A survey. 1EEE Geoscience and Remote Sensing Magazine, 2025.
Jingjing Ma, Wei Jiang, Xu Tang, Xiangrong Zhang, Fang Liu, and Licheng Jiao. Multiscale sparse cross-attention network for remote sensing scene classification. IEEE Transactions on Geoscience and Remote Sensing. 2025.
Zixian Ma. Weikai Huang. Jieyu Zhang, Tanmay Gupta, and Ranjay Krishna. m & m's: A bench- mark to evaluate tool-use for m ulti-step m ulti-modal tasks. In European Conference on Com- puler Vision, pp. 18-34. Springer, 2024.
Utkarsh Mall, Cheng Perng Phoo, Meilin Kelsey Liu, Carl Vondrick, Bharath Hariharan, and Kavita Bala. Remote sensing vision-language foundation models without annotations via ground remote alignment. arXiv preprint arXiv:2312.0666 2023.
Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. In The Tweifth International Conference on Learning Representations, 2023.
13

---

Published as a conference paper at ICLR 2026
David Montero, Cesar Aybar, Miguel D. Mahecha, Sebastian Wieneke, et al. A standardized cata- logue of spectral indices to advance the use remote sensing in earth system research. Scientific Data, 10(1):197, 2023. doi: 10. 10.1038444199--02-00096-000 URL sttpps://oot org/IC.
1038/s41599 023- 0202--0200600
Dilxat Muhtar, Zhenshi Li, Feng Gu, Xueliang Zhang, and Pengfeng Xiao. Lhrs-bot: Empowering remote sensing with vgi-enhanced large multimodal language model. In European Conference on Computer Vision, pp. 440-457. Springer, 2024.
OpenAI. Hello gpt-4o.
2024.
OpenAl. Gpt-5 is here.
5/| 2025a.
OpenAl.
Introducingoperator.
http:://opemai.commmnddd
2025b.
Chao Pang. Xingxing Weng, Jiang Wu, Jiayu Li, Yi Liu, Jiaxing Sun, Weijia Li, Shuai Wang, Litong Feng, Gui-Song Xia, al. Vhm: Versatile and honest vision language model for remote sensing image analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp- 6381-6388, 2025.
Dmitrii Pantiukhin, Boris Shapkin, Ivan Kuznetsov, Antonia Anna Jost, and Nikolay Koldumov. Ac- celerating earth science discovery via multi-agent Ilm systems. arXiv preprint arXiv:2503.05854 2025.
Chen Qian and Xin Cong. Communicative agents for software development. arXiv preprint arXiv:2307.07924 6(3):1, 2023.
Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.007. 000000007444 14, 2024.
Partha Pratim Ray. A survey on model context protocol: Architecture, state-of-the-art, challenges and future directions. Authorea Preprints, 2025.
Xubin Ren, Lingrui Xu, Long Xia, Shuaiqiang Wang, Dawei Yin, and Chao Huang. Vide- orag: Retrieval--uugeene generation with extreme long-context videos. arXiv preprint arXiv:2502.01549. 2025.
Farhad Samadzadegan, Ahmad Toosi, and Farzaneh Dadrass Javan. A critical review on multi- sensor and multi multi-platfone platffrm remote sensing data fusion approaches: current status and prospects. International journal of remote sensing, 46(3): 1327-1402, 2025.
Akashah Shabbir, Muhammad Akhtar Munir, Akshay Dudhane, Muhammad Umer Sheikh, Muham- mad Haris Khan, Paolo Fraccaro, Juan Bernabe Moreno, Fahad Shahbaz Khan, and Salman Khan. Thinkgeo: Evaluating tool-augmented agents for remote sensing tasks. arXiv preprint arXiv:2505.23755 23355. 2025.
Ahmed Shaker, Wai Yeung Yan, and Paul E LaRocque. Automatic land-water classification using multispectral airborne lidar data for near-shore and river environments. ISPRS journal of pho- togrammetry and remote sensing, 152:94-108, 2019.
Minjie Shen, Yanshu Li, Lulu Chen, and Qikai Yang. From mind to machine: The rise of manus ai as fully autonomous digital agent. arXiv preprint arXiv:2505.0203: 2025.
Zhenwei Shi and Zhengxia Zou. Can machine generate humanlike language descriptions for remote sensing image? IEEE Transactions on Geoscience and Remote Sensing, 55(6):3623- 3634, 2017.
Chenglei Si, Diyi Yang, and Tatsumori Hashimoto. Can Ilms generate novel research ideas? large- scale human study with 100+ nlp researchers. arXiv preprint XXXx:2409.044099 2024.
14

---

Published as a conference paper at ICLR 2026
Hao Su, Shunjun Wei, Min Yan, Chen Wang, Jun Shi, and Xiaoling Zhang. Object detection and instance segmentation in remote sensing imagery based on precise mask r-cmn. IGARSS 2019- 2019 IEEE International Geoscience and Remote Sensing Symposimm, pp. 1454-1457. IEEE, 2019.
Qiushi Sun, Zhoumianze Liu, Chang Ma, Zichen Ding, Fangzhi Xu, Zhangyue Yin, Haiteng Zhao, Zhenyu Wu, Kanzhi Cheng, Zhaoyang Liu, et al. Scienceboard: Evaluating multimodal au- tonomous agents realistic scientific workflows. arXiv preprint arXiv:2505. 19897, 2025.
Jiabin Tang, Lianghao Xia, Zhonghang Li, and Chao Huang. Ai-researcher: Autonomous scientific innovation. arXiv preprint arXiv:2505. 18705, 2025.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi K2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025.
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.106711 2024.
Minyang Tian, Luyu Gao, Shizhuo Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al. Scicode: A research coding benchmark curated by scientists. Advances in Neural Information Processing Systems, 37:30624-30650, 2024.
Julie Transon, Raphaäl D' A'Adrioont, Alexandre Maugnard, and Pierre Defourny. Survey of hy- perspectral earth observation applications from space in the sentinel-2 context. Remote Sensing, 10(2), 2018. ISSN 2072-4292. 2072 42224222//00//157
doi: 10:30177 URI. https: www mdp com/
Mojtaba Valipour, Kelly Zheng, James Lowman, Spencer Szabados, Mike Gartner, and Bobby Braswell. Agi for the earth, the path, possibilities and how to evaluate intelligence of models that work with earth observation data? arXiv preprint arXiv:2508.06057, 2025.
CJ Van Westen. Remote sensing for natural disaster management. International archives of pho- togrammetry and remote sensing, 33(B7/4; PART 1):1699-1617, 2000.
Jize Wang, Ma Zerun, Yining Li, Songyang Zhang, Cailian Chen, Kai Chen, and Xinyi Le. Gta: benchmark for general tool agents. Advances in Neural Information Processing Systems, 37: 75749-75790, 2024a.
Junjue Wang, Zhuo Zheng, Zihang Chen, Ailong Ma, and Yanfei Zhong. Earthvqa: Towards queryable earth via relational reasoning-based remote sensing visual question answering. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 38, pp. 5481-5489, 2024b.
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024c.
I Luting Wang, Yinghao Xiang, Hongliang Huang, Dongjun Li, Chen Gao, and Si Liu. Towards realistic casth-observation constellation scheduling: Benchmark and methodology. arXiv preprint arXiv: arXiv:2510.26291 2025a.
Qi Wang, Wei Huang, Xueting Zhang, and Xuelong Li. Word-sentence framework for remote sensing image captioning. IEEE Transactions Geoscience and Remote 5tmsinng59(1221105333 10543, 2020.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508. 18265, 2025b.
Xiaohan Wang, Yuhui Zhang, Orr Zohar, and Serena Yeung-Levy. Videoagent: Long-form video understanding with large language model as agent. In European Conference on Computer Vision, pp. 58-76. Springer, 2024d.
15

---

Published as a conference paper at ICLR 2026
Zhecheng Wang, Rajanie Prabha, Tianyuan Huang, Jiajun Wu, and Ram Rajagopal. Skyscript: A large and semantically diverse langual dataset for remote sensing. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 5805-5813, 2024c.
Marek Wójtowicz, Andrzej Wójtowicz, Jan Piekarczyk, al. Application of remote sensing meth- ods in agriculture. Communications in biometry and crop science, 11(1):31-50, 2016.
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68(2):121101 2025.
Gui-Song Xia, Jingwen Hu, Fan Hu, Baoguang Shi, Xiang Bai, Yanfei Zhong, Liangpei Zhang, and Xiaoqiang Lu. Aid: A benchmark data set for performance evaluation of aerial scene classifica- tion. IEEE Transactions on Geoscience and Remote Sensing, 55(7):3965-3981. 2017.
Gui-Song Xia, Xiang Bai, Jian Ding, Zhen Zhu, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, and Liangpei Zhang. Dota: A large-scale dataset for object detection aerial images. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3974-3983, 2018.
Junlin Xie, Zhihong Chen, Ruifei Zhang. Xiang Wan, and Guanbin Li. Large multimodal agents: A survey, arXiv preprini 240.
2024.
Zhitong Xiong, Fahong Zhang, Yi Wang, Yilei Shi, and Xiao Xiang Zhu. Earthnets: Empowering ai in earth observation. arXiv preprint arXiv:2210.066 04436, 2022.
Weikai Xu, Chengrui Huang, Shen Gao, and Shuo Shang. LIm-based agents for tool learning: A survey: W. xu et al. Data Science and Engineering, pp- 1-31,2025,
Wenjia Xu, Zijian Yu, Boyang Mu, Zhiwei Wei, Yuanben Zhang, Guangzuo Li, and Mugen Peng. Rs-agent: Automating remote sensing tasks through intelligent agent. arXiv preprint arXiv:2406.0700 2024a.
Yiheng Xu, Dunjie Lu, Zhennan Shen, Junli Wang, Zekun Wang, Yuchen Mao, Caiming Xiong, and Tao Yu. Agenttrek: Agent trajectory synthesis via guiding replay with web tutorials. arXiv preprint arXiv:2412.09600 2024b.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.003388 2025.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Yang Zhan, Zhitong Xiong, and Yuan Yuan. Rsvg: Exploring data and models for visual grounding on remote sensing data. IEEE Transactions on Geoscience and Remote Sensing, 61:1-13, 2023.
Yang Zhan, Zhitong Xiong, and Yuan Yuan. Skyeyegpt: Unifying remote sensing vision-language tasks via instruction tuning with large language model. ISPRS Journal of Photogrammetry and Remote Sensing, 221:64-77, 221:64-7,,2025. 2025.
Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges. arXiv preprint arXiv:2401.07339, 2024a.
Wei Zhang, Miaoxin Cai, Tong Zhang, Yin Zhuang, and Xuerui Mao. Earthgpt: A universal multi- modal large language model for multi-senson image comprehension in remote sensing domain. IEEE Transactions on Geoscience and Remote Sensing, 2024b.
Baichuan Zhou, Haote Yang, Dairong Chen, Junyan Ye, Tianyi Bai, Jinhua Yu, Songyang Zhang, Dahua Lin, Conghui He, and Weijia Li. Urbench: comprehensive benchmark for evaluating large multimodal models in niet urban scenarios. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 10707-10715, 2025.
16

---

Published as a conference paper at ICLR 2026
A DATASET ILLUSTRATION
A.1 DATASET COMPOSITION
The remote sensing imagery used in our dataset primarily comes from Landsat and MODIS, with additional high-resolution imagery sourced from open datasets. All the remote sensing data prod- uets are obtained through Google Earth Engine (GEE). The following Table is more detailed breakdown of the data sources, products, and their distribution:
Table 4: Sensor and Data Source Statistics
Category
Sensor/Data Source
Dataset
Samples
Spectrum
Landsat
Landsat 8 9
1684
MODIS
10273
ASTER
110
Products
GEE
LST
183
NDVI
369
GPM
160
VIRS
164
GHSL
QA PIXEL
212
NDWI
69
fire MaxFRP
194
RGB
Public Datasets
AID
169
DIOR-RSVG
7
DOTA
22
NWPU-VHR-10
21
xBD
32
Total
13737
This table provides an overview of the data sources and the distribution of samples across three categories (Spectrum, Product, RGB). includes a diverse set remote sensing products, such as Landsat, MODIS, ASTER, and various other publicly available high-resolution datasets, ensuring comprehensive coverage for wide range of Earth observation tasks.
A.2 BENCHMARK STATISTICS
Table 5: Statistics of the benchmark dataset, including average verification code length, number of images, query length, reasoning steps, and question counts for different task types.
Avg. Query Length
Type
Avg. Code Lines
Avg. Images
Avg. Steps
Question Count
IF
Spectrum Products
361.19
96.82
331.54
497.94
4.38
100
283.64
43.23
505.72
648.09
6.35
88
RGB
176.42
4.18
333.80
464.70
5.77
60
Avg. Total
288.97
55,39
393.89
543.18
5.42
71664
13737
97685
134708
1343
248
As illustrated in Table [] Earth-Bent Bench consists of 248 questions associated with approximately 13.7K images. We recruited team of domain experts to annotate these questions. The annotation
17

---

Published as a conference paper at ICLR 2026
process follows: experts first designed solution steps based on their expertise, then selected appropriate tools from the Agent Toolkit, implemented the steps by writing Python code to invoke the selected tools for data processing, and finally generated the corresponding answers. In total, the expert team annotated 1345 solution steps and produced 71, 71,664 lines of verification code for the 248 benchmark questions.
A.3 QUESTION TYPES AND CATEGORY
As shown in Figure Earth Earth-Bench questions categorized into three types based their data sources: Spectrum, Products, and RGB. The Table/6bbelow summarizes the number and proportion of questions within each category:
Table 6: Distribution of Question Types and Their Proportions
Question Types
Number
Proportion (%)
Temperature Monitoring
44
17.74
Weather Forecasting
4.44
Climate Analysis
20
8.06
Water Management
22
8.87
Pollution Regulation
14
5.65
Vegetation Monitoring
28
11.29
Disaster Judgement
9.68
Urban Management
25
10.08
Classification
15
6.05
Detection
15
6.05
Grounding
2.82
Segmentation
1.21
Counting
2.82
Change Detection
13
5.24
This table provides detailed distribution of question types within Earth-Bench, reflecting the vari- ety Earth observation tasks addressed by the dataset.
A.4 QUERY REGIMES
Earth-Bench categorizes each task into two regimes: Auto-Planning and Instruction- The key distinction is that in Instruction- Followiin, the query explicitly provides the Agent with solu- tion approach, while preserving the original intent the task. As shown Table 5 the statistics of query length highlight this difference: on average, queries in the Instruction-Folloingg regime are about 150 characters longer than those Auto-Planning, due to the inclusion of solution guid- ance. For illustration, Appendix H presents examples of the same task under both regimes, along with the performance of different L.LMs. In summary, Eestruction-Folowwe foollcti and tool-use capabilities, whereas Auto-Planning additionally evaluates their ability to decompose and plan Earth observation tasks.
emphasizes LLMs'
A.5 DATASET QUALITY CONTROL.
Our annotation team was composed of 2 computer science experts, 7 remote sensing specialists (including professor) and 3 Earth science specialists (including professor).
Three core authors who served as senior reviewers led the pipeline of dataset construction. Each senior reviewer responsible for guiding the development of the question sets and templates for
18

---

Published as a conference paper at ICLR 2026
the Spectrum, Products, and RGB categories. They played a key role in providing strategic direction for the overall question pool.
The remaining team members, consisting of graduate students and senior undergraduates, tributed in the following areas:
Creation of initial questions (approximately 400 questions)
Raw data collection
Development lython-based solution scripts
Once the questions were created, they were thoroughly reviewed by the senior reviewers. The review process focused on two key criteria:
Ensuring that the raw data involved (c.g., Landsat or MODIS) has com- plete band information within the task's time range and does not contain anomalies. Any questions and TIF files with missing or anomalous large values were discarded.
Data Integrity:
2. Task Difficulty Assessment: Senior reviewers assessed the difficulty of questions based the number and complexity of the functions defined in the Python solutions. For simpler tasks (e.g., only calculating NDVI index to finish a task), these were removed to ensure an appropriate challenge across questions.
This collaborative structure ensured that the dataset was curated by a diverse team with expertise from the full landscape of Earth observation fields, enabling a well-rounded and comprehensive dataset for evaluation.
A.6 BIAS ABLATION EXPERIMENT
To examine whether BerthBBench exhibits bias toward specific models, i.e., whether certain models inherently find its tasks easier and thus risk skewing conclusions, conducted an ablation study. Specifically, we removed all tools and allowed LLMs to directly answer questions given only the Query and Folder, then compared the results with those of tool-augmented Agents that had access to both Query and Data.
Model Performanee: No tool vs Tool use
Lauge Improveeent
No Tool
Band
Maderatio improvement
Product
Tool Use
Krak improvement
RGB
Base64 Encoding
Accuracy
50
40
30
10
GPTS-A
GFT5-B
Gomin -2, 5-A
Gomini-2 5-B
GPT- GPT-to-A
GPT-4o-B
DeepSeek-VPp V3, I-d
DeepSook- VB. 1-B
Kimik2-A
Kimik2-B
Peent may -B
Peord max
Figure 6: Dataset Ablation Experiment
As shown in Fig. 6 without tool access, mainstream LLMs achieved comparable performance across the three task types, with overall Accuracy of about 37%. This indicates that the dataset is fair in its raw form and that models cannot rely solely on parametric knowledge to solve all benchmark questions. In contrast, with tool access, different models exhibited varying degrees of improvement:
19

---

Published as a conference paper at ICLR 2026
GPT-5 achieved the largest gain, whereas GPT-4o showed modest increase. These results highlight differences in problem decomposition and tool-use capabilities among LLMs, and further corroborate the conclusions presented in the main experiments.
In the No Tool setting, LLMs/MLLMs were restricted from using any tools, and basc64 encoding was not applied in RGB tasks specifically. To ensure integrity in the comparison, we added exper- iments with base64 encoding for RGB tasks. We observed that while models like GPT-5 showed some improvement in accuracy (compared with not applying base64 encoding). the performance remained at relatively low levels. DeepSeek and Kimik2 were unable to handle RGB tasks yet. The conclusions derived from the ablation study remain unchanged.
B EVALUATION
B. EVALUATION MODEL
Our evaluation covers 13 recent LLMs, including both closed-source and open-source ones, to un- derstand their capabilities across multiple Earth observation tasks. The baseline models are listed in Table 7
Table 7: Models evaluated in our benchmark and their corresponding API references.
Model
Model Version
Links
GPT5
GPT-5
https://plat..ormoooppaa
gpt-5
GPT4
GPT-4o
http::////arrroo..paaaaa.cca///p gp
Gemini
Gemini-2.5-Flas
1odellssgeminn2255-flaa
Mistral
Mistral-Large
models/
Qwen3-Max-Previn
http::/www.albbbacl...
Qwen
Qwen3-32B
com//ealpeen/mm0061-suudd
Qwen-Plus
use use-qwen bue--wan-bb---lllinnaaa
Kimi
Kimi-K2
httas://placfomm.nnnnscht awteetttaaddooooo aw/aaas///ooaa//eee /utts///./////cccciii//.///// guide/
hwt.:/ooonnoot...ooetoooooeo.. start-using-kkii-app
Deepseek
Deepseek- V3.1
tttps://ppi-bcccs.dessseeek com
Seed
Seed-1.6
http://wwww.taeeegggaacccoooodoo 1099455
LLaMA
GLM
GLMM-4.5v
Intern
VL-335
document
B.2 EVALUATION METRIC
Formally, for each task goal g. our geoscience experts provide (i) a ground-truth final answer y", and (ii) an expert-annotated reasoning trajectory.
-" = [(ob, aj),
.... (otn a=)]
where each action is defined as
= (4i, ing, oulz),
with E E V denoting the tool identifier (from the tool vocabulary), ing E X' the input arguments, and out E O the corresponding output. In other words, each tool is characterized by its name in the vocabulary, its input arguments, and its output results.
20

---

Published as a conference paper at ICLR 2026
Given policy the agent generates output trajectory
[(00, ao), (01,a1), .... (Ons an)].
together with predicted final answer y,
To comprehensively evaluate the performance of the Agent on the Earth Benchmark, we assess its execution process from two perspectives: End-to-End and Step-by-Step. The corresponding evaluation metries are defined as follows:
End-to-End protocol. End-to-end metrics evaluate the task-level performance of the agent, inde- pendent of its intermediate reasoning. We consider two complementary measures:
(1) Accuracy. The correctness of the final answer is computed as
Acc B y²" y²}}
where 1[-} is the indicator function and G is the distribution of benchmark tasks.
(2) Efficiency. To penalize unnecessarily long trajectories, we measure the relative optimality of tool usage:
Bff(r, r*)
|++
where |r| and [7*| denote the number of tool calls in the predicted and ground-truth trajectories, respectively.
Step-by-Step protocol. In addition to outcome-based metries, we also evaluate the fidelity of the reasoning trajectory relative to expert annotations. Let t* (fi...., fm) and (f1...., tx) denote the tool sequences, and in" (in in (in m,,) the corresponding parameter sequences. We define four metrics:
(1) Aoyos-Any-Oderr (TAO). Coverage of required tools, ignoring order and duplicates:
TAO(T,
(3)
See((t))
where Set(-) extracts the set of unique tools.
(2) Tools-In-OOder((TIO) Fraction of ground-truth tools matched subsequence in the predicted sequence:
A- = max { 313 f1 < j& S n, t,. = tú vi s k}. TIO(T, = k*
(4)
m
(3) Mool-Fxact-Maacc(TEaa Length of the longest common prefix (I.CP), normalized by the ground-truth length:
Aep m
lee max (Es min(m, - vi s), TEM(T,7")
(4) Parameter Accuracy: Exact match of both tool identifiers and arguments under the prefix rule:
fparam
Eparam max { é min(m, n) :4 in; in; Vi < e Sparam
(6)
m
Here in; in denotes structural equality of arguments (e.g., dictionary match).
C BREAKDOWN RESULTS ON DIFFERENT MODALITIES
Table 8
Table 9 and Table 10 present the detailed evaluation results on different subsets of Earth-
Bench. In the main analysis, report only the overall performance across the entire benchmark to ensure clarity and comparability. Nevertheless, the breakdown results individual subsets provide valuable insights into potential directions for improving LLMs in Earth Observation tasks.
21

---

Published as a conference paper at ICLR 2026
Table 8: Performance of different LLM-based agents on the Spectrum subset of Earth-Bench. We bold the best results and underline the runner-ups.
Accuracy
Emfciency
Tool-Any-Order
Tool-In-Orde
Tool-Exact-Matel
Parameters
Model
AP
IF
AP
IF
AP
IF
AP
IF
AP
IF
AP
IF
GPT-5
61.00
64.00
3,5510
4.6657
72.67
78.32-
65.80
24.71
49.64
52.07-
21.74
24.97,
Gemini-2.5
52.00
52.00
4,3584
4.5585
71.35
71,90,
65.14
64.84
40.12
49.89,
17.57
21.41,
GPT-4o
37,00
42.00
3.7736
4.5726
69.55
72.83
56.12
62.07
48.27
51.02.
21.62
24.08
Kimik2
38.00
50.00
2.5758
3.1005
71.92
86.02:
62.27
78.91
43.04
54.81
20.73
25.62
DeepSeek-V3.I
47.00
45.00
3,9014
4,0685
75.97
67.64
66.57
50.22
50.52,
26.32
26.13
Qwen3 Max
45.00
40.00
3.1981
3.2864
77.67
74.27
66.47
65.58
33.97
48.54
16.04
24.43,
Seed-1.6 1.6
40.00
57.00-
1.7525
1,9186
55.07
63.92
42.00
54.94
24.53
34.37
12.56
16.83,
LLaMA-4
36,00
37.00
0.3648
0.4275
16.89
25.20
3.57
18.26-
2.69
13,58
2.02
8.17
Qwen- Piven-Puus Plus
33.00
36.00
2. 2,2833
2.4157
55.95
57.38
36.27
48.89
5.67
35.14,
2.87
17.47
GLM-4.5v
33.33
28.28
3.1121
3,0709;
47.53
49.87
41.93
45.63
14.26
25,22,
9.13
16.41;
Mistral
24.00
18.00.
1,3825
0.8316;
23.73
19.58
4.58
16.13
1.87
13.37
1.33
6.15
Qwen3-32B
12.00
29.00
4.3328
3,4380;
45.02
65.25
26.60
57.17-
5.53
38.86
3.43
20.52;
IntemVL-3.5
19.00
25.00
0.1127
0.2411
7.50
18.77;
3.58
16.09
0.58
13.02;
0.33
5.65.
On the Spectrum subset, the accuracy of the Agent's responses is generally below the average, and the overall efficiency is also lower than the benchmark mean. This indicates that the Agent encounters significant difficulties when addressing tasks in this subset. A likely reason is that the LLMs involved in the evaluation have limited familiarity with processing raw Earth Observation data. Furthermore, tasks in this subset often require analyzing larger number of images, making them inherently more challenging compared to tasks in other subsets.
In contrast, the Product subset, the Agent's responses substantially above the average in terms of accuracy, and its efficiency is comparable to that of expert annotations. This suggests that LLMs are more adept at handling structured, product-level information, where tasks often align with general reasoning and statistical capabilities rather than requiring specialized domain expertise.
Table 9: Performance of different LLM-based agents on the Products subset of Earth Barth-Bench. We bold the best results and underline the runner-ups.
Accuracy
Efficieccy
Tool-Any-Orde
Tool-In-Order
Teol-Exact-Match
Parameters
Model
AP
IF
AP
AP
IF
AP
GPT-5
75.00
71.59
1,7154
1,6190,
60.04
62,35,
38.43
40.44
31.52
34.03,
17.62
16.75,
Gemini-2.5
62.50
63.64-
3.0055
1.1600;
48.94
51.46:
33.82
35.26
29.54
27.63,
16.33
17.11;
GPT-4o
54.55
54.55
1.1800
1.5270,
57.27
60.11
33.89
37.80-
31.31
35.02-
19.13
18.71
Kimik2
62.50
60.23
1,2489
1.6481,
66.91
69.83,
42, 22.99
49.96
34.75
38.84
20.23
21.32
DeepSeek-V3.1
50.00
59.09
1.8111
1.6449;
73.48
72.75
43.35
46. 16
32.18
33.32;
20.50
19.92.
Qwen3 Max
56.82
61.36-
1,0688
1.0859
63.24
66.80;
40.30
44.78-
33.22
30.24
20.23
16.52.
Secd-1.6
65.06
67.05
0.8776
0.9359
54.02
55.57
36.75
37.20
28. 84
31.27
16.79
19.82
I.LaMA-4
47.73
9.83
2.51
2.61
1.31
Qwen- Plus
53.41
40.91
0.9972
1.0016
47.71
51.57
25.33
27.37
10.16
4.14
6.52
2.97
GLM-4.5v
43.18
47.67
0.9342
1.1979.
35.06
41.97-
19.36
24.68
8.77
9.04
6.60
7.77-
Mistral
36.36
22.73.
0.6263
1.0206
26.61
30.36;
12,80
13.43
8.83
10.40;
5.42
4.40.
Qwen3-32B
27.27
18.39
2.2108
0.6987,
29.66
6.97
11.81
1.93
2.70
1.14.
2.07
1.14.
LL-35
36.36
28.41
0.0495
0.0424
4.91
1.94
2.32
0.52
2.26
0.52
0.55.
For the RGB subset, the Agent demonstrates above-average performance in tool utilization and achieves efficiency close to that of expert annotations. However, its response accuracy remains substantially below the average. This limitation closely tied the capabilities of the tools within the Perception Toolkit. In certain cases, even when the Agent selects the same tools those used in expert annotations, the outputs still diverge from the ground-truth answers due to the constraints of the underlying expert models. As the first attempt to develop Agent framework for Earth Observation, our work highlights this challenge and encourages future EO Agent research to adopt more advanced expert models in order overcome these limitations.
22

---

Published as a conference paper at ICLR 2026
Table 10: Performance of different LLM-based agents on the RGB subset of Earth-Bench. We bold the best results and underline the runner-ups.
Accuracy
Emiciency
Tool-Any-Order
Tool-In-Order
Teol-Exact-Macee
Parameters
Model
AP
IF
AP
IF
AP
IF
AP
IF
AP
IF
AP
IF
GPT-5
$9.32
49.15
1.5312
1,5784:
76.61
72.09
75.04
67.08.
59.60
52.71;
46.15
40.73;
Gemini-2.5
47.46
47.46
0.7926
0.8878
48.70
60.11
29.38
50.11
21.33
45.54,
19.07
36.75,
GPT-4o
45.76
35.59
0.8779
0.8939
71.86
66.89
66.60
61.13
64.76
60.00
46.65
47.54
Kimik2
54.24
62.71
1.5311
1.4104;
75.65
80.62
71.37
79.90-
$1.48
63.32
43.12
52.19;
DeepSeek-V3.I
55.93
54.24
1.7966
89.21
85.49
87.64
71.54
74.05
53.57
57.22
Qwen3-Mre Max
49.15
38.98
0.8601
0.9785
65.25
68. 14
50.28
56.58
47.88
51.55
34.04
43.93,
Seed- 1.6
55.93
52.54
1.1948
0,9589
57.99
$7.59
44.00
47.30
34.11
43.74;
29.93
39.02;
LLaMA-4
37.29
27.12
0.3464
0,3790
27.20
36.52;
0.47
15.61
0.47
12.43
0.47
11.25
Qwen Plus
42.37
38.98
0.9721
1.0488
51.89
60.11;
29.51
42.80
24.18
41.07,
23.62
36.89,
GLM-4.5v
16.95
28.81
1.3070
1.4535
47.91
48.42
27.36
33.54
19.73
24.98
Mistral
30.51
30.51
0.7215
0.7531
36.16
45.59
22.46
40.11
21.89
37.71
19.97
31.84;
Qwen3-32B
25.42
27.12
0.7767
1.0891
45.93
56.47
27.54
41 41.69:
26.41
41.69;
25.28
37.74,
IntemVL-335
25.42
27.12
0.2398
0.2606
16.95
29.92
7.23
13.58
6.67
13.58,
4.75
11.88
Overall, the comparative analysis across subsets highlights both the strengths and limitations of LLM-based Agents in Earth Observation. While LLMs achieve relatively strong performance on Product tasks, where success relies more on general reasoning and statistical skills, they remain less effective on tasks that demand specialized knowledge, such those the Spectrum subset, which involve interpreting raw spectral data. Moreover, Earth-Agent incorporates expert models within the Perception Toolkit for tasks such as segmentation and object detection, which significantly improves performance in the corresponding scenarios. However, the generalization ability of these expert models remains limited, as their outputs do not always align with ground-trui -trun answers, even when the correct tools are selected. These findings suggest that future progress in EO Agents will depend not only on enhancing LLMs with -speciff knowledge, but also on developing more robust and versatile expert models to ensure reliable performance across the diverse spectrum of Earth Observation tasks.
D ERROR ANALYSIS
To analyze the errors made by Agents with different LLM backbones on Barth-Bench tasks, we selected GPT-5 as representative closed-source model, and Kimi-K2, Qwen3-Max, and Deepseek- V3.1 as representative open-source models. We counted the number of errors and categorized them into five types:
Unaware of Termination Conditions: failure to recognize the task's 's termination condition, leading to repeated tool calls until reaching the maximum step limit;
Tool Hallucination: attempts to invoke non-existent tools;
File Hallucination: attempts to process non-existent files, i.c. providing invalid file or folder paths as tool inputs;
Invalid Parameters: inputs that do not conform to the expected parameter format or are otherwise invalid;
System Error: system-level failures caused by the runtime environment or external depen- dencies.
Figure 7 presents the frequency and distribution of these error types. Results show that GPT-5 produced the largest number of errors, while Kimi-K2 had the fewest. Except for GPT-5, the other models exhibited similar error counts across different regimes, and their error distributions did not vary significantly, suggesting that providing more detailed execution steps does not substantially improve tool-use proficiency.
In terms of error types, GPT-5 errors were dominated by Invalid Parameters, with occasional Sys- tem Errors and File Hallucinations, but Tool Hallucinations. In contrast, the three open-source
23

---

Published as a conference paper at ICLR 2026
models demonstrated different patterns: while Invalid Parameters remained notable factor, was not the primary source of errors. Instead, nearly 60% of their errors stemmed from Halluci- nations and Unaware of Termination Conditions. We hypothesize that this difference is related to training strategies. Open-source models are often trained with reinforcement learning, which may encourage more exploratory outputs, thereby increasing the likelihood of hallucinations. Moreover, their reward functions are typically designed to shape behavioral style and output preferences rather than enhance factual knowledge, which could make models more prone to generating divergent or repetitive outputs and to overlooking termination conditions.
Error Count Analysis Across Models
300
Error Types
250
Unzware of Termination Conditions
Tool Hallucination
System Error
File Hallucisation
Invalid Panometers
150
100
GPT5-AP AP
6PT5-IF
Kimi AP
Kimi-K2-IF
Qwan3 Max
Qwan3 Mox-IF IF
beepseak-V3.1.4
Deepstokk-3.1.IF
GPT5-AP
GPT5-IF
Kimi-K2-AP
Kimi-K2-IF
23.9%
24.2%
24.2%
6.8%
7.7%
81.8%
2.3%
76.0%
3.0%
0.9%
3.2%
9.1%
13.3%
10.6%
14.7%
35.4%
33.7%
Qwen3-Max-AP
Qwen3-Mex-IF
Deepseek-V3.1-4
Deepseek-V3.tk
29.5%
26.8%
18.2%
30.5%
26.5%
4.8%
7.8%
8.9%
22.9%
22.3%
36.1%
43.2%
Figure 7: Error analysis.
E LATENCY EXPERIMENT
E. BREAKDOWN RESULTS ON LATENCY
We further break down the latency into LLM Latency and Tool Latency. It evident that the majority of the latency is spent model calls (LLM Latency), rather than external tool calls (Tool Latency). This suggests that reducing the frequency of model calls could further improve latency. The results are shown in Table
We observe that the majority of the latency is attributed to LLM Latency, while the impact Tool Latency is relatively minimal. Therefore, reducing the frequency of model calls could significantly improve overall latency. This suggests that optimizing the model call frequency would further en- hance system performance.
24

---

Published as a conference paper at ICLR 2026
Table 11: Latency Breakdown for Different Models on Barth-Bench-Lite
Model
Latency
Tool Call Number
LLM Latency
Tool Latency
Earth Earth-Agent (GPT)
9494.92s
669
6969.26s
2525.66s
Earth-Agent (DeepSeek- 133.))
4716.99s
540
3066.76s 76s
1650.23s
Earth-Agent (Kimik2)
7903.07s
502
6306.59s
1596.47s
E.2 OUR PROPOSED STRATEGY
We addressed this issue the design of the Earth-Agent architecture. Specifically, we optimized the tool design minimize unnecessary interactions with the model. For instance, by leveraging batch calculations for Earth indices, such as the NDVI calculation, we significantly reduce the frequency of model interactions, thereby lowering the overall latency. Below an example of approach:
Batch Computing Strategy
def calculate_ndre (input_nis pit__oth, input input_rrod_path curput_paath):
raaterio, open (input_nir__pa nir_band (ir_scc.sead(1) # Read the first band (assuming aingle band rasters
nir_aro:
nir_profile nir_are.profile # Get the metadata profile
with casterio.cpen (inpt as sed_are:
red_band red_aro.sead(1) Read the first band (assuming single-band rasters
nir_band np. band, dtyge-np, (loat32)
red ap.array iced band, dtyse=np.floatt2)) Float32)
hir_band np.clip inir_band, 0, 10000)
10
red_hand (t.cclip/red_bond 0, 100C0)
11
valid_mask (nix_band 0) (nir 10000)
10000; (and >- 0) & (red_bend
12
denominator = nir_band + red_band band 1e-6
(nir_band red_band) denominator
14
Set invalid pixels to -5999
ndvi i -9999
16
rdvi_prof111 www_prortle..ggeeeeeoe
17
ndvi_proflle..ppacce
dtype-rasterio. 3loat32, NOV values floating-poont numbers nodata--9999, Set NoData value
19
compress=flav Optional: compress output file
21
Save
speatiied output path
23
os.makedira (TBMP_DTH) output_path) parent, exis:_okeTrue)
24
with rasterio, (TEMP DIR out,utt_ath,, 'w', +=sdvi_profile) as det: dst. (novi. . fasterto.
26
return Result save (TEMP_DIS output_pathh'
28 8mep tool (descripticn="*.
Rotch-calcc NDVT mal tipllee pairs NTR/Red raster filas save resulta,
30
31 Paremeters:
Sear-Infrared band input_red_pache (list Paths to Red band raster files. cutput_pathe (lis:
inpat_nir__acze [atst
33
output paths (e.g., "guastionn1/xxavw_2222----
tif") for each pair.
35
36
Returns:
37
list(str]: list result messages (one per output), as returned by calculate_ndvi",
38 .*")
csleulate_barchhndwe
40
impst_nir__arii 11st (str),
imput_red_p4ths- [atr],
42
output_peths: lis:[stz]
43
list
zetura
45
calculate_ndv/ paih_ pedha out_path)
nir_path, sed_path, out_path zip input_rod_pateee output_pachs)
17
25

---

Published as a conference paper at ICLR 2026
F SCALABILITY DISCUSSION
Understanding the performance trends the number of tool calls increases is crucial for evalu- ating the system's s behavior, particularly in terms of latency and scalability under increasing task complexity.
F.1 PERFORMANCE WITH RESPECT TO TOOL NUMBER
To address this, we conducted an ablation study examining the relationship between the number of tools used and the system's performance across three SOTA models: GPT5, DeepSeek-V3.1. and Kimik2. The following Table 12 presents the results, highlighting the high performance range for each model:
Table 12:
Performance of Earth-Agent Models with respect Tool Numbers. The high perfor- mance range is highlighted.
GPTS
DeepSeck-Ve -3..1
Kimik2
Tool Numbers
Questions
Accuracy (%)
Questions
Accuracy (%)
Questions
Accuracy (%)
0
25
8.00
25
1
3
66.67
2
0.00
1
0.00
2
8
75.00
5
40.00
3
42
80.95
23
73.91
17
76.47
29
89.66
23
5
19
6
7
15
60.00
27
70.37
7
10
90.00
15
40.00
16
$6.25
8
10
80,00
8
87.50
20
65,00
9
4
75.00
10
60.00
25.00
10
54.55
40,00 5
12
3
33.33
8
75.00
66.67 9
13
6
100.00
17
70.59
20
80.00
14
75.00 4
4
50.00
3
33.33
15
2
0.00
100.00
50.00 4
16
100.00
66.67
0.00
17
3
18
50.00
25.00
2
50.00
19
3
0.00
3
0.00
0.00
20
3
33,33
3
33.33
100.00
159
100.00
From the results, we observe distinct performance trends for each model:
For Earth-Agent driven by GPT5, high accuracy is primarily concentrated within the tool number range of to 14.
For Farth-Agent driven by DeepSeek- V3.1, the high-performance range is within the tool number range of 15.
For Earth-Agent driven by Kimik2, the high-performance range falls within the tool num- ber range of to 13.
These high performance ranges align with expectations and indicate that task complexity plays key role in system performance. Using too few tools (with the extreme case being zero tools) results low accuracy, as the agent is unable to solve the task effectively, often leading to early errors. Conversely, performance tends to degrade when too many tools employed, suggesting that the current capabilities of the base LLMs may not be sufficient to handle long chains reasoning efficiently.
26

---

Published as a conference paper at ICLR 2026
F.2 PERFORMANCE WITH RESPECT TO UNIQUE TOOL NUMBER
We also investigated the relationship between the unique number of tools used and performance trends. Below a visual representation using a bubble chart, where the size of each bubble is proportional to the number of questions.
100
The size of the circle is proportional to the number of questions
80
High Performance Range
60
Acc(%)
40
20
0
2
4
5
7
8
10 11
Unique Tool Number
Figure 8: GPT5: Performance with respect to Unique Tool Number
The size of the circle is proportional to the number of questions
80
70
High Performance Range
60
50
oc
40
30
20
10
0
-10
2
10
12
14
18
20
Unique Tool Number
Figure 9: DeepSeek-VV. V3.1: Performance with respect to Unique Tool Number
27

---

Published as a conference paper at ICLR 2026
80
The size of the circle is proportional to the number of questions
70
High Performance Range
60
50
Acc(%)
40
30
20
10
0
-10
0
1
2
3
4
5
7
8
9 10
11
12
Unique Tool Number
Figure 10: Kimik2: Performance with respect to Unique Tool Number
Based on Figures and 10 we observed the following trends in the Agent system: When driven by GPT5, high accuracy primarily observed within the range of 2 to 5 unique tool calls. For DeepSeek, the optimal accuracy is concentrated the range of 2 to 6 unique tool calls, while for Kimik2, high accuracy predominantly found in the range of to 6 unique tool calls. We also observed performance decline when the number of tool calls becomes excessive. These findings align with our expectations, highlighting the current limitations of LLMs in handling tasks that involve long chains of reasoning or excessive tool interactions.
This analysis provides valuable insights into the scalability of the system as task complexity in- creases. It also offers important directions for future agent training, including (1) optimizing the agent's s startup phase and (2) developing datasets for long-chain reasoning to enhance the agent's ability to handle multiple tool calls effectively.
G TOOL KIT LIST
The Index Toolkit offers comprehensive suite of automated functions for computing wide range of remote sensing indices directly from raster data. supports efficient batch processing and covers commonly used indices related to vegetation, water, soil, snow, and burn severity, such as NDVI, NDWI, and NDBI. A detailed list of the implemented indices is provided in Table
Table 13: List of detailed information of Index Toolkit.
Tool Name
Category
Deseription Summary
calculate batch ndvi
Index
Batch-calculate NDVI from multiple pairs of NIR/Red raster files and save results.
calculate batch ndwi
Index
Batch-calculate NDWI from multiple pairs of NIR/SWIR raster files and save results.
calculate.batch.nddi
Index
Batch-calculate NDBI from multiple pairs of SWIR/NIR raster files and save results.
28

---

Published as a conference paper at ICLR 2026
calculate.batcheev
Index
Batch-caleulate EVI from multiple sets of NIR/Red/Blue raster files and save results.
calculate_batc..nu
Index
Batch-calculate NBR from multiple pairs of NIR/SWIR raster files and save results.
calculate_batch
Index
Batch-calculate FVC from multiple pairs of NIR/Red raster files and save results.
calculate batch. wri
Index
Batch-calculate WRI from multiple sets of Green/Red/NIR/SWIIR raster files and save re- sults.
calculate batch ndti
Index Batch-calculate NDTI from multiple pairs of Red/Green raster files and save results.
malculate.batch.nu
Index Batch-calcular Fire Radiative Power (FRP) masks from multiple raster files and save re- sults.
calculate batch ndsi
Index
Calculate NDSI for multiple pairs of Green and SWIR band images.
galc_extreme_snen -noom_losss_sssss -percentage_freeeeeeeeeeee from finar. map compute.tvdl
Index
Calculate the percentage of extreme snow and ice loss areas from a binary map.
Index
Compute TVDI (Temperature Vegetation Dry- ness Index) using NDVI and LST from local raster files.
The Inversion Toolkit integrates collection of algorithms for retrieving key geophysical and envi- ronmental parameters from optical, thermal infrared, and microwave remote sensing data. It sup- ports multiple retrieval methods for parameters such as land surface temperature (LST),, land surface emissivity, and precipitable water vapor (PWV), including single-channel, multi-channel, and split- window approaches. By enabling flexible, efficient, and reproducible parameter estimation across multi-source Earth Observation data, the toolkit provides a versatile foundation for quantitative mote sensing applications. detailed list of the implemented algorithms is provided in Table [4]
Table 14: List detailed information of Inversion Toolkit.
Tool Name
Category
Deseription Summary
band ratio 1st multi channel
Inversion
Compute Precipitable Water Vapor (PWV) im- age from local MODIS surface reflectance band files using the band ratio method.
1st single channel
Inversion
Estimate Land Surface Temperature (LST) us- ing the Single-Channel method, with NDVI- based emissivity estimation from RED and NIR bands.
Inversion
Estimate Land Surface Temperature (LST) us- ing the multi-channel algorithm.
split window
Inversion
Estimate Land Surface Temperature (LST) or Precipitable Water Vapor (PWV) using the split-window algorithm.
temperature emissivity separation
Inversion
Estimate Land Surface Temperature (LST) us- ing enhanced Temperature Emissivity Sep- aration (TES) algorithm with empirical emis- sivity estimation.
29

---

Published as a conference paper at ICLR 2026
modis day night_Ist
Inversion
Estimate land surface temperature (LST) from local MODIS Day and Night brightness tem- peratures using single-channel correction method.
ttm 1st
Inversion
Estimate land surface temperature (LST) and emissivity using improved Three- Temperatre Method (TTM) from three local thermal band GeoTIFF files. Uses all three bands to form a system of equations and solves per-pixel with physical constraints.
calculate mean by ndvi
Inversion
Calculate the average Land Surface Temper- ature (LST) across multiple images where NDVI is either above or below given thresh- old.
malculate_max.u ltt._maaa__ss_ddllll...
Inversion
Calculate the maximum Land Surface Temper- ature (LST) in areas where NDVI is above below given threshold.
ATI
Inversion
Estimate Apparent Thermal Inertia (ATI) us- ing the Thermal Inertia Method. This method calculates ATI as (1 albedo) (dday) (day_temp night.temp), which serves a proxy for land surface temperature stability over diurnal cy- cles.
dual polarization differential
Inversion
Dual-Polarizatio Differential Method (DPDM) for microwave remote sensing pa- rameter inversion. Supports soil moisture and vegetation index estimation with improved data handling and flexible parameters.
dual frequency diff
Inversion
Dual Dual-frequency Differential Method (DDM) for parameter inversion using local raster data. Supports inversion of multiple parameters via empirical linear models: Soil Moisture (SM): param ( band2) + beta; Veg- etation Index (VI): param alpha*(band!
band2) beta; Leaf Area Index (LAI): param = =llpha(bandd band2) beta
multi.freg .te
Inversion
Multi- Hulti-frequency Brightness Temperature Method for parameter inversion using local raster data.
chang-single.paramiinvee inversion
Inversion
Chang algorithm for inversion of single pa- rameter using multi-frequency pola-zolr microwave brightness temperatures from local raster files.
nasa-team_see www.eeeeeeeeeeeeeeeeeeeeeeeeeeee lic..concentration
Inversion
Estimate Sea Ice Concentration using NASA Team Algorithm from local passive microwave brightness temperature GeoTIIF files.
dual.polarization.raaio
Inversion
Estimate Vegetation Water Content (VWC) Soil Moisture (SM) using Dual-Polariztionn Ratio Method (PRM) from local passive microwave brightness temperature GeoTIFF files. The polarization ratio computed as: (V H) / (V + H), where V and H are brightness temperatures of vertical and horizontal polar- izations.
30

---

Published as a conference paper at ICLR 2026
calculate water.ttbdittt -ntu
Inversion
Calculate water turbidity in NTU (Nephelo- metric Turbidity Units) from red band raster file and save the result to specified output path.
The Perception Toolkit provides comprehensive set of remote sensing perception tools, covering wide range of tasks such as scene classification, object detection, and change detection. In addition, supports threshold-based segmentation and offers series post-processing utilities for bounding box and contour refinement. Overall, the toolkit enables diverse perception tasks on RGB remote sensing imagery, including scene recognition, semantic segmentation, and spatiotemporal change detection. A detailed list of the implemented tools is provided in Table
Table 15: List of detailed information of Perception Toolkit.
Tool Name
Category
Description Summary
MSCN
Perception
MSCN is scene and land-use image classi- fier, effective for categories such as Airport, BareLand, BascballField, Beach, Bridge, Cen- ter, Church, Commercial, DenseResidential, Desert, Farmland, Forest, Industrial, Meadow, MediumResiidential Mountain, Park, Parking, Playground, Pond, Port, RailwayStation, Re- sort, River, School, SparseResidential, Square, Stadium, StorageTankss, and Viaduct.
RemoteCL.P
Perception
RemoteCLIP scene and land-use image classifier, specialized for categories such Airport, Beach, Bridge, Commercial, Desert, Farmland, FootballField, Forest, Industrial, Meadow, Mountain, Park, Parking, Pond, Port, RailwayStation, Residential, River, and Viaduct.
Strip R CNN
Perception
Strip R CNN is remote sensing object de- tection model with a strong focus on mar- itime and ship-related targets. Compared to SM3Det, it particularly specialized in de- tecting and localizing different types of ships and naval vessels. This model is highly effee- tive at detecting the following categories: L3 ship, L3 warcraft, L3 merchant ship, L3 air- craft carrier, Arleigh Burke, Container, Ticon- deroga, Perry, Tarawa, Whidbeylsland, Com- manderA, Austen, Nimitz, Sanantonio, Con- tainer, Car carrierB, Enterprise, Car carrierA, Medical
31

---

Published as a conference paper at ICLR 2026
SM3Det
Perception
SM3Det is remote sensing object detection model. Given an input image and a natu- ral language promtt specifying the target ob- ject (e.g.. "plane", "ship", "storage tank"), it detects all instances of that object and turns their bounding boxes, This model is particularly strong detecting and localizing the following categories:plane, ship, storage tank, baseball diamond, tennis court, basket- ball court, ground track field, harbor, bridge, large vehicle, small vehicle, helicopter, round- about, soccer ball field, swimming pool.
RemoteSAM
Perception
RemoteSAM is remote sensing visual grounding model. Given an input image and a text prompt describing a region of interest (e.g., "the football field located on the west- ernmost side"), it outputs the corresponding bounding box coordinates.
InstructSAM
Perception
InstructS AM is an instruction-guii counting model for remote sensing images. Given an in- put image and natural language prompt spec- ifying the target object (e.g.. "storage tank", "football field"), it detects and counts the num- ber of instances matching the description.
SAM2
Perception
Use SAM2 to segment the input image and re- turn the path of the segmented image.
ChangeOS
Perception
Use ChangeOS to detect the change between two images and return the change mask. Can also be used to segment building by provid- ing same image path in pre image path and post_image_path.
threshold segmentation
Perception
Perform threshold- based segmentation on a single-band raster image. The function reads a raster image from the specified path, converts it to a binary mask by applying a fixed thresh- old, and writes the resulting binary image to new file. Pixel values greater than the thresh- old are set to 255 (white), and values less than or equal to the threshold are set to (black).
bbox expansion
Perception
Expands bounding boxes by given radius and returns the expanded bounding boxes.
count.above.trreshol.
Perception
Count the number of pixels in an image whose values are greater than the specified threshold.
www..cooonnceed.coooooeeee
Perception
Read binary image and return the count of connected components.
bboxes2centroids
Perception
Convert bounding boxes from [x min, y min, x max, y _max] format to centroid coordinates (x, y).
centroid distance extremes
Perception
Compute pairwise distances between centroids and return both the closest and farthest pairs with their indices and distances.
calculate bbox area
Perception
Calculate the total area of list of bounding boxes in [x, y, h] format.
32

---

Published as a conference paper at ICLR 2026
The Analysis Toolkit provides suite of statistical and spatiotemporal analysis methods tailored for remote sensing and geoscience data. Its functionalities include classical time-series trend detection and decomposition techniques such as linear regression, the MaannlKendall test, Sen's 's slope estima- tion, and STI. decomposition. It also supports change-point detection and seasonal analysis based on autocorrelation. In addition, the toolkit integrates spatial statistical approaches, including hotspot direction analysis, as well as methods for spike detection in numerical sequences. A detailed list of the implemented tools is provided in Table 16
Table 16: List of detailed information of Analysis Toolkit.
Tool Name
Category
Description Summary
compute linear trend
Analysis
Computes the linear trend (slope and intercept) of time series by fitting line of the form: = a x + using the least squares method.
mann kendall test
Analysis
Perform the non-parametric Mann-Kendall trend test a univariate time series. The test evaluates whether there is a monotonic upward downward trend without requiring the data to conform to any particular distribution.
sens_slope
Analysis
Compute Sen's Slope estimator for a univari- ate time series. Sen's Slope is a robust non- parametric method for estimating the median rate change over time, often used with the Maan-Kendall test assess both trend and magnitude.
stl.dmcompose
Analysis
Apply Seasonal -rend decomposition using LOESS (STL) to univariate time series. De- composes the series into trend, seasonal, and residual components.
detect.change www.tehhhanepnnnneeee ppints
Analysis
Detect struetural change points a univariate time series using the ruptures library with the PELT algorithm. A change point marks a loca- tion where the statistical properties of the sig- nal shift (e.g.. mean or variance).
autocorrelation function
Analysis Compute the Autocorrelation Function (ACF) for univariate time series. The ACF measures the correlation of the series with its own lags, which is useful for detecting seasonality, per- sistence, and lag dependence.
seasontseas accf ssar
Analysis
Detect the dominant seasonality (period) in a univariate time series using the Autocorrela- tion Function (ACF). The method searches for significant peaks in the ACF beyond lag=1 to identify repeating cycles.
Analysis
Compute the Getis-Ord Gi* statistic for local spatial autocorrelation method identifies statistically significant spa- tial clusters of high (hot spots) low (cold spots) values using a user-specified spatial weight kernel.
raster image. This
33

---

Published as a conference paper at ICLR 2026
Analysis
Analyze the main directional concentration of hotspots in binary hotspot map. The function counts the number of hotspot pixels (value=1) each cardinal direction relative to the map center, and returns the dominant direction.
count spikes from. values
Analysis Count the number of upward spikes in a se- quence of numerical values. A spike is defined as a positive difference between consecutive valid values greater than the given threshold.
The Statisties Toolkit offers a comprehensive set of functions for descriptive statisties, image-based statistical analysis, and geospatial data processing. Its capabilities cover the calculation of classical statistical measures such as mean, variance, and skewness, as well as the extraction of statistical information from imagery and intersection- threshold analysis. In addition, the toolkit provides fundamental arithmetic operations, temperature unit conversions, and image differencing functions. also supports essential preprocessing tasks, including radiometric correction and cloud masking. Overall, the toolkit enables flexible and efficient extraction and analysis of statistical features from geoscience and remote sensing data. A detailed list of the implemented tools provided in Table
Table 17: List of detailed information of Statistics Toolkit.
Tool Name
Category
Description Summary
coefficient of variation
Statistics
Compute the Coefficient of Variation (CV) for dataset. The CV is defined as the ratio of the standard deviation to the mean and is com- monly used as normalized measure of disper- sion.
skewness
Statistics
Compute the skewness of a dataset, which measures the asymmetry of the probability dis- tribution.
kurtosis
Statistics
Compute the kurtosis of dataset, which mea- sures Tailednessöf distribution.
cale batch image mean
Statistics
Compute mean value an batch of images.
calc.batch.iman
Statistics
Compute the standard deviation (spread of pixel values) for batch of images.
calc.batch. bagc. image_median median
Statistics
Compute the median pixel value for a batch of images.
calc_batch wwwp:battchhmmm....ee __in
Statistics
Compute the minimum pixel value for batch of images.
calc_ batch baaeee Ima max
Statistics
Compute the maximum pixel value for a batch images.
calc.batch. skewness
Statistics
Compute the skewness of pixel value distribu- tions for batch of images. Skewness quanti- fies the asymmetry of the distribution:1.. Posi- tive skew longer right tail; 2. Negative skew + lonneer left tail; 3. Zero skew - symmetric distribution.
calc_batch image kurtosis kurtoss
Statistics
Compute the kurtosis of pixel value distribu- tions for batch of images. Kurtosis measures the failednessöf the distribution relative to a normal distribution.
34

---

Published as a conference paper at ICLR 2026
calc.batch cale.batchimmge_ssm
Statistics
Compute the sum of pixel values for a batch of images.
baaatch_imae_hhaee -percentage
Statistics
Compute the hotspot percentage (fraction of pixels above threshold) for batch of images.
calc_batch, malc_batch_imag_hhmeee
Statisties
Create binary hotspot maps for a batch of im- ages, where pixels below specified threshold are set to 1 (hotspot) and others set O. The output is saved as Geo files, preserving georeference metadata from the input images.
difference
Statistics
Compute the absolute difference between two numbers.
division
Statistics
Perform division between two numbers.
percentage change
Statistics
Calculate the percentage change between two numbers, useful for comparing relative growth or decline.
kelvin to celsius
Statistics
Convert temperature from Kelvin to Celsius.
celsius_to kelvin
Statistics
Convert temperature from Celsius to Kelvin.
fox__vauu__an _indxx
Statistics
Find the maximum value in list and return both the maximum value and its index.
valuv
Statistics
Find the minimum value in a list and return both the minimum value and its index.
ceil number
Statistics
Return the ceiling (rounded up integer) of a given number.
multiply
Statistics
Multiply two numbers and return their prod- uct.
get_list www.listtojeet_.aaiineeess
Statistics
Retrieve elements from list using list or tu- ple of indices.
mean
Statistics
Compute the arithmetic mean (average) of a dataset.
calculate calculate.treeshold..o ratio
Statistics
Calculate the average percentage of pixels above given threshold for one or more im- ages and a specified band.
cale batch fire pixels
Statistics
Compute the number of fire pixels (FRP ? threshold) for batch of images.
srcate_fire_iccrasee mmp
Statistics
Create a binary map highlighting areas where fire increase exceeds specified threshold.
Seintff_firee.prnee
Statistics
Identify fire-prone areas from hotspot map based on a given percentile threshold.
get pet.perccenile_vae value from image
Statistics
Calculate the N-th percentile value pixel values in raster image, and return it a native Python type matching the image's data type.
image_division.ne
Statistics
Calculate the mean of pixel-wise division be- tween two images or between two bands of the same image.
calculate calculate_intereettin intersectionn sew.recctiioneereenttgeee
Statistics
Calculate the percentage of pixels that simul- taneously satisfy threshold conditions in two raster images.
35

---

Published as a conference paper at ICLR 2026
calc.batch ima.eemmannnaaaeeeeeeemmmm mean mean
Statistics
Compute the average of mean pixel values across a batch of images.
wew_battchimmage_eee aaaaaagemmea_maaa
Statistics
Compute the mean pixel values of a batch of images and return the maximum mean.
calc_batch. cale_batch_imag__mmee maagge__eaa_maa.._m min
Statisties
Compute the batch-wise statistics across mul- tiple images, including: Mcan of mean values, Maximum of maximum values, Minimum minimum values.
calc_batch. imagge image_mean mean threshold
Statistics
Calculate the percentage count of images whose mean pixel values (in specified band) are above below a given threshold.
calculate_ multi band_threshhlddd _ratio
Statistics
Calculate the percentage of pixels that simul- taneously satisfy multiple band threshold con- ditions.
cout..pieess.attffyyn..ccoooioo
Statistics
Count the number of pixels that simultane- ously satisfy multiple band threshold condi- tions.
count muuntiimaassexcceednnn chheeee ratio
Statistics
Count how many images have percentage of pixels above or below a threshold that exceeds specified ratio.
average ratio exceeding threshold
Statistics
Calculate the average percentage of pixels ceeding a value threshold, considering only images where the ratio greater than speci- fied ratio threshold.
www._inaggs_xxeeddigggg .multiplier
Statistics
Count how many images have a mean pixel value above or below multiple the over- all mean pixel value across all images.
calculate. band mean_ by conditionn
Statistics
Calculate the mean value of a target band over pixels where a condition band satisfies a threshold.
cale threshold value mean
Statistics
Calculate the mean value of corresponding raster pixels in path2 where the raster values in pathl exceed the given threshold.
calculate difference
Statistics
Calculate difference between two tif files (im- age_b image_a) __) and save result.
subtract
Statistics
Subtract two images and save result.
calculate_area
Statistics
This function calculates the area of non-zero pixels in the input image and returns the result.
grayscale.tt to.ooomaa
Statistics
Apply colormap to grayscale image and save as a color image.
get filelist
Statistics
Returns list of files in the specified directory.
rudiometric.commction..
Statistics
Apply Landsat surface reflectance (SR_B*) radiometric correction.
ayply_cloud_mast
Statistics
Apply cloud shadow mask to single Landsat 8 surface reflectance band using QA PIXEL band.
36

---

Published as a conference paper at ICLR 2026
G.1 TOOL PROMPT
To better illustrate the functionality of the toolkits, we provide a representative example. Specif- ically, focus the 1st_multi_channe! tool, which estimates LST using multi-channel algorithm. This method leverages multiple thermal infrared bands from remote sensing imagery and applies empirical formulas to derive accurate LST values. The corresponding implementation is provided below:
Tool Example
Ancp.tool (descripticn-"'
katimate Land Surface Temperature (LST) using the mulzi-channel sigorithm.
Requires local input files:
Two thermal infrared bands (e.g., Band 31 and Band 32) aa GeOTIFF flles.
Parametera:
band31_peth (str): Path to local GeoTIFE file for thermal band ("11 Ymu m)) band32_path (str): Path local Georiry file for thermel band 32 ("12 Yau output_path (str): Relative path for the output raster fila, e.g. "question17/
15t_2022-01-16.000
10
Returns:
12
str: Local file path of the exported LST image.
13 (*))
def lst_multi_channe (bans3l_path:: panh?2_path; output_path: ste)
15
Description:
17
Estimate Land Surface Temperature (IST) using the milti channel algerithm.
this method combines thermal infrared bands
atmosphezic effects.
19
20
Porameters:
band31_pan (str): Path GestIFs file for thermal band ("11 Amu hand32_path (atr): Path to Geo"TPR rile for thermall band 32 (712 Amu m)
22
output_path Relative path for output GeOTIFF
24
Returns:
26
str: Pull path to the saved LST GeSTIFF
27
import 05, rasterio
29
import numpy as np
31
with rasterio. open (band3i_path) sre3l:
sre31. resl..redd/p (1)d(1)).)). castyy float32) profile sre3l.profile
34
rasterio, (band32_pant path) sre32:
36
band32 3rc32.read/33 .astype inp. Cloat32)
38
1.022
0.47
40
0.43
band31 (band3: band321
43
profile. updaze wiw.eeeeeeeeettsss.............. count"l, comprese=(lzs")
45
os.makedirg ra (TRMP_DTH output_path) parent, exis:_okeTrue)
47
48
with rasterio, open (TPMP__T DTR output_pathh 'u', eeprofile) as dat: (1st. (.a... [laa32),,
50
return Result saved at [TSMP_DIR output_path!'
37

---

Published as a conference paper at ICLR 2026
H EARTH-AGENT WITH DIFFERENT LLM BACKBONES
Spectrum: [Climate Analysis]
reflectance data over City, bosed NDVI and thermal band 10 using the ningle-chonell method, then calculate the average LST surmer and autunn, and determine the mean diffarence to arsess secsonal temperature variation between these two periods.
Using
10.89K
C/ 12.42K
b. 14.75K
BT10 x 21
B4 21
S GPT-5
"name" fillie
"luput": "di "atr_pall'' Thonhmnnkllts/ppooimm111
Step
"output": (*New York, 2022-01--110 BTIG.I". NNww 2022-12-28 65.in")
"name" single chundi
"input": putl" bha.hhurk/daaaaggeeeeono Suaatteeee Yoek 2022-06-03 BTI0.Sf....
"benchmarkk cbes conee /New York 2002-11-36, WTIINF)) "hir puti": dettmmmmeccttteeeiiiinn NNew
Step 2-22
York, 2022-06-03 5aaif "knnlmmriblttappeeeeii New York 301241 1-26. 65.if"). "ouipet pull?'
l'questind 2022 2022-06-05.ff "questiond lst 2022 11 11-26tif2) "red pati": bennhhmrrdddttiqqueeeee dnteiqittioon N/New
York 2002-05-00
3012-1 b4
"ostpur":
ousiquestion 11-26.1f"..... "Result out outiquestion 03.df")
Step 25
"name" : diffecence
"Input" (*x": 42966477666611166 6,"", 845990078054)
"output": "12-416966014522"
Final
"final answer": "C"
Step-by-Step:
1
TAO: 1.0
2
TIO: 1.0
TEM: 0.4
4 Param.: 0.0
End-to-End:
Efficiency: 5,0000
Acc: 100%
GT
Tool 1
Tool
Tool
x2
Tool
Tool get_filelist
Tool 2: lst lst__ingle_chamee
Tool 3: pal__batch_image_menn meee__nimmn__meenmem
Tool difference
Tool 5: calculate_batch_ndvy
Tool 6: nolculate_meon_lst_by....
Agentt
GPT-
Tool 1
Tool
x21
Tool
x2
Tool
K Kimi-k2
Teal
Tool
Tool
FAIL
Deepseek V3,1
Tool 1
Tool 2
x21
Tool
x2 FAIL
Figure 11: Example of Climate Analysis with Spectrum Data under the Auto-Planning Regime.
38

---

Published as a conference paper at ICLR 2026
Spectrum: (Climate Analysis]
Instrcttio Following
Quesstion: Based on Landsat 8 thermal bend 10 and reflectance bands (Red and NIR) data over New York City in 2022, first list the input TIFF Files, then colculate NDVI and estimate lond surface temperoture (LS using the single chonnel NDV VI based methad, Clossify the images into four seasons (spring, summer, outumn, winter) based acquisition dates, compute average
autumn, and finally calculate the difference between these two seasons to analyze seasenal temperature variatior.
8. 10.89K
12.42K
6. 14.75K
BT10 B4 21 21
GPT-5
"name": get
"Input": [ poth"_pth"" "aunhhearrddduuuuqueiins
Step
"output": (NNew York, 2022-01-10 Bf...."..... N'Nww York 2022-12-18 b5.60**)
"name": lst, single channel
"ispat": path": math":("bemchmie ("beccmarkdnnaeeeeeeioin
York 2022-06-03
Step 2-22
"benchmark daaaqucctoood11 York 2022-1 11-26 BT10.f") "nir path": (ommccaariiddtta/queeeaa N/ew
York 2022-06--08 baaif".. "Benchmarkiee detimmakidaiiiiiiiioiii (Nww York 2022- 1-26, bSaif", "ouiput patt""
("question5 2022-06-03.if "question fhs 1-26.6"1) "red. ("benchmark feencmmrkkcazrqquesiiinee /New
Yodk, 2022 03 bl.if" daacbmakkdnn-quusssonn..... York, 2022-11-26 bi.ie")
"output":
cutiquestionS//1e 2002-11-26tif...
ousiquestion 2023-06-03.aff
Step 24
"name": image monn moan
"input": (fflee list: [outquestionS 1/sst 2002-06 00.00666066......... "omsiqaesiionSS/ 2022-1 1126..if))
output": ("298.81 81599977801 (2000044....6629000000 (296003766661117
Final
"tinal answer": 'C*
Step-by-Step:
1
TAO: 0.75 2
TIO: 0.8
TEM: 0.4
4 Param 0.0
End-to-End:
Efficiency: 4.8000
Acc: 100%
Tool
Tool
+>
Tool 3
x2
Tool 4
->
Tool 1: get_filelist
Tool 2: ist_single__hhrr channel
Tool difference
Tool 5: aatullate_batch_ndvii
Agent
Tool
Tool
x21 +>
x2 + Tool 3
GPT-5
K Kimi-K2
Tool
Tool
Tool
Tool
c
X
Deepseek V3.1
Tool
Tool
Tool 3
FAIL
Figure 12: Example of Climate Analysis with Spectrum Data under the Instruction- -oaietiiinnooooinnnggg Regime.
39

---

Published as a conference paper at ICLR 2026
Autonomouu
Spectrum: [Disaster Judgement]
Question: temperature vegetation indices (NDVI L.ST) the Centrol Valley California, U.S., from May to October 2021, calculate the number of times when more than 40% of the area exhibited drought index value exceeding 0.7, indicating widespread extreme drought.
The proportion of pixels with TVDI values exceeding 0.7 surpassed 40% occurred 10 times
B. The propertion of pixels with TVDI values exceeding 0.7 surpassed 40% occurred 8 times
The proportion pixels with TVDI volues exceeding 0.7 surpassed 40% occurred times The proportion of pixels with TVDI values exceeding 0.7 surpassed 40% occurred times
NDVI 11
LST 11
GPT-5
"name"" get filelist
Step 1
"Imput": [ "'drr path"; "benchmaukkdaatqqqesiiiss
"output": ("Central Valley_2021-05..1 2021-05-09 LST.tif" /CCeraal Valley 2021-00-09 NDVLiif. CCental Valley_ 110yy2221-400 1-05- 25 L... "Central Valley 2021-48-25 NDVL.if" "..,"Ceetall Valley. 2021-10-16
NDVLtiP]
"name"; compute jidi
"input": {
"ndvi "nthi_path"" ("Central Valley 2021-05-09 NDVLiif ..... "Central Valley 2021-10-16 NDVL.f"")
Step 2-12
"Ist path": ("Central Valley 2021-00-09 LST. "Central Valley_2021-455 2021-05-09 LST.sf]
"oupput "output_path", path": S'questionSivvi 2021 5021-005-09-8
tif", ...., Qupesiiosivvi__222--0-000 2021-005409.if7)
"output": ( ['Result saved at outiquestionS/viii 2021-00--9..if
"Result saved ouriquestionSSvvii 2021-10-16.t1"]
"uame": count images excceding, threshold "I
"image patis" ("outiuccsionnttvvi
..... "outiqucctionS/tv. 2021-10-16.ipp
Step 13
"mode". "ahove". ".
"tatio :hreshold":440
"value chreshold": 0.7.
"outpat":"""" '4"
Final
"Ginal answer": "C*
Step-by-Step: TAO: 1.0
TIO: 1.0
TEM: 0.67
Param.: 0.33
End-to-End: Efficiency: 4.33
Acc: 100%
GT
Tool
Tool
Tool
Tool get_filelist
Tool 3:
Tool compute_tvdl
count biinges_eexeeddngggtreeeeieeeee
Agent
Tool 4: aalulaat__trresold_rco
Tool
Tool
x11
Tool
C
GPT-5
K Kimi-K2
Tool
Tool 2
x11 >
Tool
C
Deepseek V3.1
Tool
Tool
x11 >>
Tool 4
12 C
Figure 13: Example of Disaster Judgement with Spectrum Data under the Auto-Planning
Regime.
40

---

Published as a conference paper at ICLR 2026
Spectrum: [Disaster Judgement]
Inatruction Pollzwing
Question: Based temperature ond vegetation indices (NDVI and LST) the Centrol Valley of California, U.S. from May To October 2021, first list the input TIFF files, then calculate the Temperature Seerr---eeeeeee Index (TVDI)
date, compute proportion pixels with TVDI values exceeding Vegetation Dryness finally count the number of times this proportion surpassed 40%, indicating widespread extreme drought.
image,
The preportion of pixels with TVDI values exceeding 0.7 surpassed 40% occurred 10 times
B. The propertion of pixels with TVDI values exceeding 0.7 surpassed 40% occurred 8 times
The proportion pixels with TVDI volues exceeding 0.7 surpassed 40% occurred times The proportion of pixels with TVDI values exceeding 0.7 surpassed 40% occurred times
NDVI 11
LST 11
6PT-5
"name"" get filelist
Step 1
"Imput": [ ('ddr__paut path"; "fenchmurrkaatssquesiiosss
"output": ("Central Valley_2021-0555 2021-00-09 LST.tif, "Cenmal Valley 201_p__222110-0___11 NDV..iV'. Valley_ 2021-10-
16 L.T.iff p..," "Central Valley 2021-10-16. NDVLie)
"name"; compute Ivdi
"input": {
"ndvi "ntvi_path": ("Central Valley 2021-05-09 NDVLiif .... "Central Valley 2021-10-16 NDVL.E"")
Step 2-12
"Ist path": ("Central Valley 2022-00--0 LST. "Central Valley, Valley_2021-10-U 3021-10-16
LST.sff]
"output_pane path": (iquestionSivdi 2021 (021100-09945 tif",... ...., "questionS/vvi 201.://0iii22211101666111
"output": ["Result saved at outiquestionnStvii 2021 -05-49,.ii" "Reault saved outiquestionn//vvi 2021-10-16..if)
"uame": count images excceding, threshold "I
"image patis" ("outiuccsionntvvvi
..... "outiqucttion//tv. 2021-10-16.ipP
Step 13
"mode". "shove". ".
"tatio :hreshold":440 40.
"value chreshold": 0.7.
"outpat":"4"" '4"
Final
"Ginal answer": "C*
Step-by-Step: TAO: 1.0
TIO: 1.0
TEM: 0.67
Param.: 0.33
End-to-End: Efficiency: 4.33
Acc: 100%
GT
Tool
Tool
Tool
>> C
Tool get_filelist
Tool 3:
Tool 2: compute_tvdl
count_ingges_exee wewrseeeeeeennnteeeeeeieeeee
Agent
Tool 4: paalltte__threshhd_rrttti
Tool
Tool 2
x11
Tool
C
GPT-5
K
Tool
Tool
x11
Tool 4
Tool 3
+ C
Kimi-K2
Deepseek V3.1
Tool
Tool 2
x11
Tool 4
x 12 +> C
Figure 14: Example Disaster Judgement with Spectrum Data under the Instruction-
Following Regime.
41

---

Published as a conference paper at ICLR 2026
Spectrum: [Temperature Monitoring)
Question: Using Landaar TOA date (Band (O) in Death Valley National Park duringg the 2021 heotwave scason Line- September) calculate how many days showed over 50% of the srea with land aurface temgerature turex above
315
A-5 days
10 daya
12 days
D. 18 doys
BT10 x 18 B4 x 18
B5 18
K'Kimi-K2
"axane":s"
"Iupurt: puth"" "teschnakksrrrepettri"""
Step
"sntpat": D"Dth Valkey 2021-05-45 Hrione"....."eee. TDett Valee 2011-09-26 biter)
"some" Scceet itzagee cxcoeding throhhold unto
"Iupar" II TTrrige pullo"; handurark qpeatttin (allay_3000006-000 BiTeaue.....
Ldittiquaaioo Dwatt Valley 145-36 31100)
Step 2-
Eirebeld" "valer taraball"-077
quo)
"oulpar":" 14"
".t..iiggc..car clarnd
"inpat" (e wtt.e Vally,
Step 6-19
ITT0 -baachoortdddm queriial---ee Desth Valley 2021 22 BTT6-UF) "red pettf:
I'oenchunnke detthannndddiiiieteeeeeee Desth Valee 06-05 bi.td" 'benctunud (pasiippeett Valley 2021-294 tift. "ox putl: (bonchmurkddtte Det Valley 3021-06.
"'ee beoiurandddoaapee Deats Valley 2021, 06 k5t) "uotpor, peth?, ("quetion7] 2001 06 "questaa7 for 2321 26.F1
"ostpurt: [Bond smed at ontiuuskor 2001 66
www.geiggeeee....oo pppp_eewedeg_.trsbii.ooo "laput": P'image ["sotiquciisttee "varee 315,
Shep 20
"ostyu!": '5"
Final
"fiaal anowcet: 'W"
Step-by-Step:
TAO: 1.0 TIO: 1.0
TEM: 0.67
Param.: 0.33
End-to-End:
Efficiency: 6.67
Acc: 100%
Tool
Tool
Tool
filelfll lst_single_chanee
Tool calc_botch_immee Baancch_ingg_eeeeee wwwphhotttoppeeeeeeeeee
Tool b:calcalat__bbatheee
Tool per_llis__ojectt.v windeees__ indexes
3: count, images exceeding, threshold ratie
calculate, calosdene_rhreshhldd.eo
Agent
Teol
Tool
Teol
15-3
Tool
Tool
Taol
<h-> Foll
K -2
|->
Tool
14->
Tool
Tool
Tool
Deepseek V3.1
Teol
Tool
Teol
Fail
Figure 15: Example of Temperature Monitoring with Spectrum Data under the Auto- Auto-Planning Regime.
42

---

Published as a conference paper at ICLR 2026
Spectrum: [Temperature Monitoring)
Question: Using
(Band Valley
#csson September),
inpur
colculate astinate
lemperoture (LSTT) using single-chonee method. ecch maage comps proport of placls LST volues exceeding 315 and finall count the number days when more than 50% the area sur possed this temperature threaheld..
days
days
12 days
D. 18 days
BT10 18
B4 18 B5 18
H Kimi-K2
"came": get Gliat
Step 1
"ingur": l'de path" Panchraakklaakqquellls?
"eaqut": ("Dethh Valley_2001 prrinne Valley_
"nmne": cakoatii Jeich
"inprr: "isput poths":| "renstemolllle al Vallay
fif "sencmmetoooo I Valley 19-26, (5 "mpprt red pothe"
("haschaant damoraato (ideet Valley 2011-00-12200
gromen Lidees Vallay,
Step
20 be tir). "ooget patta" ('gustinaialy Tindel 2001-0000000000 ur... "guesion7iichi Bariidiiii110 2021-89-20.11
"ouiput II RReel ondigaatinn 201--6-66........
outicoettion Tradii 29-26
"nsme": single
Step 3-17
"Inpal" ('bi "bendlnnek wowe 2021-06-06 BTTR pull::
"beachnnkk iettrn//iiitttttttteaaoao Valay 2021-05-05 putt bbaatlaadd ditegpattoon
Valley 2021-05-05
fouppu pudi" "quetioo71a 2021 301-06-06.1P1
[Rssel aediusstinn7/ss 2021-06-06un)
"ssme"scouat eccooding ttresold aste
"iapet"
wttp://ooooo..00.........aa N011006000...... "oddiusstiaa 2a 202100000000111000000011
"volo Bhrehudd 315,
Step 99
Eurskood' "along" bose"
"oxtjue": vgo
Final
"Binl onsomes""
Step-by-Step:
TAO: 1.0 TIO 1.0(3) TEM: 0.67
Param.: 0.33
End-to-End:
Efficiency: 6,3333
Acc: 100%
Tool
Tool 2
Tool
Tool 1: get filelistt
Tool 4: cal: batch, inage__htsspot____ perceettteeeeeeegge
Tool écelecdate bwt.hh.cchhhhcccc
Tool 2: "st_Single_chhle cchanel
get object
Tool count. Inages, exceediing whreshald_
Tool 8: calculate, threshald_raric
Agent
6PT-5
Teol
Teol
Toal 6
Tool 16
Tool
K
Kini KZ
Tasl
Tool
Teal
Deepseek V3.1
Tool
Figure 16: Example of Temperature Monitoring with Spectrum Data under the Instruction- Following Regime.
43

---

Published as a conference paper at ICLR 2026
Spectrum: [Urban Management]
Autonomous Planning
QuQustioo:: Using TES-derived land surface temperature from ASTER Bands 10-14 on June 15, 2022, over the specified Los Angeles metropolitan area, calculate the percentage urban pixels exhibited 300 combined with emissivity 0.96, indicating urban heat island intensity during early summer.
A. 15.34%
B. 28.67%
C. 37.93%
41.13%
GPT-5
"name" filelist
"input": { "dir haaaa//bbnnmmmrrssuoorotto cataiquestion11))
Step 1
"output": [' ("lst"" ("lst_ttsss.ts"t tes resui..if")
calculate multi band threshold
Step 2
"Inpar":: "image Buclss_pass"__ "benchmark: isppported data/question2 l/ ressesiittt"',
*band conditions" [[0.300, "above"] [.[1 0.99,, 11.00.6,"bcloow "belool
"output": 41111331533356644
Final
"final "C"
Step-by-Step:
TAO: 0.67 TIO: 0.33
TEM: 0.33
Param.: 0.0
End-to-End:
Efficiency: 0.6667
Acc: 100%
GT
Tool 1
Tool 4
Tool 3
D
Tool 1: get_fleelist
Tool satiairree_emmmiiitsseeeeeeee
Tool
Tool 2: pale_batch_image_ste
Tool 5: calc_batch_imag__n
Tool 3: valculate_multi_baae thteeshhhd__rratiii
Tool calc_batch_imape htt___ig_e_eee_o_____re
Agent
GPT-5
>> Tool
x3 >>
D
K
Kimi- K2
Tool
Deepseek V3.1
Tool
Tool
Tool
x3 D
Figure 17: Example of Urban Management with Spectrum Data under the Auto-Plamning Regime.
44

---

Published as a conference paper at ICLR 2026
Instruction Following
Spectrum: [Urban Management)
Question: Using TES-derived land surface temperature from ASTER Bands 10-14 on June 15, 2022, over the Los Angeles metropolitan area, first list the input TIFF files, then apply the Temperature
Emissivity Separation (TES) algorithm to estimate land surface temperature. Finally, colculate the percentage of urbon pixels where LST exceeds 300 and emissivity is below 0.96, indicating the intensity of the urban heat island during early summer.
A. 15.34%
28.67%
37.93%
D. 41.13%
6PT-5
get Fileiist
"input":: "dir path" hhnchemrkkenukkkrrrrreedddd 2u
Step
"output": ("answer_222 ("lmssor_ll.y"""""MM \llss
los result.tif")
"name": calculate, mulri band threshold, natio
Step 2
"input": "image path!: Sennchmarr/supporiic dataaqucstion2211 1/stt tes resultuif"
"band conditions": [[0,300, "above"]][ (1.00%, "bolow
"output":
Final
"final answer": "C*
Step-by-Step:
TAO: 0.67 2
TIO: 0.33
3
TEM: 0.33
Param.: 0.0
End-to-End:
Efficiency: 0.6667
Acc: 100%
GT
Tool
Tool
D
Tool 1: get_filelist
Tool
Tool sccalc_batch_mmgg__seeweee
Tool 2: hall__bach_image..
Tool 5: calc_batch_image,
Tool 3: hatpeee_itii_bind_teeeeeteee
Tool 8:
Agent
GPT-5
Tool
Tool
K
Kiml-K2
Tool 1
Tool
Deepseek V3.1
Toal 1
x3 +>
Tool 3
+> D
Figure 18: Example of Urban Management with Spectrum Data under the Instruction- Following Regime.
45

---

Published as a conference paper at ICLR 2026
Spectrum: 2g (Vegetation Monitoring!
Question: output from ASTER thermal near Tamanrasset, Algeria, calculate the proportion of the area where emissivity variation (Ac) exceeds 0.05, indicating possible land cover heterogeneity.
region
A. 7.63%
B, 11.13%
21.85%
b. 29.39%
Deepseek
V3.1
"name": get filliit
Step 1
"Inpat": "di
("di_path"" path": "boschaarridddaaaqqusioiii
"otputt: [-2020 BT 10.i0" ....
.... 12020.220000 03, [4.607)
"name": temperature emissivity sepantion
"input": band patts" hww.:e/www..iiauussssseeeeeeeeee
10.if
14.6f").
Step 2
'repeesentatve band indoc": 3,
puth quesstoon23tes oulputaii"
outpot": ("Resttt saved ontipeestion2/ees. ourpur.ii""
name" : calcolate area
Step 10
Inpart: "input image__pattttt 1outiqusstoo22/tss outpun.tif",
ostpot" "2381880.0"
Final
answer":
Step-by-Step:
TAO: 1.0
TIO: 1.0
TEM: 0.67
Param.: 0.33
End-to-End:
Efficiency:3.3333
Acc:
GT
Tool
Tool
Tool
Tool get_filelist Tool calc_batch_imas hongge_ottpptt____ wtwp:hoto___pcecetoeeeeooooo
Tool 3: threshold_ segmmnttissgmmnnniiionn
Tool 4: galculate_threshld_rroo Tool 5: identify_fire, ttt_f_ffee_o___e_e_e
Tool 10: calc_batch_imoge__s soct_jjmeee_sum Tool 11: cal__bateh__lngg__httppp
7: temperature_emissiee emissiivity_seporationn Tool
hnww_llss_percenntgee
Tool count_pixels pwtpplls_satiifyiigg.coco.
Tool 13: htapeee_utti__andd_hhhhhlldddedd
Tool get_perrenttil__woe hitprcattiiialllee_rr__ieeeeeee foruuu__ff__meeeeee
Agent
GPT-5
Tool
Tool 13
Tool
Tool
K Kimi-K2
>> Fail
T
Tool
Deepseek v3.1
Tool 4
x2
Tool
Tool 10
Tool 9
Figure 19: Example of Vegetation Monitoring with Spectrum Data under the Auto-Planning Regime.
46

---

Published as a conference paper at ICLR 2026
Spectrum: Xi |Vegetation Monitoring)
Based TES output from ASTER thermal bands on Morch 24, 2020, Sahora Desert region near Tomonnasset, Algeria, first list the input TIFF files, then apply the Tempencture and Emissivity Seporation (TES) algorithm proportion of the area where emissivity variation (Ax) exceeds 0,05, indicating potential land cover heterogeneity.
Question:
land temperature emissivity, Finally, colculate
11.13%
C. 21.85%
29.39%
Deepseek
V3.1
"name": get
"inpor": { "dir path": lomchhmnkkddaa//qeeeeeeet
Step
"oupptt" [-2020 BT. 10 ....... 12020 BT [4.01"]
"name": tempentuue emisivity. apparation
"inpur": band pathe": ("benchmarkole 2000
prnncbmarkdstt//qqeeeet "representative band index": 3,
24 14
Step 2
"output path" "questio255 outpotzil"
outpot" ("Result soved at outopastiion22es, output HPP
name" calosiate aror
Inputt's "Inper aaagg_ pati": "outipustion237es outgurti"
Step 9
output": "2351880.0"
Final
auswer":
Step-by-Step:
TAO: 1.0
2
TIO: 1.0
3
TEM: 0.67
Param 0.33
End-to-End:
Efficiency: 3.3333
GT
Tool
Tool
Tool 4
Tool 1: get_filelist 2: Baachh_mma__hhtspt__poo
phreshold_segmentte
Tool Siculltt_thrrehold_rrtt
Tool 5: identify_flire_prme__rm
Tool temperature_emiinn hatrttuee_emmiiittteeeettttteeee
Tool 11: cal__botcch_mmgg__hateee imwggeeettttttt_ttiii
Tool 10: calc_batch_image__ee _mmage_mummmm
wnww_less_percentog__Pe
count_pixels_ jiixelssatisffing_commm
Tool 14: Bil_percetill_vvlleefrrree
Agent
Tool
Tool 7
Tool 4
x2
Tool 13
Tool 4
Tool 12
x2
Tool
x5
GPT-5
K Kimi-K2
Tool 1
Tool
Tool
Tool 14
Tool 12
K2
Tool 13
Tool
Tool 11
Tool 4
Tool
Deepseek V3.1
X
Tool 8
Tool 3
Tool 6
Tool 10
Tool 9
Figure 20: Example of Vegetation Monitoring with Spectrum Data under the Instruction- Following Regime.
47

---

Published as a conference paper at ICLR 2026
[Pollution Regulation]
Question: Based on our_refl_be pur_refi_bOe betermine whether ocean Turbidity in the Arctic Oceon increased or decreased over this period, and report the magnitude of the change.
the Arctic Ocean, colculate
A. The overage ND TI decreased slightly from about -00472 in July 2012 to about -0,0502 July 2022,
meaning turbidity weakened and the magnitude change roughly 0.0030.
The average NDTI increased from about -O 000550622 in July 2012 To about -0.047156 July 2022, so turbidity increased (less negative ND T1) with magnitude about 0.003006.
The average
increased from about -0.0472 to -0,0502, indicating much clearer water, with magnitude
near 0.00$0.
Average volues roughly -0.0502 -0.0472 show detectable change turbidity and 2022.
Average TI stoyed about -0,05 for both yeors,
change.
B1
B4 x 72
6PT-5
"mame": get fieeiit
"input": Penschmrrkdaa//uueeiiniss
Step
"outpat": ("Arri Occan refi, 200-30022--00100 sif" Arrctic Ocean reff 194 2022-07-29..ff1
"mame": calculato basch, ndti
"input": "input palks" ("hecchmrrrddmtiippcitooon
2012--7--
Ol ..... /wwechmarr/aaat/eeeeeeeoe /ttp///eeii/m//////a/a Ocean sur refl 2022-07-29.iiP) "input, green, paths":
("benchmark duraaqqestion 82/Arctic Oecam sur b01 201 2-17-00-01 lif",...
Step 2-3
castemmuurrrnn/uueeiitteio /AArcii Ocean retl, b01 2022 29.nf") "output, paths": ("question 82/ndi, 2012-
07-01.af" Iqucstion122luti 2022-07-29.06")
"output": ("Result quesiiotio 2012 2012 0012207-01.000 u........... ("Rssll ourigucation 2022-07-29.x077
"name":
"Iapur": (--13.09997301180000000
Step 7
"output": 113.00551177614477777
Final
Tiaal answer": "H"
Step-by-Step:
TAO: 0.5
2
TIO: 0.03
TEM: 0.03 4 Param.: 0.03
End-to-End:
Efficiency:0.2258
Acc: 100%
GT
Tool
Tool
Tool
Tool 1: get_filelist
Tool calculate_ndti
Tool 3: cal__batch_imag__men
Tool 4: difference
Tool bidullate_batch_ndtt
Tool 7: calc__al_tteeeeeeeeeeeeeeeeeeeee lmagg__eee__mmaannm
Agent
GPT-8
Tool 1
Tool 5
x2
Tool 7
x2 +
K Kimi-K2
Tool
Tool
x2
Tool
x2
X
Deepseek V3.1
Tool
Tool
x2
Tool
X3
Tool 4
> A
X
Figure 21: Example of Pollution Regulation with Products Data under the Auto-Planning Regime.
48

---

Published as a conference paper at ICLR 2026
Products: [Pollution Regulation]
Insinaciion Following
Question: Based the refi_ 101 and aar_refl__ee b04 data of the Arctic Ocean, calculate the NDTE and compute the overage ND values for July 2012 and Jull 2022, Determine whether occan furbidiity in the Arctic Ocean increased decreased over this period, and report the magnitude of the change.
A. The overage NDTI decreased slightly from about -0,0472 in July 2012 to obout -0,0502 in July 2022,
meaning turbidity weakened the magnitude change roughly 0.0030.
B The averoge ND TI increased from about 0.050162 in July 2012 to about -0 -0.047156 in July 2022, so megnitude about 0.003006.
turbidity
(less negative Nb
C. The overage
increased from about -0.0472 to 0.0502, indicating much clearer water, with magnitucle
near 0.0030,
Average
roughly 0.05002
detectable change turbidity 2012
and 2022.
Average stoyed about -0.05 for both years,
virtually change.
B1 x 72
72
GPT-5
"name": get
"inpet": "dir pach": "benchmark desphm/kkkdaaeqeeee////883
Step
"output": ["Arctic Ocean, sur pefl, 07..2000 M'rrtic Ocean sur 604_2022-00 0022-07-29.00 07:29.017))
"name": calculate butch, ndu
"luput":"uput red, pathi": htt.://monn.tttalaussioooooooooo 82/Aretic Ocean aur reft M44 2012-07- Ol .... "benchmark lutssquuetiiiinnnnntoooo
504 2022-07-29xif1) "input, groen
("beschmark/dat/.c htt.://ai...aauuuuiiis///////// ///rriic Opean, sur refl. b0] 2012-07-000 077-011.00 0i.ie"......
Step 2-3
"benchmark daaaa/uutittuiiiinnn A2/Arctic Oceam sur ref). 501 20l__222--0 00:22-072220 07-29.162)) (9.6i"". "elii"")"""u "oupput paths": ("quescion 82/nddi 2012- 07-01.ff"
"question 82/ndii 2002-47-29..i""
"output": [''Result outiquettoon 82/ndii 2012-417-01
R"Rssltt
outiquestion R2lndi 2032-07-29
"name": mann
"input": ("-13.0959750)--555 S11.S/50......000000111555551
Step
"outpat": 51500
Final
"final answer": "B"
Step-by-Step: TAO: 0.5
TIO: 0.03
TEM: 0.03
Param.: 0.03
End-to-End:
Efficiency: 0.3548
Acc: 100%
(
6T
Tool
Tool 2
Tool 3
x2
Tool get_filelist
Tool 2: calculate_ndtii
Tool 3: cal__batch__mage_menn
Tool 4: difference
Tool 5: aalculate_batch_ndte
Tool 7: batchh_mage__ment
Tool get__ist__oject_wii___
Agent
SPT-5
Tool 1
Tool 5
x2
Tool 7
K
Kimi-Ka
Tool 1
Tool 3
beepseek V3.1
Tool 1
Tool
x4
Tool
x2
Tool
x3
Tool 4
Figure 22: Example of Pollution Regulation with Products Data under the Instruction- Following Regime.
49

---

Published as a conference paper at ICLR 2026
Products: [Urban Management)
Autoromess Planning
Based the built_ soilt_vvlume____ notame__ttalll date of Shanghai from 1980 to 2025, calculate the total change in build Building volume for Shanghai between 1985 and 2020 and compute the percentage change.
Question:
The tofal change is 10,392.58 and the percentage change is 179.96%
The total change 5,777 68 and the percentage chonge 100.34%
The total change 16, 16.70..2 27 and the percentage change is 279.96%.
D. The total change 8,392.58 and the percentage change is 159 159.96%. 96%
change
percentoge change
Build volume x 46
Deepseek
V3.1
"name": Fileist
"Input": "dir pata wwwpd//mmmdaaaaaaaaaooooooaaa (atmmm///daaaaaessiiss503"1"
Step
"output": (1 Shunghii builk, volume, total, (980-01-01 -0l..i....... ol..af"....."Shhghh built volume total 2025-01-01.aie
"name"" oale image
"inpet": "ile lise: d'bemcmmartiiiiisieeeeeee BB Shanghai. built volume total, 1985-01-0L.iP;
Step 2
"benchmark data questinn SeetllonnneeSatneaea Stanghai built volume total 2020 01-01sif]
"ourput": (504476554.00 14091027671.0
"name": percentage change
"Input": (Ta":
Step
"oufput":
Final
"final "A"
Step-by-Step:
TAO: 0.75 2 TIO: 0.25
TEM: 0.25
Param.: 0.25
End-to-End:
Efficiency: 1.0
Acc: 100%
C
GT
Tool 1
Tool
Tool
Tool 4
A
Tool get_filelist
Tool aale_batchhimmge_meee difference
Tool 4: percentage_change
Tool cale__artch_imoe_ssvi
Agent
Tool
Tool
Tool
Tool
GPT-5
K Kimi-K2
Tool
Tool
Deepseek
Tool
Tool 5
Tool
Tool
>
Figure 23: Example of Urban Management with Products Data under the Auto- Auto-Planning Regime.
50

---

Published as a conference paper at ICLR 2026
Products: [Water Mangement)
Asfonameca
Process
(August 2022) determine peak vegetation coverage
extremes.
0.202200-001 (NDVI: 0.5636) 8. 2022-08-03 (NDVI: 0.5443) C. 2022-08-12 (NDVE 0.4755)
(NDVI: 0.4958)
B2
B4
Kimii-K2
"unme": get, filelix
"Inpur"s "dir, path" "bencimmnnkedrrkuuussestei
Step
"outpur": ('Gevvall uur refl 2012-05-01 siff.... Genevval h0t 2012-08-.0.iit ''Gmevv-Lakie sur zufl b00 2012-085-0 lif" "Genove Lake sur rell, 5022 2023-08-30.11")
"name": calcubste idvi "Iaput": frod, band file": ("eechhaar data dataiquestonn Genea Lake sur reft bot 2022 oi.-o.if.......
fenchmarkiilteeeeee ilttequuiiin 255iannossaa sur rell 2002-00-30017
Step 2
"nir band file barchteerklattiqquee 25 sur reft bo2 2022.05.
Hift "henchmark dattaiqueiinn 25.0mmeva-1
bdo 2013-06-10.111
"output": peeovs 2021-0400 Tonnsay 2002-05-10.51)
"unme": cale batch, Image mean
Step 5
"imput" (filc lie! l'mneea ndvi 3022-08-01..0 ii .i... "gencou ndvi 2022-08-30..071
"omport: (0.5636, 0,54,,, 00733)
Final
"final answer":
Step-by-Step:
1
TAO: 0.67 2 TIO 0.5
TIO:
3 TEM: 0.25
4
Param.: 0.0
End-to-End:
s Efficiency: 1.2500
6 Acc: 100%
GT
T
Tool
Tool
x4
Tool
Tool 1: gelelil
bwwchhhmmaeee....eaaaaooaoooaoae max_value_one aidue__on__dded colc_barchh_mmge_mma calc__atch_immge_meeee
Tool 2: calculate_ndvi
Tool colc_
Tool 7: get_geeeeeeeeee sercentile__vlue__fre
Tool wattttt_imgeeehasppppoo perceettper
Agent
GPT-5
Tool
Tool 2
Tool 6
C
X
K Kimi-K2
Tool
Tool
Tool
Tool
Tool
Deepteek V3.1
Tool
Tool
Tool
Tool
Teal
Tool
Figure 24: Example of Water Management with Products Data under the Auto-Planning Regime.
51

---

Published as a conference paper at ICLR 2026
Products: [Water Mangement)
Pollowing
Question: ief the area where NDVI greater than 0.3 as vegetation cover. Based on the multispectral Pemote sensing data of Lake Genevo August 2022, colculate the NDVI vegetation index, determine the doily vegetation coverage proportion of lake area, identify the with highest NDVI during this month.
A.2022-08-01 (NDVI: 0.5636)
E. 2022-06 08-03 (NDVI: 0.5443)
C. 2022 (NDVI: 0.4755)
D. 2022-08-29 (NDVI: 0,4958)
B2 x 31
B4 31
K Kimi-K2
"aame": get filelist
"Isput"s { "dic pama" sencchmakddaaiqussiiiioo
Step
"'urput"::
["Gieneva-Lake, sur reft 501 2022-08-01.aif--.... (Geneva-Lake sur reft b01 2022-08-300f"
VGeenvvad ake reff NO2 2022-08-0 68-08.000000 HP, Gaeeeva-Lake roft, b02 2022-08-30.687
"nume": ndvi
"Iuput": "red, band file": ("benchmadk: lattaiqustioon 25 sur_ reft h01 2022-00-01 dif'.......
"benchmarkddaas srenechmrrkddaniqsssstooo 25 lake sur jonf hot 2022-08-30.if7
Step 2-5
band file": I"benchmark quta/qustiioonnt Geneva- Lake
ai........ www.beehckkkkkkk............c... dataqquesionn 25 sur refl, b02 2002208-30..1
"outpet": (" ('geneva ndvi 2022 01 .i....... "geneva, 2022 30.n""
"name": cale batch image, max "Input": ("flle list" ("Menchnarrkklttalgeettiii iggeneve ndwi, 3022-08-01..8
Step 14
"bunchmurk quteiutttoitnttt 25 ndwi 2022-08-04.if")
"ostput": [143 5744016 3496854 22.188842..2999342227777911
Final
answer":
Step-by-Step: TAO: 0.33 2
TIO: 0.25 3 TEM: 0.25
4 Param.: 0.0
End-to-End:
Efficiency: 3.7500
Acc: 100%
GT
Tool
Tool 2
x3
Tool 3
Tool 4
x2
Tool get_filelist
Tool 2: calculate_ndwi
cale_ image_mman calc_boteh_imoge_medin
Tool max_value_ondd_ieee
Tool aale_batch_immg__mar
Tool weteeprrnnlleevleeee_eoee
Tool 8: sastchhimog__hhttpttpprrree
Agent
GPT-5
Tool
Tool 3
Tool
K
Kimi-K2
Tool 1
Tool 12
x4 ->
Tool 6
Tool 12
x4
Tool x4 A
Deepseek V3.1
Teol
Tool
Teol 3
Tool B
Figure 25: Example of Water Management with Products Data under the Instruction- Following Regime.
52

---

Published as a conference paper at ICLR 2026
Products: [Weather Forecasting]
Autonomous Planning
Question: Based on the precipitation index of the Congo Rainforest from May to May 31, 2025. Colculate daily overage rainfall the region during this period, and linear trend determine the rainfall trend over these 31 days.
The daily average rainfall is 6.18 mm, and the rainfall shows slightly increasing trend.
The daily average rainfall is 6.18 mm, and the rainfall shows slightly decreasing trend.
The daily average rainfall is 8.25 mm, and the rainfall shows slightly increasing frend. daily average rainfall
shows slightly decreasing trend. The daily average rainfall is 5.03 mm, and the rainfall shows slightly decreasing trend.
mm.
precipitation 30
K K2
"name"; get filelist
Step 1
Inputt": "dir path": "enncmokkddnsgpssiiiin
output": ("Coggoo precipinaiion 2025-05-01,00 ..... CoCggo, precipitation. 2005-05530.iiP)
"ame"" ": cale butch image meam
Step 2
"Inpur": "file lust: ('bacciarrkiaatt/quueiinn 45- Congo precipiazion 2025-05 05.............
"benchmark danniquettion (45/o precipitation 2025-05-30..i
output": (10.)::666111000000 (16.119555471667968..... 121.
"name": mean
Step 4
"inpat":'s" 161155555777677688.... 1277787777777)))
"outpur": "6.54 63546880737922200
Final
Step-by-Step:
TAO: 1.0
TIO:
TIO: 0.75 TEM: 0.5
Param.: 0.5
End-to-End:
Efficiency: 1.0
Acc: 100%
C
GT
Tool 1
Tool 2
Tool
Tool 4
get_filelist
Tool 2: calc__atch_image_nee
Tool 3: mean
Tool 4: Samputt__ineer_treeee
Tool howg_baarchimmge_mme__neeee
Agent
Tool 1
Tool
Tool
Tool
GPT-5
K Kimi-K2
Tool 1
Tool 2
Tool 4
Tool 3
Deepseek V3.1
Tool 1
Tool 5
Tool 2
Tool
Tool
Figure 26: Example of Weather Management with Products Data under the Auto-Planning Regime.
53

---

Published as a conference paper at ICLR 2026
Instrccion Pollowing
Products:
[Weather Forecasting)
Question: Analyze the precipitation index the Congo Rainforest from May 1 to May 31, 2025, Colculate the daily average rainfall of the region during this period, and linear trend to determine the rainf olll trend over these 31 days.
A The daily everage rainfall is 6.18 mm, and the rainfall shows slightly increasing trend. the rainfall
slightly decreasing trend. The daily overage rainfall is 8.25 mm, and the rainfoll shows slightly increasing trend.
daily overage roinfall
The daily overage rainfall 8.25 and the rainfall shows slightly decreasing trend.
The daily averege rainfall is 5.03 mm, and the rainfall shows slightly decreasing trend.
precipitation 30
K Kimi-K2
"name": fileiist
"Inpur": "dir pash": danccmmankddttt/qussssiia.a
Step 1-2
outpor": (MCCoggo precipitttion 2025-05-01..i WCCoogoopopeee precipptationn 2025-05-30..1")
"name": cale image
lecr "inptt" "ile lise" l'benchmark/ dataeiuusstion (4:iiioonnSSSmmmma Congo precipitation 2025-05-01..ff
Step 3
"benchmark question C5nong precipinatoon___emcc 2025 05 30.cf"))
"oupput" [16.110559871669988...... 12.771505732727705)
"name": compate linear, trend
Step 4
"Input":""" (16. 168.1888597777778888.... 12.779905752722255
"outpur": 999251999524 7711
Final
ammer'!
Step-by-Step: TAO: 1.0
TIO: 1.0
TEM: 0.25
Param.:
End-to-End:
Efficiency: 1.2500
Acc: 100%
GT
Tool
Tool
Tool
Tool get_filelist
Tool 2: cale_ calc_bbtcch____ paatchimage__neee mean
Tool mean
Tool 4: compute compute_linear Jinelinneeeer trend
Tool 5: gal__batch_imoge_mmen___oo
Agent
1
x2
Tool
Tool
Tool
Tool
GPT-5
K
Kimi-K2
Tool 1
x2
Tool
Tool
Tool 4
Deepseek v3.1
Tool 1
Tool
Tool
Tool
Figure 27: Example of Weather Management with Products Data under the Instruction- Following Regime.
54

---

Published as a conference paper at ICLR 2026
RGB
(change detection]
Phanning
Based on satellite imagery before and after the naturd discster, determine how many buildings Were comple lely destroyed using pre- and post-dleuxter sutellitt inages.
Question:
ANNom building war completely destroyed!
8:"1 building was completely destroyed. 6.2 building completely destroyed
0.'3 building completely destroyed'
H K2
"mame"s pet, Olleis
"Ingut": Mirr pot.ttbbbbbbbroie... data queetien225"
Srep
"catpul": ("time_postt pag" /"imee_pm
"name": ChargeDS
image padi prejpng".
imape pulll: post gagg"
Step
"cater pall'" "qusttoe22ShilEiee pretii"
"cutpu!": 'Falled
"same": count connecial componaets
"lapu!": "maage patti: "deuooed tuildigs SC"
Step
"ontpas":
Final
"Btaal anreer": 'A""
Step-by-Step
1
TAO:
TIO:
3 TEM: 1.0
Param.: 1.0
End-to-End: Efficiency: 1.67
Acc: 0%
GT
Tool
Teal 2
Tool I: get filelit
Tool ChangeOS
Tool 3: count_ catnt__coneeee__ccoo compppeed_oompoenttt
threshold seaeerttooggee
Tool 6: RemoteSAM
InstructSAM
Tool
Tool 9: RemoteCLIP
colculate area
Tool 11: grayscele_ to te_colormap
subtrcct
Tool 13: get percentile. value. from image
Tool 14: Bal_ulote_if_ddffeee
Agent
Teal
Teol
Tool
Tool
Toolt
Teal
Teol 10
Tool
Tool 3
H"
Tool
Tool
Tool
X
Deepseek
Tool
Tool
Tool
Tool
V3.1
Figure 28: Example of Change Detection with RGB Data under the Auto-Planning Regime.
55

---

Published as a conference paper at ICLR 2026
Rilli
detection)
Sused sutellite imagery before ofter the netural disuseer. the first obiuin the tolen before ofter the derect buildings both time identiify which buicding nave been comple completely destroyid,, count their tatll number.
information,
bailding complately dastroyed" building comaletely cestroyed.
building completely destroyed buiding comp erely destrayed!
Kim
"aame":get fieist
"lapa": pall Teaatmaai "1
Step
"Natowr": (Minse pootppe" ('inn, preass"
"sane"" Changels
"Ippa": Tee image put": "iru prapee"
Step 2
ppod imse pull "iin potaon";
"eerpet patt Socooee25hiiiigg pros
"sarpar": "Pailed medd mraee"
MAsstryed
Step
"cutpul": "tp
"Tiaal "orver":
Step-by-Step.0
C TAO:
TI: 1.0
3 TEM: 1.0
Param.: 1.0
End-tooEnd: Efficiency: 2.0
Acc: 100%
GT
Tool 3: get fieliet
Teal ChangeOS
Tool 3: count. corneccte__compprcee
Tool 4: srggmttissegmentttion
Teal 5: SM3bet
Teel RatroeeAM
Tool 7: InstructSAM
Teal MSON
Tcol 9: RemoteCL1P
Tool grayacale, to colorrup
Tool 12: subtrecT
get_percentlk www.e_from_ineeeeeeeeee fron_inage
Tool calculate difference
Agent
Tool
Tool
Tool 2
Teal 12
Tool 4
12-20m
Tool
Teal
FAIL
H
Tool
Tool
Tool
Tool
Tool
Tool
Tool
Tool
Despseek V3.:
Teal
Tool
Tool 10
Figure 29: Example of Change Detection with RGB Data under the Instruction-Follovii Regime.
56

---

Published as a conference paper at ICLR 2026
Autonorious Plansing
RGB:0 [classification)
Queestion: Based on the following images, every image belorgs to Airport, BareLand, BasebaliFied Beach, Bridge, Center. Church, Conmercial, DenseResidential, Desert, Formland. Forest, Inckstria. Mcodow, MediinnResidentaal Mountain, Park, Farking, Playground, Pond, RuilweyStation, River: School, SplrseResidetiil. Squire, Stadium, SterageTanks Vioduct, determine the number images captured in bareland areas.
8.'8"
C."10'
666
RGB 12
GPT-5
"nare"":gut get llllii
"input": dir beecc//eeeeeceeeieeie.e datti/gnettonnn
Step
"outpar": ("Auppe ".... "L.jog'
"naree": XSCN
"impot": "ingut_image imagg_peel pell" /bwnshoood//eeequooiitt
Step
"wulpet", "eansiiied, class 'shand', 'oudileauel 0.764G5907122771 layd5 preliction! [[Sulaxlt
l'Aipatt 8.030224 815231
211662222442266666 ('Dasarf,
00119900228073377) (Beud) 0.0000.1155007:333311
"mume": MSCH
"ingut": "kipat image pull"" "benclatari Atnpl/aaaa//daaaaueiili//......
Step
"outperti: (predictdd plase SppesaResidrrial; 'confidence" 0311951667668666 "InpS predictions: [[SparscRovilaaiat 0.81 13516372600066 Stung-- 0u0154294290 6467), (MMaaddn''
(1.0 1857446555199977 (Photf 3011 179826577744444 ( (River, 010100000 11 17683499777)
Final
"final anowsett TA"
Step-by-Step
TAQ: 1.0
TIO:
TIO: 1.0
TEM: 1.0
Param.: 1.0
End-to-End:
Efficiency: 1.00
Acc: 100%
GT
Tool
Tool 2
>> A
Tool 1: get_filelist
Teal 2: MSCN
Agent
Tool
Tool
GPT-5
K
Kimi K2
Tool
Tool
Deepseek v
Tool
Tool 2
12 A
Figure 30: Example of Classification with RGB Data under the Auto-Plannins Regime.
57

---

Published as a conference paper at ICLR 2026
RGB: [classifecation]
Quesstion: Based following images, every image belongs (Airpert, BareLond, BaseballField. Beach. Bridge, Center, Church, Commercial. DerseResidentiol, Desert, Formlond. Forest. Industrial. Meadow, MediumResidentiol, Mountain, Park, Parking, Playground. Fond, Port, RailweyStarion, Resort, River, SperseResidentidl, Square,
Storage Tanks, Viaduct), information, then use the classificction tool to assign category to each image, and finally determine the number of images captured in bareland areas.
step
Ar"3 3
6'10""
D.'6*
RGB x 12
GPT-5
"name""ggu get llllii
"input": dir "beccheannieaeeeeee.aee 1
Step
"outpar": ("Auppe ".... "L.jog'
"naree": XSCN
"inpot": "ingut iwagg_peill pell" "Bonchoook j1A
Step
"outper": "[peedicted clrset "Sebool; "confidenes' 7886108100 trep gradictions (CSoboot,
002022:81522120000 (1areLaadd 0111257213422660 ('Desset
001599000 (Bescit" 101.50111500000000000
"narae": XSCN
"Inpur": "ingut iaage patl': "Besslent//aaxpppeelll1........"
Step
"outgrat!: " 1'poddcted class:: SparaaBeudettaf 'saudfieeco'' 01115515572660664, "lup5 productous? [CSpaneRasteuial, 0.81 19910332600669) (Stunge"annks" 00019224295011666T1 ('hleadow',
0.11882346655508957, (Pond' 0.01 191766662767700
(Rivee) 000000000000666222255000
Final
"fimal ancwsett TA"
Step-by-Step
TAQ: 1.0
TIO:
TIO: 1.0
TEM: 1.0
Param.: 1.0
End-to-End:
Efficiency: 1.00
Acc: 100%
GT
Tool
Tool 2
>> A
Tool 1: get_filelist
Teal 2: MSCN
Agent
Tool
Tool
GPT-5
K
Kimi K2
Tool
Tool
Deepseek v3.1
Tool
Tool 2
12 A
Figure 31: Example of Classification with RGB Data under the Instruction Follooiin Regime.
58

---

Published as a conference paper at ICLR 2026
RGB:
[detaction]
Question: part regional sports infrastructure audit, you "asked with fattiating accupied by baseball diamondia using bourding boxea(6SD m/ pa)).
2"
8."About 1500 m"
C'"About 3500t-3000 "2
80119
K'Kimi-22 K2
"aome": Delist
"inguf": hatt:////aacoooooooooooooooo
Step
"outpat":
[oamppu"=="PP00000 [1780822 pigl")
"aume": InstaiAMM
"ingut": "ispst image,
promppt diumnedd
Step
"ourptt" Failed call model
"Input": "bboxes" [2226657656110006 29100/0000.2000117777114111 391555501013 (6280125999244 (83.22994/2180551
Step 4
2201::29990005512222220055333330 1027 823664331441999 221.6941788.35466]
"outpat": 055
Final
"final ancwer":
Step-by-Step:
TAO: 1.0
T1O
2
TIO: 1.0
TEM: 0.0
Param 0.0
End-to-End:
Efficiency: 1.33
Acc: 0%
GT
Tool
Tool 2
Teol 3
Taol get__illllee filelist
Tool 2: SM3Det
Tool 3: calculate bicculate_bbbxxcoo
Tool 4: InstrucTSAM
Agent
Tool
Tool
Tool
x .*
K
Kimi-K2
Tool 1
Tool 4
Tool 2
Tool 3
Deepseek v3.1
Tool
Tool
Tool
x -> B
Figure 32: Example of Detection with RGB Data under the Auto-Planning Regime.
59

---

Published as a conference paper at ICLR 2026
RGB: [detection]
Follwwing
As part a regional sports infrustructure oudft, you are tasked with identifying baseball clamonds visible in the satellite imagery and estimating their total area using bounding boxes (SSD 0.13 m/px).
Question:
Al'bout 500 m 2"
8/About 1500 m 2"
About 80119
K' K2
"name": wwt.///nnnnnneeeeeeeeeeeieiinnne
"Input"s "
Step1
"name": InstarAAM
"inpul" "ispu, image path "senchomekk "anntommmesaaaaauuusie/000000000
"est peomp"" "baschall dimmndd"
Step
"outpor": Failed model
"name": calculate bbos,
"inpul": "Mhuss"" (15246667765,,224666 731455555331166 464444222222444 418.529914422811664844
(8207712416099957,, 512099977115522111444444553 10270226645744777 721.65417813546611
Step
"ostput": "96/738.04630033"
Final
"final answer":
Step-by-Step:
TAO:
2 TIO: 1.0
TEM:
Param.:
End-to-End:
Efficiency: 1.33
Acc: 100%
GT
Tool
Tool 2
Tool
Tool gilelil
Tool 2: SM3Det
Tool calculate_bbox_aree
Tool InstructSAM
Agent
GPT-5
Tool 1
Tool
x2
Tool
K Kimi-K2
Tool 1
Tool 4
Tool 2
Tool
B
Deepseck V3.1
Tool
Tool
Tool
Tool
Figure 33: Example of Detection with RGB Data under the Enstruction-Follwwee Regime.
60

---

Published as a conference paper at ICLR 2026
RGB: [Visual Grounding)
Autumationaa thanring
Question: Sared following mages, what the centroid-to centroid distance between the two plane image? (SSD 0.14 m/px)
Abbuut 265
Abouutt
About 280
GPT
"axms"s Edlist
"Inpui": 11
Step
"outpat": "[FFLEEE
"uame": 500
"Inpat": "laput image path": put.:
"teo prompe: "plane"
Step
"outpet": "[[522 4512103379632, 243 89258468500612 8562773025911246, 22477020000044422220000000000000 (833019999 (202808729144222 261 04944199,,, 66380094199977 622113669222
"asare: controu ditaace estremes
"input" "cantroid"" (699.30) 38158, 34421009440231)..... (1033345799995555 101113759619100)
Step
"ostput" "Eroc: ToolEExcppionn calling toestrod
...
detiowl "ya your maiaaii..
Final
"Final smower"="A"
Step-by-Step:
1 TAO: 0.8
TI0:088
TEM:0.8
Param.:C.8
End-to-End: Efficiency: 1.0
Acc: 100%
GT
Tool
Tool
Tool
get__filelin feleliil
Tool SM3bet
Tool centroid centroid_distance,
distance_extrrmms extremes
Tool 5: multiply
Tool 6: difference
Tool add
division
Teol 9: calculate bicculate_bbbx__reee
Agent
Tool
Tool 2
Tool 3
Tool
Tool 5
GPT-5
K
Tool
Tool 3
Tool
Tool
Deapseek v3
Tool
1ool
Tool
Tool
12->
Tool
Tool
2->
Tool
2-
Tool 10
Tool
50
Figure 34: Example of Visual Grounding with RGB Data under the Auto-Planning Regime.
61

---

Published as a conference paper at ICLR 2026
RGB: (Visual Grounding)
Question: Based the folloiin images, the first step to obitoin the folder in formaa ion then detect all the image, "hair centrolds, Find the forthest disfonce.
centrolde, calculate distances between each poir
A-Abbuut 265 m. 8. A.TAbout 270 m. C.'About 275 About 280
GPT-5
"name": fielist
"sapat's potht "benceeessbbbbbbeee question23"")
Step
"output": "F""1192,,ug"""""
SM3Da
"Inpat" iiput image path": hew.eeoneeeiiiiiiniiiieeeeeesosr
"ond pomple: "plane"
Step
"eutpul".
322308150:34212,, 263 545.4599,,229222 1995688804995577 9221599993
"sume":
centroid distunce extremes
Step 4-5
"ocationl" [6963671126644438, 210201052292244... (104442000....011022229295535550 210
"output": "Emor ToolExppioon "Emor calling toel leentrudd, wwtpiidddsiiieeeeeeeee extremes dafined your mintilon."
Final
"Boal ouewer": TC*
Step-by-Step
TAD: 0.8
TIO: 0.8
TEM:0.8
Param.:0.8
End-to-End:
5 Efficiency: 1.0
Acc: 0%
GT
Tool
Tool 2
Tool 3
Tool 4
Tool
Tool 1: get_filelist
Tool 2: SM3Det
Tool 4: sentroid_distance__oo
Tool 5: multiply
Tool 6: difference
Tool
add
Tool 8: division
Tool calculate_bbar biliiatt_bbbx_areee
Agent
Tool 1
Tool 2
Tool 3
Tool 4
6P
M
Tool
2-
Tool
x4 x2-
Teol
C
Kitii-K2
x
Deepseek V3.1
Tool
Teal 2
Tool
Tool 6
Tool
Tool
Figure 35: Example of Visual Grounding with RGB Data under the Instruction-Followe Regime.
62

---

Published as a conference paper at ICLR 2026
I CASE STUDY: COMPARE WITH OTHER AGENTS
Products: [Urban Monagement)
Ground Truth
&'Question: Analyze the nighttime light intensity frend Leon (2013.2024) using linear regression.
Operator
Sloppe
Slalpp Intercept:
MSX
S'Slope: 0.14; Intercept: 2.05"
b Slope 0,36: Intercept: 2.12*
Manus
Coze
Eorth-Agent(GPT3)
:B
Eorth-AgentDeepseee-V....
:B
avg_rad xad
Figure 36: A Question Case of the Urban Management Task using Products Data with Re- sponses from Different Agent.
Management)
Ground Truth
Question: Compare nighttime light intensity Chile from Janwary to March 2015, and calculate their average intensity difference.
Santiago
Operater
MGX
>C
A'4.87"
8.'5.02"
C15,21
Manus
0.55.66
Coze
Eorth-Agent(6P55
:c
Earth-Ager(DeeseekkVJJ
X
Figure 37: A Question Case of the Urban Management Task using Products Data with Re- sponses from Different Agent.
Ground Truth
Question Based determine how mony buildings were completely destroyed using pre- and post- discster satellite images.
Imagery after natural disaster,
Operater
"None building completaly destroyed",
destroyed.
building was completely destroyed " building was completaly destroyed"
M6X
:FAIL
Manus
:B
Coze
:FAIL
Eorth-Agent(PT5)
:A
Earth-Agpen(Deepseeee......
Figure 38: A Question Case of the Change Detection Task using RGB Data with Responses from Different Agent.
63

---

Published as a conference paper at ICLR 2026
RGB: X [elasifiiation]
Quuestioo: Based on the following images, every image belongs to Airport, BareLand, BasebaliFieid, Beach, Bridge, Center, Church, Commercial, RenseResidential Desert, Formland. Forest, Industrial, RailwayStation,
Ground Truth
Rediinnesiionticl Mountain, Pork, Parkingg
River, School, Sparss SparseResiientiol Square, Stodium, Storage Tonks, Viaduct, determine the number images coptured bareland areas.
Playground,
Operator
8,"8"
M6X
:FAIL
C"10"
D '6"
Manus
Coze
FAIL
Eorth-Agont(6PT)
Earth-Agan(Deppseek-B.
R86X12
Figure 39: A Question Case of the Classification Task using RGB Data with Responses from Different Agent.
[classification]
Question Raded the following images, image belongs (Airport, BoreLand, BasebaliField, Beach, Bridge, Industrial, DenseResidential, Desert, MediunResidential Mountain, Park, Parking, Playground, Pand, Part, RailwayStation, Resort, River, School, Spar dedeeidenntill, Square, Stadium, Storage Tanks, Viaduct), determine the number of images huptreed mountain areas.
Ground Truth
Church, Commercial,
Operator
MGX
B""""
2-2'
D
Manus
Coze
-FAIL
Eorth-Agent(GTT5)
:B
Eorth-AgantDeepsekkkk..
:B
RGBX11
Figure 40: Question Case of the Classification Task using RGB Data with Responses from Different Agent.
64

---

Published as a conference paper at ICLR 2026
Boli
(danection)
Ground Truth :B
A Question: As port of regional sparts infrestructure audit, you are tasked with estinsting tenal occupied by boseball clamands msig bounding boves(G60 0.13 m/po).
Operator:B
MGX
FAIL
X
3500 "22 Aboburr 80119 nn
X
Manus
:A
X
Coze
FAIL
Earth- Earth-Agent(GPT5) FAIL
X
Earth-Agent(DeepSeek V3..
Figure 41: A Question Case of the Detection Task using RGB Data with Responses from Dif- ferent Agent.
RGB:
(Visual Grounding)
Ground Truth
d Question: Boned on the following images, calculate the centroid-to torceddd---- distonce between the two farthest plane in the Image? (SSD = 0.14 m/px).
Operater
About C"About 275 b. "Abbut 280 m."
MGX
=FAIL
Earth (grth-Agee((6P15)
:A
Earth-Aggen(DaeeSeekkaaa
Figure 42: A Question Case of the Visual Grounding Task using RGB Data with Responses from Different Agent.
65


Reviews:
[{"review_id": "1UZCXDsDgK", "reviewer": "Reviewer_a78N", "summary": "The paper introduces Earth-Agent, a novel agent-based framework for Earth Observation (EO) analysis using multimodal large language models (MLLMs). Unlike previous EO models that rely mainly on RGB data and limited reasoning, Earth-Agent integrates RGB and spectral EO data and leverages MCP tools for cross-modal, multi-step, and quantitative spatiotemporal reasoning. It can dynamically employ expert tools for tasks such as geophysical parameter retrieval and spatiotemporal analysis. To evaluate its performance, the authors propose Earth-Bench, a benchmark comprising 248 expert-designed tasks and 13,729 images, offering dual-level evaluation on both reasoning and outcomes. Experiments across various LLM backbones show that Earth-Agent significantly outperforms existing MLLMs and general agent frameworks.", "questions": "- One aspect that is unclear is how the proposed method handles cases where a tool produces an incorrect output. The framework does not appear to inherently address such failure cases. For example, if an error occurs during the first tool call, it may propagate and amplify throughout the subsequent reasoning chain. Did the authors encounter or observe such cases in their experiments?\n\n- Throughout the paper, the authors repeatedly state that the dataset is expert-curated. However, the paper does not specify who these experts are or provide details about their backgrounds. It would be helpful to clarify whether the curators come from computer science with remote-sensing expertise, or if they are domain specialists primarily from the remote-sensing or Earth observation community.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- Solving earth observation tasks using an agentic solution is quite novel and timely. Bringing a tool-use angle from agent solutions to improve performance seems very innovative.\n- The proposed benchmark can be useful for the research community in terms of evaluating agentic systems.\n- Experiments are thorough and comprehensive. Findings are supported by enough experiments.\n- Paper is well-written and easy to follow.", "weaknesses": "- In any agentic system, a comparison of latency with a single-agent or single-model baseline is expected. Although the performance gains are significant, it is also important to account for the computational cost, including the number of LLM or tool calls and the total end-to-end response time. Such comparisons are important to demonstrate the overall efficiency and effectiveness of the proposed agent framework. Please add a discussion around it in the paper.\n\n- A discussion of performance trends w.r.t. to the number of tools used for a given task is important to understand how the proposed agentic system behaves as task complexity increases and the number of tool calls grows. It would also help evaluate the system’s latency and scalability under such conditions. I would like to see such ablation studies included in the paper to better demonstrate the effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "msriNf0QGA", "reviewer": "Reviewer_1ij5", "summary": "This paper has two main accomplishments: First it proposes an agentic framework for earth observation integrating 104 tools and models. Secondly they prepare a benchmark (Earth-Bench) of 248 expert-curated questions with 13,729 image.  They then evaluate the agent on the benchmark, comparing it to openai operator and Manus. They also demonstrate that the model outperforms remote sensing foundation models.", "questions": "Please respond to the weaknesses I highlighted:\n1. Explain more about the benchmark and why you made the choices you did when constructed it\n2. Explain the relationship of your toolset to your benchmarks and what you did to make sure you weren't cherry picking tools for benchmark questions\n3. Explain how your agent does on the cited benchmark above.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The sterngth of this work are the results: both with the agentic system design, the benchmark design, the demonstration of SOTA results across a range of existing remote sensing benchmarks . The agentic system uses a wide set of specialized tools that were reasonably selected and the authors evaluated this system across a range of LLMs. They also constructed a smaller benchmark (earthbench-lite) to enable comparison against generalized agents and show that their agent outperforms consistently", "weaknesses": "When reading the paper I would have liked to learn more about the benchmark itself. How it was constructed, what are the types of question in it, and so forth. Some information is in the appendix but I feel more detailed analysis would be helpful. \n\nAlso the authors have a long tool list and they also built a benchmark. We need to understand the relationship of the tool list to how the benchmark was constructed: at an extreme limit you could have included tools that were specifically chosen to solve individual sets of questions and thus push benchmark performance way up but in a way that is not extensible to larger sets of questions not included. I'd feel better about this if the authors could give a rationalization for their toolset that is independent of their benchmark.  Some solace is given since they outperform previous benchmarks but as they themselves point out the benchmarks are not really comparable. \n\nOne benchmark that might be more useful to compare against is this one: https://openreview.net/pdf?id=oaYShIy3Xe.  I'd be curious how well this system does against that benchmark", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "otePkWNfUF", "reviewer": "Reviewer_ic4k", "summary": "This paper proposes Earth-Bench, an expert-certified benchmark accompanied by an agentic framework for evaluation. It includes 248 questions, making it the largest Earth Observation (EO) benchmark to date in the LLM-agent domain. Using their Earth-Agent framework, GPT-5 successfully resolves around 60% of the questions, and the overall ranking aligns well with current expectations of LLM capabilities.\n\nHowever, the paper lacks clarity regarding the dataset curation process, e.g., whether each question was reviewed by multiple experts. Additionally, several inconsistencies and issues within the benchmark data, as well as insufficient detail about the true number of unique questions (i.e., whether it is genuinely 248 questions or just 124 formulated in two ways), raise concerns about the benchmark’s reliability and robustness.", "questions": "**Clarification of Anti-Planning and Instruction-Following**\n\nCould the authors clarify whether all 248 questions each have both an auto-planning and an instruction-following version, or whether 124 are auto-planning and the other 124 are instruction-following? If it is the latter, then claiming to have 248 unique questions seems somewhat misleading, as there would actually be only 124 distinct questions expressed in two different formulations.\n\n**Dataset Composition**\n\nHow many products or sensors are included in the dataset? So far, I can identify Landsat and MODIS (ASTER), and potentially some high-resolution imagery (Page 52) with unspecified sources. It would be helpful if the authors could describe the data sources, products, and collections in more detail, along with their distribution.\n\n**Figure 7 Inconsistency**\n\nIn Figure 7, there appear to be more than 300 error counts, even though the dataset only contains 248 questions. Could the authors clarify how these counts are computed and why they exceed the total number of questions?\n\n**Evaluation Without Tool Use**\n\nHow do the authors implement and evaluate model performance without tool use? For RGB-based questions, how do the models perceive or interpret the images if they are not allowed to invoke perception modules?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "1. Overall Evaluation:\nThis paper explores a wide range of tools and evaluates how effectively current LLMs, acting as agents, can leverage these tools for question answering. The experiments are comprehensive, and the results are clearly presented and well-analyzed.\n\n2. Contribution of Toolset:\nThe predefined set of 104 tools covers a broad spectrum of Earth Observation (EO) tasks and represents a valuable contribution toward advancing tool-augmented agents in the EO domain.", "weaknesses": "While this paper seems to propose a genetic framework and a comprehensive benchmark, I found several issues that compromise my ratings for this work. Specifically about the **rigor and the reliability of the dataset**. Below are two of the main weaknesses I observed\n\n**1. Unclear data annotation pipeline**\n\nThe authors stated, “From these data sources, a team of domain experts ...,” but no further information is provided. For example, what is the background of these domain experts? Is there a cross-reviewing process? What happens if inconsistencies arise? Was an LLM used during the curation process? How did the experts formulate the questions?\n\nWithout a clear description of the question validation process, it is difficult to ensure that the benchmark is trustworthy and reliable.\n\n**2. Questions about the benchmark dataset**\n\nI reviewed some of the benchmark data provided, and found several questions. \n\n**- Question Predictability:**\nOn Page 37, the question involves Landsat 8, which has a 16-day revisit cycle. Thus, between June and September, only about 8 +- 1 observations are available. Thus, even without reading the question carefully, one could guess any option less than 8, which again raises concerns about how informative the task is. (The options provided are (1) 5, (2) 10, (3) 12, and (4) 18 days. )\n\n**- Incorrect Formula Range:**\nOn Page 43, the Normalized Difference Turbidity Index (NDTI) is reported with values ranging from −666 to 100. However, NDTI is typically computed as (red − green)/(red + green), meaning its range should be [−1, 1]. For example, Option B claims: “The average NDTI increased from −666.47 to −61.17,” which is mathematically implausible and impossible. Feel free to correct me if the authors use different metrics.\n\n**- Methodological Justification:**\nOn Page 51, the method count_skeleton_contours is used to estimate the number of houses, but no reference or justification is provided. A citation or methodological rationale is necessary to validate this choice.\n\n**- Question Design Issues:**\nFor example, on Page 53 (Line 2808), only four images are provided, but the answer choices are (A) 3, (B) 8, (C) 10, (D) 16). It seems trivial to select “A” without even understanding the question, which undermines the reliability of the benchmark. A similar issue appears on Page 63 (Line 3356).\n\n**- Self-Conflicting Content:**\nThe questions on Pages 55 and 56 appear inconsistent. The questions ask about the total area. However, the first question asks about \nsports infrastructure including both baseball and tennis courts, while the second focuses only on baseball courts, yet the answers are both 1500m*2, which is confusing to me. The two questions should have different options and the current version makes the intended reasoning unclear.\n\n**- Ambiguity in Measurement Definition:**\nOn Page 65, the task asks for the distance between the two farthest planes in the image, but it is ambiguous how this distance is defined. By contrast, Page 58 explicitly specifies centroid-based distance, suggesting that the two questions target different notions but are treated as equivalent in the benchmark.\n\nI understand that some technical details may have been omitted for brevity, and I would appreciate clarification from the authors regarding these issues and any potential misunderstandings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9VojGL5112", "reviewer": "Reviewer_38ix", "summary": "Earth-Agent extends multimodal LLMs for Earth observation (EO) by enabling spectral data understanding, multi-image reasoning, and scientific tool use. It employs a ReAct-style LLM agent that dynamically invokes 104 EO expert tools via a Model-Context Protocol for spatiotemporal analysis and workflow planning. The authors introduce Earth-Bench, a 248-task benchmark (13,729 images) spanning RGB, spectral, and geospatial data, evaluating both final accuracy and reasoning quality. Experiments using backbones like GPT-5, Kimi K2, and DeepSeek-V3.1 show Earth-Agent dramatically outperforms prior MLLMs and generic agents on EO reasoning and traditional benchmarks, establishing a new agentic paradigm for scientific Earth analysis.", "questions": "- The evaluation uses expert-curated solution trajectories for step-by-step scoring. How does Earth-Bench handle cases where the agent’s reasoning differs from the expert’s? For instance, if an agent uses a slightly different sequence of tools or a valid alternative formula to reach the correct answer, do the metrics give it credit? Is the benchmark tolerant to creative solutions? A discussion on how variability in reasoning is treated would be useful for users of Earth-Bench.\n- The identified tool and file hallucinations indicate the agent sometimes loses grounding in the environment. What strategies do the authors think could reduce these errors? For example, would fine-tuning the LLM on a corpus of tool-use transcripts or adding a verification step where the agent double-checks the existence of a tool/file before calling it help? During development, did the authors try any prompting techniques to address this?\n- The domain of tool-using LLM agents is evolving (e.g., SciToolAgent for scientific workflows, etc.). Could the authors contrast Earth-Agent with such systems? Is the main difference primarily the focus on Earth science modalities, or are there methodological innovations that set it apart?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Earth-Agent enables multimodal, multi-step EO reasoning by dynamically invoking 100+ expert tools, which is far beyond prior static MLLMs.\n- Earth-Bench with 248 tasks and 13k+ images rigorously evaluates both reasoning and results, setting a new standard for EO agents.\n- Consistently outperforms general agents and prior MLLMs across modalities and EO benchmarks.\n- Extensive comparisons and ablations across LLM backbones validate tool-use effectiveness. Also identify and discuss failure modes like tool/file hallucination, offering clear improvement directions.", "weaknesses": "- This zero-shot agent sometimes makes mistakes that a more specialized model might avoid, like tool hallucination and file hallucination. These errors suggest the LLM doesn’t fully understand the tool API or environment state at times. A weakness of the paper is that it does not attempt any intermediate fine-tuning or training to address these issues.\n- The approach relies on a single “expert” solution path for each task. It’s unclear how the evaluation handles an agent that finds a correct answer via a different valid approach. The authors do measure partial matches, which helps, but the paper doesn’t fully discuss if an agent could be penalized for creative but correct reasoning that deviates from the expert trajectory. This could be a minor weakness if the evaluation isn’t robust to alternative solutions. Clarification would be helpful to ensure the benchmark is fair to different reasoning styles.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)