Title: GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution
Venue: NeurIPS 2025
Accepted: yes
Keywords: Remote Sensing

MLLM

Vision-language Dataset

Ultra-High-Resolution
Abstract: Ultra-high-resolution (UHR) remote sensing (RS) imagery offers valuable data for Earth observation but pose challenges for existing multimodal foundation models due to two key bottlenecks: (1) limited availability of UHR training data, and (2) token explosion caused by the large image size. To address data scarcity, we introduce **SuperRS-VQA** (avg. 8,376$\times$8,376) and **HighRS-VQA** (avg. 2,000$\times$1,912), the highest-resolution vision-language datasets in RS to date, covering 22 real-world dialogue tasks. To mitigate token explosion, our pilot studies reveal significant redundancy in RS images: crucial information is concentrated in a small subset of object-centric tokens, while pruning background tokens (e.g., ocean or forest) can even improve performance. Motivated by these findings, we propose two strategies: *Background Token Pruning* and *Anchored Token Selection*, to reduce the memory footprint while preserving key semantics. Integrating these techniques, we introduce **GeoLLaVA-8K**, the first RS-focused multimodal large language model capable of handling inputs up to 8K$\times$8K resolution, built on the LLaVA framework. Trained on SuperRS-VQA and HighRS-VQA, GeoLLaVA-8K sets a new state-of-the-art on the XLRS-Bench. Datasets and code were released at https://github.com/MiliLab/GeoLLaVA-8K.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the key challenges of applying Multimodal Large Language Models (MLLMs) to ultra-high-resolution (UHR) remote sensing imagery. The authors identify two primary bottlenecks: the scarcity of high-resolution training data and the \"token explosion\" problem, where the large image dimensions lead to computationally prohibitive sequences of visual tokens.\nTo tackle these challenges, the paper makes three main contributions:\n* Data: It introduces two new, large-scale VQA datasets for the RS domain: SuperRS-VQA, a manually annotated dataset with UHR images (avg. 8K resolution), and HighRS-VQA, a larger, semi-automatically generated dataset with medium-to-high resolution images (avg. 2K resolution).\n* Method: It proposes a novel two-stage token management strategy to handle UHR inputs efficiently. This involves (i) background token pruning, which uses semantic clustering to merge and remove redundant background tokens (e.g., ocean, forest), and (ii) anchored token selection, which leverages attention from the ViT's [CLS] token to retain the most semantically critical foreground tokens.\n* Model: It develops GeoLLaVA-8K, the a remote sensing focused MLLM capable of handling 8K resolution inputs. By integrating the proposed token reduction techniques and fine-tuning on the new datasets, GeoLLaVA-8K achieves new state-of-the-art results on the challenging XLRS-Bench benchmark, outperforming strong open and closed-source models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The paper targets a highly significant and difficult problem at the frontier of MLLM capabilities. Scaling models to UHR imagery is a critical next step for many real-world applications, particularly in remote sensing where fine-grained detail is paramount. The work is timely and the problem is very well-motivated.\n* The creation and release of SuperRS-VQA and HighRS-VQA are substantial contributions in their own right. Addressing the data scarcity bottleneck with two high-quality, high-resolution datasets will undoubtedly be of great value to the research community. The methodology for creating HighRS-VQA, including the use of an influence-based data selection pipeline (LESS), is sophisticated and demonstrates a commitment to data quality.\n* The proposed token reduction strategy is elegantly motivated by a series of pilot studies (Sections 4.1, 4.2) that clearly demonstrate the redundancy of background tokens and the importance of sparse object tokens in RS imagery. The two-step approach of first pruning the background and then selecting the foreground is technically sound.\n* The experimental evaluation is comprehensive and convincing. GeoLLaVA-8K's performance on the difficult XLRS-Bench is impressive, setting a new state-of-the-art and outperforming even much larger models like Qwen2.5-VL-72B and proprietary APIs like GPT-4o. \n* The paper is well-written, clearly structured, and easy to follow. The figures, particularly Figure 1 (overview), Figure 5 (motivating token pruning), and Figure 7 (method schematic), are excellent aids for understanding the paper's core concepts and contributions.\n\nWeaknesses:\n* The \"Background Token Pruning\" assumes that backgrounds (like forests, water) are semantically homogeneous and can be easily clustered and removed. This might fail in complex, heterogeneous scenes (e.g., suburban areas, sparse agriculture) where the distinction between \"background\" and \"foreground\" is blurry, potentially causing important contextual information to be discarded. Moreover, the \"Anchored Token Selection\" assumes that the [CLS] token's attention is the optimal proxy for semantic importance for all VQA tasks. The [CLS] token is trained for global image classification. For reasoning tasks that depend on the relationship between multiple small but critical objects, the [CLS] token might focus on a single large, prominent object, causing the smaller but contextually vital objects to be pruned. This could be a significant limitation for complex reasoning.\n* The method is motivated by reducing the computational burden of UHR images. While it successfully reduces the number of visual tokens fed to the LLM, it introduces a non-trivial pre-processing step involving iterative clustering and attention map computation. The paper provides no analysis of the computational overhead (e.g., latency, FLOPs) of this method itself.\n\nQUESTIONS:\n* This work is motivated by efficiency, but the efficiency of the method itself is not measured. Could the authors provide a brief analysis of the computational overhead of your method? A small clarification here would make the practical benefits clearer and provide a more complete picture of the practical trade-offs involved.\n* The use of [CLS] attention for selecting important tokens is a key design choice. Could you enhance the appendix with a small qualitative analysis on this point? For example, select 1-2 examples from the XLRS-Bench where the question involves reasoning about multiple small objects or their spatial relationship. Please visualize the [CLS] attention map and the final selected tokens. Does the attention successfully capture all task-relevant objects, or does it sometimes focus on a single, most prominent one?\n\nLIMITATIONS:\nThe main paper includes details about the limitations of this work. The authors separately discuss the limitations and impacts for the model (GeoLLaVA-8K) and the datasets (SuperRS-VQA & HighRS-VQA), covering important angles like sensor scope.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis presents GeoLLaVA-8K, a multimodal LLM designed for high-resolution remote sensing tasks. The authors introduce SuperRS-VQA (avg. 8,376×8,376) and HighRS-VQA (avg. 2,000×1,912), the highest-resolution vision-language datasets in RS to date, covering 22 real-world 7 dialogue tasks. The authors also propose two strategies: Background Token Pruning and Anchored Token Selection, to reduce the memory footprint caused by the high resolution while preserving key semantics. Ablation studies demonstrate the effectiveness of the design. Experiments show that GeoLLaVA-8K achieves the best performance compared to closed-source or open-source models.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- The datasets presented in the paper have a large number of samples and cover diverse tasks, which can be very helpful for future research.\n- The proposed designs for the model are well-motivated by the features of RS data (e.g., a large fraction of background regions) and lead to better performance. Background Token Pruning and Anchored Token Selection could also be easy to implement.\n- The paper writing is clear and easy to understand.\n\n**Weaknesses:**\n\n- The comparison with other models is only conducted on the XLRS-Bench, while preliminary experiments for the designs are also performed on the same benchmark. More datasets for evaluation are needed to show the generalizability of the method.\n- Only certain types of tasks show performance benefit, like RP, CCR, and OLUC. But on tasks like RC, AD, and OC, the performance is actually not superior.\n\nQUESTIONS:\n- Similar to the weakness, it would be helpful to show a comparison with other datasets.\n- What is the difference between the tasks that GeoLLaVA-8K performs better or worse, and what could be the potential reason?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces GeoLLaVA-8K, a remote sensing (RS)-specific multimodal large language model designed to address two key challenges in ultra-high-resolution (UHR) RS imagery: scarce training data and token explosion. The authors first construct SuperRS-VQA (8K resolution) and HighRS-VQA (2K resolution), two datasets covering 22 real-world RS tasks, providing high-resolution data for model training. Observing that RS images feature redundant background tokens and sparse target tokens, they propose Background Token Pruning to remove redundant background information and Anchored Token Selection to preserve target-related tokens, reducing computational overhead while maintaining key semantics. Experiments show that GeoLLaVA-8K achieves state-of-the-art (SOTA) performance on the XLRS-Bench benchmark, outperforming all existing models in 8K-resolution RS tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. High-Resolution Datasets：SuperRS-VQA and HighRS-VQA fill the gap in UHR RS training data, offering 10× higher resolution than previous datasets. Data quality is enhanced via influence function-based selection, aligning with real-world RS scenarios.\n2. Efficient Token Compression：Background pruning reduces 50% of redundant background tokens, while Anchored Token Selection leverages attention mechanisms to retain target-relevant tokens, enabling 8K processing without memory overflow and improving computational efficiency.\n3. Superior Performance：With only 7B parameters, GeoLLaVA-8K outperforms 72B closed-source models, excelling in complex tasks like object counting and route planning, which validates the effectiveness of domain-adapted modeling.\n4. Clear Research Logic：The paper presents a coherent research pipeline, from problem identification to dataset construction and strategy design, ensuring accessibility for readers.\n5. Open-Source Reproducibility: By committing to release datasets (SuperRS-VQA/HighRS-VQA) and code, the research provides a standardized framework for the RS community. This fosters reproducibility, with 7B-parameter model training procedures detailed sufficiently for replication using 16 A100 GPUs.\n\n**Weaknesses:**\n1. Incomplete Experimental Validation：While supplementary materials include ablation studies, the main text lacks detailed descriptions of experimental procedures, especially insufficient ablation analyses that hinder the completeness of findings.\n2. Experiments on token compression ratios are insufficient, making it difficult to quantify their impact on overall performance.\n3. Lack of comparisons between models trained solely with high-resolution data versus those using both data and compression strategies, limiting clarity on strategy contributions.\n4. In certain scenarios (e.g., specific sub-tasks in Table 4), GeoLLaVA-8K underperforms baseline models like LLaVA-Next, suggesting potential negative impacts of token compression strategies in niche contexts.\n\nQUESTIONS:\n1. Could the authors supplement detailed ablation results in the main text to clarify the individual contributions of increasing high-resolution data, Background Token Pruning, and Anchored Token Selection strategies? For example, how does removing each strategy alone affect model accuracy on XLRS-Bench?\n2. Could the authors provide experimental data on different token compression ratios to demonstrate how compression ratios influence both computational efficiency and semantic retention? Are there optimal ratios for different RS tasks?\n3. Was a control experiment conducted to compare models trained solely with high-resolution datasets versus those using the token compression strategies? Please clarify whether performance improvements primarily derive from data resolution or strategy efficiency.\n4. Could the authors analyze specific failure cases where GeoLLaVA-8K underperforms baseline models (e.g., LLaVA-Next in certain sub-tasks) to identify whether token compression causes information loss in scenarios with dense small targets or complex backgrounds?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}