When Large Vision-Language Model Meets Large Remote Sensing Imagery:
  Coarse-to-Fine Text-Guided Token Pruning
Authors: name: Junwei Luo, name: Yingying Zhang, name: Xue Yang, name: Kang Wu, name: Qi Zhu, name: Lei Liang, name: Jingdong Chen, name: Yansheng Li

Efficient vision-language understanding of large Remote Sensing Images (RSIs)
is meaningful but challenging. Current Large Vision-Language Models (LVLMs)
typically employ limited pre-defined grids to process images, leading to
information loss when handling gigapixel RSIs. Conversely, using unlimited
grids significantly increases computational costs. To preserve image details
while reducing computational complexity, we propose a text-guided token pruning
method with Dynamic Image Pyramid (DIP) integration. Our method introduces: (i)
a Region Focus Module (RFM) that leverages text-aware region localization
capability to identify critical vision tokens, and (ii) a coarse-to-fine image
tile selection and vision token pruning strategy based on DIP, which is guided
by RFM outputs and avoids directly processing the entire large imagery.
Additionally, existing benchmarks for evaluating LVLMs' perception ability on
large RSI suffer from limited question diversity and constrained image sizes.
We construct a new benchmark named LRS-VQA, which contains 7,333 QA pairs
across 8 categories, with image length up to 27,328 pixels. Our method
outperforms existing high-resolution strategies on four datasets using the same
data. Moreover, compared to existing token reduction methods, our approach
demonstrates higher efficiency under high-resolution settings. Dataset and code
are in https://github.com/VisionXLab/LRS-VQA.