Title: GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Geolocalization

Image-to-GPS retrieval

Reranking

Large Multi-Modal Models
Abstract: Worldwide image geolocalization—the task of predicting GPS coordinates from images taken anywhere on Earth—poses a fundamental challenge due to the vast diversity in visual content across regions. While recent approaches adopt a two-stage pipeline of retrieving candidates and selecting the best match, they typically rely on simplistic similarity heuristics and point-wise supervision, failing to model spatial relationships among candidates. In this paper, we propose **GeoRanker**, a distance-aware ranking framework that leverages large vision-language models to jointly encode query–candidate interactions and predict geographic proximity. In addition, we introduce a *multi-order distance loss* that ranks both absolute and relative distances, enabling the model to reason over structured spatial relationships. To support this, we curate GeoRanking, the first dataset explicitly designed for geographic ranking tasks with multimodal candidate information. GeoRanker achieves state-of-the-art results on two well-established benchmarks (IM2GPS3K and YFCC4K), significantly outperforming current best methods. We also release our code, checkpoint, and dataset online for ease of reproduction.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes GeoRanker, a distance-aware reranking framework for the task of “worldwide image geolocalization”. The approach uses large vision-language models to encode the interactions between the query and candidates, and introduces a multi-order distance loss to enable the model to capture rich spatial structure. Moreover, the authors construct a new dataset, GeoRanking, which is tailored for spatial ranking tasks with multimodal annotations, including GPS coordinates, textual descriptions,  and image data. The experimental results demonstrate that GeoRanker achieves superior performance on two widely used benchmark datasets (IM2GPS3K and YFCC4K), surpassing existing methods with a clear margin.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Worldwide image geolocalization is an interesting and meaningful task, and the motivation is clearly stated.\n2.\tThis paper is well-written, easy to read and follow.\n3.\tThe proposed method is novel and the experimental results are solid, outperforming current SOTA methods on two well-established benchmarks (IM2GPS3K and YFCC4K).\n4.\tCode, checkpoint, and dataset are publicly available.\n\nWeaknesses:\n1. There is an obvious formatting problem in the paper: the paragraph between L126 and L127 lacks line numbers.\n2. The authors propose the GeoRanking dataset as a contribution. However, in Section 3.1 (i.e., GeoRanking Dataset Construction), there is no explicit description or summary of what the dataset consists of. Instead, most of the content is devoted to describing how to encode database samples and queries to complete retrieval candidates. Although there is some additional introduction about GeoRanking in subsequent chapters, the section title and the paragraph content don't match well. \n3. GWS15K [1] is a more challenging test dataset for worldwide image geolocalization. Some prior studies (e.g., PIGEOTTO [2], GeoCLIP [3], and GeoDecoder [1]) do not perform well on it. I would like to know how the proposed GeoRanker performs on the GWS15K dataset.\n4. The paper lacks analysis of failure examples, which is important for understanding the model's limitations and guiding future improvements. Including some qualitative analysis of incorrect predictions would strengthen the work.\n\nReference\n\n[1] Brandon Clark, Alec Kerrigan, Parth Parag Kulkarni, Vicente Vivanco Cepeda, and Mubarak Shah. Where we are and what we’re looking at: Query based worldwide image geo-localization using hierarchies and scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23182–23190, 2023.\n\n[2] Lukas Haas, Michal Skreta, Silas Alberti, and Chelsea Finn. Pigeon: Predicting image geolocations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12893–12902, 2024.\n\n[3] Vicente Vivanco Cepeda, Gaurav Kumar Nayak, and Mubarak Shah. Geoclip: Clip-inspired alignment between locations and images for effective worldwide geo-localization. Advances in Neural Information Processing Systems, 36:8690–8701, 2023.\n\nQUESTIONS:\n1. Between L126 and L127, the authors claim that “These adapters, along with the GPS encoder, are trained using an InfoNCE loss to align the query and candidate representations, as in G3.” I'm not sure what the “GPS encoder” here is.\n2. The authors use MP16-Pro as a database in constructing the GeoRanking dataset, but where is the query from? Could you describe in detail the composition of the GeoRanking dataset?\n3. In Section 3.4 (i.e., Inference), the equation (6) does not include negative samples (C_neg). However, in Figure 2, negative samples are involved in the inference process. Does there exist an expression error about equation (6)?\n4. According to the ablation study (i.e, Table 2), it seems that using GPT4V to generate some candidates even brings more improvement on IM2GPS3K  than the second-order distance loss. Have you tried using other LVLMs to generate candidates? How were the results?\n5. About Figure 4 (i.e., Time efficiency). I want to know for different numbers of candidates, where do they come from, only retrieved candidates or both retrieved and generated candidates? Does the time to generate candidates using GPT4V be included in the inference time?\n\nLIMITATIONS:\nLimitations are included in the appendix. GeoRanker introduces an additional ranking stage, resulting in substantial computational overhead during inference. The authors offer two possible approaches to alleviate this problem.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nGeoRanker is a novel distance-aware ranking framework that utilizes large vision-language models for worldwide image geolocalization, focusing on modeling spatial relationships among candidate locations. It introduces a multi-order distance loss to rank both absolute and relative distances, enabling better reasoning over structured spatial relationships.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* **Well-written and Easy to Understand:** The paper is clearly articulated and accessible.\n* **Extensive Ablations and Benchmarks:** The authors have provided thorough experimental validation.\n* **Significant Performance Improvement:** The paper demonstrates a notable leap in performance.\n* **Logical and Elegant Intuition:** The concept of encoding cross-similarity is intuitive and makes strong logical sense.\n\n## Weaknesses\n\n* **Missing Citations:**\n    * [1] GOMAA-Geo: GOal Modality Agnostic Active Geo-localization, Neurips 2024\n    * [2] OpenStreetView-5M, The Many Roads to Global Visual Geolocation, CVPR 2024\n    * [3] Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation, CVPR 2025\n    * [4] GaGA: Towards Interactive Global Geolocation Assistant, Arxiv 2024\n\n* **Dataset Contamination:** The use of Mp-16 is a concern due to known contamination with evaluation frameworks, which can unfairly benefit retrieval-based methods like G3 or GeoRanker. To address this, results on OSV-5M [2] would be more convincing, as it employs a 1km separation radius between test and train data, preventing data leakage. Additionally, OSV-5M is not based on YFCC, ensuring compatibility with other benchmarks without contamination.\n\n* **Unclear Benefit of VLM Usage:** The rationale for employing a Vision-Language Model (VLM) is not clear. It is questioned whether positional embeddings for GPS coordinates could achieve similar results, and if the extensive parameters of a VLM are truly necessary.\n\n* **Philosophical Concerns with Retrieval Methods:** There is a fundamental philosophical issue with retrieval-based methods for geolocalization. As the dataset grows, performance mechanically improves, potentially making the task trivial if global image coverage is achieved. This approach is seen as diminishing the inherent complexity and \"beauty\" of geolocalization, a task that ideally involves information extraction, analysis, critical thinking, and world knowledge, rather than mere retrieval.\n\nQUESTIONS:\nOne question i have, is could you ablate the number of samples in the retrieval pool?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces GeoRanker, a novel distance-aware ranking framework for worldwide image geolocalization. The task of predicting GPS coordinates from images globally is challenging due to the vast diversity in visual content across regions. Existing methods often rely on simplistic similarity heuristics and point-wise supervision, which fail to model spatial relationships among candidate locations.\n\nGeoRanker addresses these limitations by leveraging large vision-language models (LVLMs) to jointly encode query-candidate interactions and predict geographic proximity. It incorporates a multi-order distance loss that ranks both absolute and relative distances, enabling the model to reason over structured spatial relationships.\n\nThe contributions of this work are:\n\n-C1. GeoRanker Framework: Introduction of GeoRanker, a distance-aware ranking framework that models spatial relationships among candidate locations using a multi-order distance loss and large vision-language models.\n\n-C2. GeoRanking Dataset: Creation of GeoRanking, the first dataset tailored for spatial ranking tasks, featuring rich multimodal annotations including GPS coordinates, textual descriptions, and image data to facilitate future research.\n\n-C3. State-of-the-Art Performance: Achievement of state-of-the-art results on two well-established public geolocalization benchmarks (IM2GPS3K and YFCC4K), demonstrating substantial gains at fine-grained localization levels, and effectiveness shown through comprehensive ablations. The authors also provide their code, checkpoint, and dataset online for reproducibility\n\nSTRENGTHS AND WEAKNESSES:\n1. Strengths:\n\na.  A Rather Clever New Framework: The paper puts forward GeoRanker, a fresh and rather astute framework for worldwide image geolocalization that's commendably distance-aware. It quite smartly tackles the spatial relationships among potential locations, which is a significant step up from some of the more simplistic methods we've seen.\nb. Making Good Use of Modern Tech: It very wisely employs those large vision-language models (LVLMs). This means it's adept at understanding how queries and candidates interact, and rather accurately predicts geographic proximity – quite the modern approach.\nc. The Multi-Order Distance Loss is a Stroke of Genius: This particular loss function is a rather neat innovation. It allows the model to rank distances, both absolute and relative, which in turn means it can reason about structured spatial relationships with a good deal more sophistication, leading to improved accuracy.\n\n2. Weaknesses - The Sheer Scale of the Task: While GeoRanker is clearly a step forward, the business of worldwide image geolocalization is inherently a colossal challenge. The visual diversity across the globe is simply immense. Even with such an advanced model, truly capturing every local visual nuance remains a persistent hurdle. For GeoRanking, the first dataset for geographic ranking with multimodal information. Could you provide a bit more detail on the scale of this dataset – for instance, the number of images, unique locations, and the diversity of the textual descriptions?\n\nQUESTIONS:\n1. Could you elaborate a bit more on the specific architecture of the Large Vision-Language Models (LVLMs) you've employed within GeoRanker? Were these pre-trained, and if so, on what sort of data?\n2. The \"multi-order distance loss\" sounds rather interesting. Could you perhaps walk us through a more detailed example of how it operates, particularly in distinguishing between absolute and relative distances during training?\n3. You mention jointly encoding query-candidate interactions. What specific mechanisms or fusion techniques are used to achieve this effective joint encoding?\n4. How sensitive is the performance of GeoRanker to the choice of the underlying LVLM, or to the specific hyperparameters of the multi-order distance loss?\n\nLIMITATIONS:\nA Bit Resource-Intensive (One Might Infer): Given the reliance on those hefty Large Vision-Language Models, one might reasonably surmise that training and perhaps even running GeoRanker could demand a fair amount of computational horsepower. This isn't explicitly stated as a limitation in the paper, but it's often the case with such powerful models, which could be a bit of a sticky wicket for some researchers lacking the necessary computing muscle.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces GeoRanker, a framework for worldwide image geolocalization that refines the retrieve-and-rank paradigm. Instead of relying on simple similarity scores, GeoRanker employs a Large Vision-Language Model (LVLM) to learn a fine-grained, distance-aware scoring function that explicitly models the relationship between a query image and multiple candidates. The core of the method is a multi-order distance loss that supervises the model on both the absolute ranking of candidates and the relative distance gaps between them. To facilitate this, the authors created and have released GeoRanking, a large-scale, multimodal dataset for this task.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The paper convincingly argues that existing geolocalization methods fail to model the rich spatial relationships between candidates.\n\n2. GeoRanker achieves state-of-the-art performance on two standard benchmarks, IM2GPS3K and YFCC4K, outperforming 11 baselines, including the most recent SOTA methods. \n\n3. The paper introduces GeoRanking, a dataset specifically designed for learning to rank geographic entities. With 100,000 samples and 2 million query-candidate pairs containing rich multimodal information (images, GPS, text), this dataset is a valuable contribution to the community. Its public release will likely foster future research in geolocalization, multimodal learning, and information retrieval.\n\n**Weaknesses:**\n\n1. During inference, the proposed method integrates candidates generated by GPT-4V, a proprietary, expensive, and non-deterministic model. This dependency raises significant concerns regarding the reproducibility and accessibility of the results. \n\n2. The authors identify that GeoRanker introduces computational overhead compared to retrieval-only methods. While the efficiency analysis in Figure 4  shows GeoRanker is faster than a generic LVLM prompting baseline, this comparison is not sufficient. A direct comparison of inference latency and computational cost against the primary SOTA baseline, G3, is missing. Without this, it's hard to assess the practical trade-offs of the proposed method. \n\n3. Here are the design choices for the multi-order loss function that could benefit from deeper justification:\n\n* The choice to set the hyperparameter for the first-order loss to 1 is counterintuitive. This simplifies the Plackett-Luce loss to only supervising the ranking of the single best candidate, seemingly underutilizing the ranking formulation. The ablation study shows that performance degrades for $K^{(1)} > 1$. The accompanying explanation of a \"train-test mismatch\" is brief and requires more detail to be fully convincing.\n\n* The formula for calculating $K^{(2)}$, the number of pairs in the second-order loss, is complex and presented without an ablation study or strong intuition. It is unclear if this specific formulation is critical or if a simpler heuristic would suffice.\n\nQUESTIONS:\n1. The method's reliance on GPT-4V for candidate generation is a central concern for reproducibility. Could you provide a higher granular breakdown of performance? Specifically, what is the accuracy of GeoRanker when using (a) only the top-k retrieved candidates, (b) only the candidates generated by GPT-4V, and (c) the combined set? This would clarify the exact contribution of each candidate source. Furthermore, to improve accessibility, have you considered using a powerful open-source LVLM (like the Qwen2-VL-7B you use as a backbone) for the candidate generation step as well? An analysis of this trade-off would be very valuable.\n\n\n2. The Prompting baseline is an important point of comparison for ranking ability and efficiency. Could you clarify its implementation? Was this a zero-shot evaluation where the LVLM was asked to choose the best location from a list in a single pass? How does this setup compare to the RAG pipelines used in other baselines like Img2Loc and G3? Clarifying this would help situate GeoRanker's contribution more precisely within the landscape of LVLM-based geolocalization methods.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA model for geolocalization based on G3 is presented. Authors hypothesize that previous models fail to correctly rank the list of candidates even though the initial set of candidates might contain correct predictions. To address this the authors preprocess the MP16-Pro dataset including multiple candidates per query and train a model with a novel two order loss that acknowledges the spatial relation between candidates. They perform experiments on standard datasets obtaining state-of-the-art results and improving over the G3 baseline.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThis paper addresses a very interesting idea that has not been widely studied before: the ranking of visual geolocalization systems. As the authors describe, the top-k candidates retrieved by G3 contain better candidates than the final answer of the model. This suggests that explicitly addressing the ranking can bring benefits to the model.\n\nThe multi-order loss seems like a neat way of addressing the issue. Besides it’s relevant to bring the  Plackett-Luce loss into the visual geolocalization field for future research.\n\nThe experimentation shows that the model obtain the best performance to date, improving over the G3 baseline, even though it uses a very similar approach. Ablative experiments show the benefits of the different components of the method.\n\nWeaknesses:\n\nThe pipeline is very similar to the work of G3. Authors include a novel loss and a preprocessing of the dataset to accommodate such loss, but most of the architecture remains very similar. The claim of curating a new dataset should be heavily downplayed, as this is just a preprocessing of an existing dataset. To alleviate the problem of the pipeline being similar to G3, I recommend the authors to explicitly explain G3 pipeline in the text and to highlight the differences.\n\nThere is a big focus in this paper on how it addresses the ranking of candidates. However the metrics from the experiments, although they show improvements in global geolocalization, do not measure ranking effectiveness. The only experiment that addresses the ranking is the 4.8 Case Study, but this is just qualitatively. I invite the authors to include quantitative experiments with metrics from the “learning to rank” literature to showcase the improvements in ranking capabilities.\n\nQUESTIONS:\n- Can the authors give a detailed summary of the similarities and differences with G3 to more easily assess the novelty of the paper?\n- Could the authors include some metrics, or quantitative experiments that showcases the improve in ranking performance? This could strength the paper and the ranking issue formulation.\n- This comment is not specific to this paper but rather to the field. The field is very related to Visual Place Recognition, however I see that the two field have developed their own literature with little overlap. I wonder if some of the advances of Visual Place Recognition to create robust and discriminative embedding could be applied to this field to improve the candidate retrieval phase. In the other way around, this field could also help in Visual Place Recognition. Could the authors provide some insights regarding this?\n\nLIMITATIONS:\nThis kind of works can have a societal impact regarding privacy violations, as models can retrieve the close location of images. Yet the authors do not discuss this.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-21"}