Title: LocDiff: Identifying Locations on Earth by Diffusing in the Hilbert Space
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image geo-localization

location encoding

diffusion
Abstract: Image geolocalization is a fundamental yet challenging task, aiming at inferring the geolocation on Earth where an image is taken. State-of-the-art methods employ either grid-based classification or gallery-based image-location retrieval, whose spatial generalizability significantly suffers if the spatial distribution of test images does not align with the choices of grids and galleries. Recently emerging generative approaches, while getting rid of grids and galleries, use raw geographical coordinates and suffer quality losses due to their lack of multi-scale information. To address these limitations, we propose a multi-scale latent diffusion model called LocDiff for image geolocalization. We developed a novel positional encoding-decoding framework called Spherical Harmonics Dirac Delta (SHDD) Representations, which encodes points on a spherical surface (e.g., geolocations on Earth) into a Hilbert space of Spherical Harmonics coefficients and decodes points (geolocations) by mode-seeking on spherical probability distributions. We also propose a novel SirenNet-based architecture (CS-UNet) to learn an image-based conditional backward process in the latent SHDD space by minimizing a latent KL-divergence loss. To the best of our knowledge, LocDiff is the first image geolocalization model that performs latent diffusion in a multi-scale location encoding space and generates geolocations under the guidance of images. Experimental results show that LocDiff can outperform all state-of-the-art grid-based, retrieval-based, and diffusion-based baselines across 5 challenging global-scale image geolocalization datasets, and demonstrates significantly stronger generalizability to unseen geolocations.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a novel framework for image geolocalization, called LocDiff, which aims to infer the geolocation where an image was captured. LocDiff addresses the lack of multiscale information in previous approaches by leveraging latent diffusion modeling in the Hilbert space of Spherical Harmonic coefficients. This representation is both dense and easier to linearly decode back into coordinate space. The framework introduces a new architecture for the diffusion process that combines a conditional SirenNet, conditioned on the image and timestamp, with a UNet. The authors compare then LocDiff to existing geolocalization methods and show that it generally outperforms them, particularly when paired with a retrieval-based approach such as GeoClip.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper is particularly well written, with the method described in meticulous detail.\n- The approach is sound and well justified, building on components proven effective for modeling our spherical Earth.\n- When paired with GeoCLIP, the approach outperforms baselines across multiple scales.\n- Numerous ablation studies in the Appendix demonstrate the effectiveness of the modules of LocDiff.\n\nWeaknesses:\n- Both spherical harmonics and diffusion models are known to be computationally intensive at inference time. While the authors provide an insightful analysis of computational complexity during training in Appendix A.7, it would be valuable to include results on inference time, ideally with a comparison to other methods.\n- LocDiff appears to rely on GeoCLIP (or more generally, on a retrieval-based component) to outperform other baselines on the Im2GPS3k and YFCC-26k datasets, especially at fine-grained scales. This is not a big issue *per se,* but I think the authors should acknowledge this dependency more explicitly, ideally in the introduction and possibly in the abstract, to provide a fairer representation of the method's standalone capabilities.\n\nQUESTIONS:\nI am open to increase my rating if the authors address the concerns previously raised:\n\n1. Provide a comparison of inference time of LocDiff against the baselines.\n2. Clarify the extent to which LocDiff depends on GeoCLIP when claiming superior performance over other approaches in the introduction and abstract.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of determining where a photo was taken. It introduces LocDiff, leveraging diffusion models to enhance geolocation accuracy and scalability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nQuality:\n\nThe paper proposed a novel encoding framework called Spherical Harmonics Dirac Delta (SHDD) with clear derivations and mathematical formulations. It also propose a novel SirenNet-based architecture (CS-UNet) to learn an image-based conditional backward process in the latent SHDD space by minimizing a latent KL-divergence loss.\n\nClarity:\n\nThe paper is generally well-written and logically structured. Key contributions, such as SHDD encoding and the hybrid LocDiff-H approach, are clearly explained.\n\nSignificance:\n\nThe introduction of the hybrid approach (LocDiff-H) is significant as it combines generative and retrieval-based methods, potentially paving the way for more robust geolocation systems.\n\nOriginality:\n\nThe introduction of SHDD KL-divergence as a loss function is innovative and aligns well with the problem's requirements.\n\nWeakness:\n\nQuality:\n\nSome hyperparameters (e.g., degree $L$) are chosen arbitrarily without a clear justification beyond empirical testing. A more systematic approach to selecting these parameters would strengthen the paper.\n\nClarity:\n\nSome sections, particularly those involving mathematical derivations (e.g., spherical harmonics computation), could be more accessible to a broader audience. Simplifying or visually representing these concepts might aid comprehension.\n\nSignificance:\n\nWhile the paper demonstrates strong empirical results, its societal impact is limited. The authors acknowledge this by stating that their work does not have direct societal implications. However, discussing potential future applications could highlight the significance of their contribution.\n\nOriginality:\n\nWhile the paper builds on existing works (e.g., Sphere2Vec), some of its extensions, such as the hybrid approach, are not thoroughly explored. A more detailed discussion of these contributions' uniqueness compared to prior work would enhance originality.\n\nQUESTIONS:\n1. What is the rationale behind selecting L=47 as the maximum degree for spherical harmonics? Did you explore higher degrees beyond L=47, and if so, what were the results? How does this choice balance spatial resolution with computational feasibility?\n\n2. The paper uses the MP16 dataset for training, which may introduce a bias towards certain geographic regions or image types. How does this affect the generalizability of your model? Have you tested its performance on geographically diverse datasets?\n\n3. The SHDD encoding dimension grows quadratically with L, leading to high computational costs. How does this scale in practice? Have you evaluated the trade-offs between spatial resolution and computational efficiency for larger values of L?\n\n4. The hybrid approach (LocDiff-H) relies on a gallery of candidate locations for retrieval. How sensitive is the model's performance to the choice and distribution of these gallery points? What happens if the gallery does not contain locations similar to those in the test set?\n\n5. The ablation studies focus on different encoders and losses but do not compare against other state-of-the-art methods like PIGEON. How does your method perform in comparison to these models?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a generative approach for global-scale image geolocalization, which aims to infer an image's geographic location on the Earth's surface. The authors introduce a novel location encoding-decoding strategy, termed Spherical Harmonics Dirac Delta (SHDD) representations, coupled with a UNet-based diffusion framework. This design addresses key limitations in previous methods, including distributional misalignment and insufficient multi-scale context modeling. Extensive experiments across five global-scale geolocalization datasets demonstrate the effectiveness of the proposed approach. Moreover, the method exhibits strong generalization to regions that were not seen during training.\n\nSTRENGTHS AND WEAKNESSES:\nPros:\n- The motivation is clear and convincing. Representing the embedding space with Spherical Harmonics is a natural solution for tasks involving the global mapping on the Earth surface. Moreover, the problems/challenges of using the Spherical Harmonics for diffusion models are clearly identified in this paper.\n- This paper proposes a novel pipeline for image geo-localization combining the Spherical Harmonics Dirac Delta (SHDD) Representations and a latent diffusion model. Although the spherical harmonics has been explored in some geo-based tasks like the general location embedding design and geo-spatial foundation models.It is interesting to see its adaptation into the image geo-localization task.\n- Stronger performance. Experiments on 5 datasets demonstrate the stronger performance of the proposed method. Moreover, it is interesting to see that the SHDD significantly increases the generalization ability.\n\nCons:\n- My primary concern is the potential computational overhead introduced by both the spherical harmonics basis and the diffusion process. These components are likely to significantly increase the model's complexity and training cost. However, the main paper lacks a comparison of computational efficiency metrics (e.g., GFLOPs, model size, training/inference time) between the proposed method and baseline methods.\n- Given the paper’s focus on location representations, a more comprehensive comparison with alternative methods is expected. Although two methods (RBF and Sphere2Vec) are briefly compared in Appendix Table 6, the performances of other representations mentioned in Section A.2 are not reported. Furthermore, it seems that RBF and Sphere2Vec appear to collapse in performance, it is not clear if they are implemented properly. \n- The introduction of the Dirac delta function is a crucial step in adapting spherical harmonics to represent specific Earth locations. A clearer explanation of how similarity is computed (e.g., if there is any closed-form expression?), would greatly enhance the reader’s understanding of the proposed mechanism. \n- Finally, the method section is overall not easy to follow, especially considering that this paper focuses on the image geo-localization, while it is sometimes hard to connect the variables and notions in the SHDD modeling with the image geo-localization task. For example, in line 214, what do the spherical function F and \\delta denote, an image or the location embedding? and in line 216, what is p(\\theta, \\phi) and q_e respectively in the context of geo-localization?\n\nMinor issue: \n- The metrics used in Tables 1 and 2 are not defined. Are they top-1 accuracy (ACC), or some other measure? A brief description in the captions or main text would be helpful.\n\nQUESTIONS:\nSee comments\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nLocDiff is a multi-scale latent diffusion model for image geolocalization, addressing limitations of existing methods that struggle with spatial generalizability and multi-scale information. The model introduces a novel Spherical Harmonics Dirac Delta (SHDD) Representation framework for encoding and decoding geolocations, which tackles issues like sparsity and non-linearity in the encoding space. A Conditional Siren-UNet (CS-UNet) architecture is proposed to learn the image-based conditional backward diffusion process within this SHDD space.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* **Well-motivated from a theoretical point of view:** The paper presents a strong theoretical foundation for its approach.\n* **Strong experiments:** The experimental setup is robust, and the **generalization experiments are particularly appreciated**, demonstrating the model's performance beyond training distributions.\n\n## Weaknesses\n\n* **Prediction Averaging Strategy:** The current approach of **averaging samples for prediction might lead to incorrect predictions, especially in the presence of multiple modes**. I suggest the authors explore a strategy similar to [1] by leveraging guidance to collapse the predictions, which could also lead to faster sampling as only a single point would be needed instead of 16.\n* **Benchmark Contamination and Data Source:** As highlighted by the authors, MP16 suffers from contamination issues with the benchmarks. It would be beneficial to **train on OSV-5M, which guarantees a 1km border to test samples**, making comparisons to [1] more comparable. Furthermore, the different data source (not YFCC) makes all benchmarks inherently different from the training set.\n* **Proof of SHDD Space Efficacy:** The main claim that the **SHDD space is responsible for improved fine-grain geolocalization is not sufficiently proven**. The comparison is primarily against [1], which has entirely different design choices, training steps, and datasets. **Ablation studies are crucial here**, perhaps training SHDD space diffusion with [1]'s training setup, and vice-versa, training [1]'s architecture on LocDiff's setup.\n* **Evaluation of Generative Aspect:** The paper lacks an evaluation of the model's generative aspect. **[1] proposes specific benchmarks for probabilistic geolocalization that should be utilized**. It's important to check whether the model achieves better probabilistic modeling or if it simply collapses to improve geolocalization performance.\n* **Lack of Samples:** The paper would greatly benefit from the inclusion of **visual samples** to demonstrate the model's performance qualitatively. Could make the paper more \"fun\" to read as well.\n* **Inference Efficiency Claim:** The claim of being more inference efficient than [1] due to the Riemannian framework is questionable. The authors sample with 200 steps, whereas [1] achieves maximum performance with 16 steps. Additionally, going from 3 dimensions to 4096 dimensions should drastically impact sampling (and training) time. This claim seems incorrect. **A proper benchmark for inference and training time would be highly valuable** to substantiate this claim.\n\nI put weak accept for now as i think this paper would benefit from adressing the previous concerns, but i find this paper to be good and will likely increase my rating depending of the authors response.\n\n[1] Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation, Dufour et al, CVPR 2025\n\nQUESTIONS:\nI suggest the authors add some pseudo-code to help with implementation + understanding as the very heavy math approach can be hard to parse for a part of the community.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}