LocDiff: Identifying Locations on Earth by Diffusing in the Hilbert Space
Authors: Zhangyu Wang
Zeping Liu
Jielu Zhang
Zhongliang Zhou
Qian Cao
Nemin Wu
Lan Mu
Yang Song
Yiqun Xie
Ni Lao
Gengchen Mai
Venue: NeurIPS 2025 poster ()
Keywords: Image geo-localization
location encoding
diffusion
Primary area: deep_learning

Abstract: Image geolocalization is a fundamental yet challenging task, aiming at inferring the geolocation on Earth where an image is taken. State-of-the-art methods employ either grid-based classification or gallery-based image-location retrieval, whose spatial generalizability significantly suffers if the spatial distribution of test images does not align with the choices of grids and galleries. Recently emerging generative approaches, while getting rid of grids and galleries, use raw geographical coordinates and suffer quality losses due to their lack of multi-scale information. To address these limitations, we propose a multi-scale latent diffusion model called LocDiff for image geolocalization. We developed a novel positional encoding-decoding framework called Spherical Harmonics Dirac Delta (SHDD) Representations, which encodes points on a spherical surface (e.g., geolocations on Earth) into a Hilbert space of Spherical Harmonics coefficients and decodes points (geolocations) by mode-seeking on spherical probability distributions. We also propose a novel SirenNet-based architecture (CS-UNet) to learn an image-based conditional backward process in the latent SHDD space by minimizing a latent KL-divergence loss. To the best of our knowledge, LocDiff is the first image geolocalization model that performs latent diffusion in a multi-scale location encoding space and generates geolocations under the guidance of images. Experimental results show that LocDiff can outperform all state-of-the-art grid-based, retrieval-based, and diffusion-based baselines across 5 challenging global-scale image geolocalization datasets, and demonstrates significantly stronger generalizability to unseen geolocations.

LocDiff: Identifying Locations on Earth by Diffusing in the Hilbert Space
Zhangyu Wang Zeping Liu', Jielu Zhang 7.3 Nemin Wu², Lan Mu Yang Song", Yiqun Xie
Zhongliang Zhou², Qian Cao²; Ni Lao
Gengehen Mai
University of Maine, SEAI Lab, University of Texas at Austin, University of Georgia, *University of Maryland, Google LLC, *Open AI, "Harvard University CCoresppondig author.
Abstract
Image geolocalization is fundamental yet challenging task, aiming at inferring the geolocation on Earth where an image taken. State-of-the-arr methods employ either grid- bl-ed classification spatial generalizability significantly suffers the spatial distribution of test im- ages does not align with the choices of grids and galleries. Recently emerging generative approaches, while getting rid of grids and galleries, use raw geographical coordinates and suffer quality losses due to their lack of multi-scale information. To address these limitations, we propose multi-scale latent diffusion model called LoeDiff for image geolocalization. We developed novel positional encoding- decoding framework called Spherical Harmonics Dirae Delta (SHDD) Represen- tations, which encodes points spherical surface (c.g.. geolocations into Hilbert space of Spherical Harmonics coefficients and decodes points (geolo- cations) by mode-seekin architecture (CS-UNet) to learn an image-based conditional
gallery-based based image-location retrieval, whose
Earth)
on spherical probability distributions. We also propose
novel SirenNet- backward process in the latent SHDD space by minimizing latent KL-divergence loss. To the best of our knowledge. LocDiff is the first image geolocalization model that performs latent diffusion in multi-scale location encoding space and generates geolocations under the guidance of images. Experimental results show that LocDiff can outperform all state-of- f-ttheeertttt grid grid-based, retrieval-buae bassed.edddd......eeaeaaeee and diffusion- based baselines across 5 challenging global-scale image geolocalization datasets, and demonstrates significantly stronger generalizability to unseen geolocations.
Introduction
Location decoding, i.e., predicting geolocations from given context, remains challenging and rarely studied problem which has potential applications in numerous tasks including trajectory synthesis (35), building footprint segmentation [[4], and the widely studied image geolocalization
tasks [38 45 As one of the representative tasks for location decoding, image geolocalization
aims at predicting locations on Earth based on a given image, which potentially ranges from many types such as wild life photos, street views, and remote sensing images. However, unlike image classification, solutions to image geolocalization are less mature because their ground-trumt -tuths are locations represented by real-valued coordinates the spherical surface. While regression models are commonly used to predict real-valued labels, they are proven to be tricky to train and perform especially poorly on the global-scale image geolocalization problem due to the highly complex and non-linear mapping between the image space and the geospatial space [46] 17 As an alternative transductive solution, researchers employ pre-defined geographical classes (e.g. divide Earth into
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

highly non-linear
Coordinate
(1)
Position Encoding Space
(2)
Location
Space
Embedding Space
PE
NN
Position
Location
Encoder
Encoder
(DFS, Spherical Harmonics)
sparse
(Sphere2Vec)
(4a)
dense
Encoding Space
SHDD distance measure is an upper bound W2
SHDD Encader
D Peelllllo (0a en)
SHDD
Decoder
(4b) dense
Figure 1: Multi-scale latent diffusion for image geolocalization. Black solid/dotted arrows denote encod- ing/decoding steps. Orange modules are learnable, while blue modules are deterministic with no learning parameters. (1) It is difficult to diffuse in the position encoding space (27] because valid positional encodings are sparse which leads to difficulties in diffusion model training and decoding. (2) The locational embedding
space is dense and can perform diffusion processes, but the non-linear mapping between the position encoding and location embedding space makes decoding back correct coordinates extremely difficult. Minimizing distances in the location embedding space may not minimize geographic distance. (3) The SHDD encoding space is dense every point e in this encoding space corresponds to a spherical function F.. whose difference from the spherical Dirac delta function ((0)-40) (0) of the ground truth location (Do. éo) is measured by the reverse KL-divergence E. (4) The SHDD decoding addresses the non-linearity problem. The heatmaps (4a), (4b) represents the distance from the spherical point represented by the cmbeading/encoding to the yellow star point in the middle. The distance measured by SHDD is significantly smoother.
disjoint or hierarchical grid cells) 46 38 or geo-tagged image galleries (c.g. a set of reference geotagged images) 45] map the real-valued ground-trutt coordinates to discrete labels (e.g. the ID of the grid cell or the ID of the reference image in the gallery), subsequently transforming the image geolocalization problem into a special case of image classification or image- seageeiggg/iaag-lleeeeee retrieval task. For example, [L]kNN Ao), CPlaNet (38). and PIGEON [I0] partition the Earth's 'ss surface into non-overlapping cells and convert the image geolocalization problem into an image classification problem. GeoCL.IP 45 uses contrastive learning framework to align pretrained image embeddings with geographical location embeddings in the gallery. However, the spatial resolution of these approaches is constrained by the size of the cells or the availability of gallery images/locations.
Generative models such as diffusion models
score-based generative models 42), and
flow matching models (20,, have demonstrated great capacity directly generating continuous outputs such as images and modeling their complex distributions. They are commonly applied to points in Euclidean spaces 42
or the geometric structures defined in Euclidean spaces 449. This motivates us to develop diffusion-basee based image geolocalization methods that decode locations on the spherical surface with a finer spatial resolution by using multi-scale location representations(22 (24 26 37) without dependence on predefined grid cells galleries [46] 38 45 10]
However, simply performing diffusion or flow matching the original coordinate space faces two major challenges. First, geographical locations reside an embedded Riemannian manifold instead of an Euclidean space" This makes noise adding/removal problematic. If we use coordinates on the manifold such as longitudes and latitudes, the noise adding/removal will lead to projection distortions, especially near the polar areas. If we use coordinates in the 3-D Euclidean space, every borward/backwart step will likely output points that not the spherical surface [8], which requires noise adding/removal in the tangent space and re- re-projection back to the spherical surface, such in Diff 3 [8] and FM R [8) The re-projection operation introduces extra computational costs. Secondly, and more importantly, raw coordinates lack multi-scale information which is necessary for modeling complex spatial distributions (27 37 This practice can yield suboptimal
'Geographical locations are distributed on 2- 2-dimensional Riemannian manifold (i.e., the sphere surface) embedded in the 3-D Euclidean space.

---

geolocalization results. As later shown in Table compared with generative models operating directly on raw coordinates (e.g.. Diff R FM R and RFM S (.) our multi-scale latent location diffusion model LocDiff demonstrates superior performance on 2 image geolocalization datasets across all evaluation metries measured in various spatial scales.
In order to achieve multi-scale modeling power for complex distributions over space, neural location 18 37
encoding methods [25] 23 27 26
48 50] commonly adopt multi-scale position encoding with deterministic transformations followed by a learnable location embedding layer. However, despite their wide applicability, neither the position encoding nor the location embedding space is suitable for developing a location diffusion model due to sparsity problem during diffusion and non-linearity problem during decoding as illustrated in Figure [1) and (2). Therefore, we hypothesize that
The ideal space to develop latent diffusion models for spherical location generation should be both dense and easy to find projections back to the coordinate space.
Motivated by this, we propose a novel spherical position encoding method called Spherical Harmonics Dirac Delta (SHDD) Representation. Figure 3) and (4) illustrates how method addresses the sparsity problem by encoding a spherical point (fo, ④o) as a spherical Dirac delta function (0) In the SHDD encoding space, every point uniquely corresponds seen as an approximation of a spherical Dirac delta function. The level of noise & can be continuously measured by the reverse KL-divergence between Fe and 0(60.00) Then the latent diffusion in the SHDD encoding space equals gradually adding noise to the ground-truth (00.0o) (forward process) and finding a sequence of F. that gradually reduces & (backward process). During decoding, the learning-free SHDD Decoder evaluates the corresponding spherical function Fe and decodes it as the spherical point whose corresponding spherical Dirac delta function minimizes E. Figure ((b) demonstrates that our SHDD encoding space shows less decoding non-linean non-linearity than existing location representation learning methods such as Sphere2 [27] and SH (37]. Therefore, diffusion in the SHDD encoding space will be more stable and easier to converge.
spherical function F. and can be
Equipped with the Hilbert (i.e. decoder, we can now perform conventional latent diffusion for location generation. We propose a novel Serennet- architecture called Conditional Siren-UNet (CS-UNet) to learn the conditional backward diffusion process, i.e, to generate spherical points from random Gaussian noise given conditions such as images and texts. We call the integrated framework, including SHDD encoding, CS US-UNet latent diffusion, and SHDD decoding. the LocDiff model. Results show that LocDiff outperforms all state-of-the-ar models on 5 global-scale image geolocalization datasets large scales (250 km, 750 km, 2500 km), and by combining retrieval-basee models (See Appendix A).6). the hybrid variant LocDiff-H achieves superior performance at small scales (1 km, 25 km). LocDiff also outperforms the recently proposed generative geolocalization model RFM S [8) which shows the advantages of our latent diffusion approach over generation in the original coordinate space. Moreover, we demonstrate that LocDiff is more spatially generalizable to unseen locations than retrieval-based models such as GeoCLIP 45).
Euclidean) SHDD encoding space and the SHDD
2 Related Work
Geolocalization by classification and retrieval. Traditional geolocalization methods typically employ either classification approach or an image retrieval approach. The former divides the Earth's
surface into non-overlapping or hierarchical cells and classifies images accordingly [31 46 30
I0]
while the latter approach identifies the location of a given image by matching it with a database of
Image-location pairs [39, 52 51 45). Using fewer cells results in lower location prediction accuracy
while using smaller cells reduces the number of training examples per class and risks overfitting [38].
On the other hand, retrieval- based systems usually suffer from poor search quality and inadequate coverage of the global geographic landscape.
Geolocalization by diffusion and flow matching in the original coordinate space. Dufour et al 8 recently proposed a set of generative image geolocalization models on the original coordinate space, including Diff TR33 FM RR3, and RFM Sz. The first two models perform diffusion and flow matching on the 3D Euclidean space respectively and project the output back on the sphere. RFM S2, instead, performs flow matching on the spherical Riemannian manifold by projecting back and forth
3

---

to the tangent space of each location. However, operating on the original coordinate space cannot capture the rich multi-scale geographic information required for modeling complex distributions on the spherical surface. Our LocDiff overcomes this limitation by performing diffusion in a latent space which captures rich multi-scale location information.
Riemannian generative models. Conventional generative models cannot function well in the spherical coordinate space (e.g., 3D coordinates representing points on a sphere) due to the projection distortion and sparsity issue. There are two common strategies to address this problem. The first strategy is to project point on the sphere to its tangent space (an Euclidean space), add/remove noise in the tangent space, and then spherical re-project the noised/denoised point in the tangent space back to the surface (36) like RFM S2 [8] does. The second strategy is to derive formulas for direct Riemannian diffusion [5]. The main drawback of both strategies is their computational complexity. In the first strategy, each projection operation takes time, making sampling acceleration based on DDIM [4]] impossible, because the projections are accurate only when the diffusion steps adequately small. For the second strategy, the Riemannian diffusion formulation is much more complicated than the Fuclidean version. The model architectures, training tricks, and other useful techniques developed for conventional diffusion models not be easily transferred.
Location Embedding. The distinction between positional encoding and location embedding lies in semantics: the positional encoding is only a task-agnostic transformation of the coordinates x, but the location embedding carries task-specific information. For example, it contain information about spatial distributions of species trained on geo-aware species fine-grained recognition tasks
7 50 21) Some prior work, e.g., NeRF (28), utilized positional encoding to represent location information. This task-agnostic method focuses on capturing the position or order of elements within a sequence. In contrast, many location encoders are specifically designed to capture context-aware location information 24 Please refer to Appendix A.2 and (2) for more
[22 27 26 48
details.
3 Preliminaries
Real Basis of Spherical Harmonics Let = ()) be location on the spherical surface using angular coordinates where O E [0, T) and [0, 2T). For any function F(0, on the sphere, there exists a unique infinite-dimensionall real-valued vector of coefficients {Cim} (we may call it Φ) where I is called degree and is called order and Y.. (0, Φ) is the real basis of spherical harmonics at degree I and order m. The detailed computation of Yim can be found in Appendix A.. In this way, any function on the sphere can be uniquely represented by its coefficient vector.
coefficient vector) such that v(0, 6), F(0,
Ei= -im--CCimYin
A.33
Spherical Dirac Delta Function Similar the Dirac delta function in Euclidian space we can define a spherical Dirac delta function is a probability density function over the spherical surface
whose mass all concentrates on one point: )(,,,o)) (0, = 0o = =0: o = du It can uniquely
otherwise
represent any point (0;, Φ;) on the sphere by mapping it to (0,, Representing point function allows us use spherical harmonics represent points on the spherical surface.
4 LocDiff Framework
In this section, we will introduce the theory and techniques we employ in our LocDiff model that enable spherical location generation via latent diffusion. Our aim is to find position encoding space that does not suffer from the sparsity problem and the non- non-liiea problem so we can efficiently perform latent diffusion. We first analyze what properties we need achieve this and propose the Spherical Harmonics Dirac Delta (SHDD) Encoding- -eeloinng framework accordingly. Then we prove that SHDD satisfies all the desired properties. Following that, we propose the Conditional Siren-UNet (CS-UNet) architecture to learn the conditional backward process for latent diffusion. We also develop computational techniques based on the properties of SHDD representation so that the training and inference of LocDiff are efficient.

---

4.1 Problem Setup and Intuitions
As we have outlined the introduction, our goal is to find a position encoding method that encodes the spherical surface into a dense subset of R (ideally the entire RR') and accurately decodes points back spherical coordinates. There are several mathematical properties such position encoding and decoding method should have. For rigorous discussions, we give definitions of the aforementioned properties and demonstrate how they guide the finding of our SHDD encoding-decodingg
framework.
Definition 4.1 (Coordinate Space), A Coordinate Space C can be any space with a parametrization, such Euclidean space with the Descartes coordinate system. In this paper, C always refers the unit sphere surface embedded in R with the conventional angular coordinate system (0, Φ).
Definition 4.2 (Position Encoding and Position Decoding). A Position Encoder IPE C - IR is an injective function, usually d > 3. SpE. = PE(C) C I is called the Position Encoding Space. A Position Decoder PD, IRd + C is surjective function.
The sparsity problem: Since are projecting set of 2-dimensional points in C' into high- dimensional Euclidean space Spz, dense filling is impossible.
However, if we define difference measure & R x R + R, then Seg can be partitioned by the following equivalence relation:
e~e + min E(e, s) arg min E(e's)
(I)
#ES
SEST
that is, we can assign every points E Spe to the nearest positional encoding (consequently, spherical
point) in terms of &. We say the E-equivalence classes densely fill R. Further, learning-free decoder exists
PDe(c) := E PE(p)) = arg min E(c, PE(())
(2)
PEC
If E is continuous, i.e.
Vs E Spe, (c s) (E(c,s)
(3)
then the sparsity problem is resolved. Since now diffusion in StE equals random walk among spherical points and a small perturbation in Spg will not result in an abrupt jump on C.
The non-linearity problem: Since the diffusion model has intrinsic randomness, it is possible that the generated e corresponds to a wrong S. If the mapping between s and its corresponding spherical point p p (s) is highly non-linear (e.g.. in the location embedding space), the decoder PIDg will then be very unstable (see Figure []. Thus, we hope that for a large tolerance 17 >> 0 and small shift A > O, the following property holds for our decoder PDg:
(4) where de is the distance in the spherical coordinate space (e.g., the great circle distance). If this property is satisfied, the non-linearity problem is resolved.
Vs E Seg, E) < 17 de (P) < A
It is not an easy task to find such E, especially considering computational constraints (e.g., it is impossible to exactly evaluate the arg min function in Equation [ Fortunately, we find that by treating spherical points as special spherical functions and representing them using Spherical Harmonics coefficients, we can define & as spherical KI.-divergence which satisfies all the desirable properties mentioned above, thus addressing the sparsity and the non-linearity problems as whole. Moreover, the choice of Spherical Harmonics coefficients also enables efficient computation.
4.2 Spherical Harmonics Dirac Delta (SHDD) Encoding
As discussed in Section B we can represent spherical points as spherical Dirac delta functions, and a spherical Dirac delta function can be encoded as an infinitediimensional real-valued coefficient vector, i.c. a point in a Hilbert space: PESHDD (0o, Φo) : U Un==1 [Cim], where U denotes vector concatenation. Thus, the spherical harmonics coefficient vector be used to uniquely represent a point (fu, Φu) on the sphere. In practice, we truncate the coefficient vector up to its leading (L + 1)² dimensions, where L is the maximum degree of associate Legendre polynomials:
PE (fo, Φu)
[Ctm]
(5)
[= 0 m==1
5

---

We call this vector the (L+ 1) d-ddere Spherical Harmonics Dirac Delta (SHDD) Representation of (0g. do) and PESHDD the SHDD encoder. Each SHDD representation corresponds to an approxi- mation of the true spherical Dirac delta function ((), whose probability density concentrates in region surrounding (Oo, ④o) rather than a single point, enabling differentiable comparison between Dirac delta functions. The Legendre polynomials have finer granularity as their degree L increases, which makes SHDD representations, like other frequency- based location encoding methods such as Sphere2Vec [27] capable of capturing multi-scale spatial information.
Since for spherical Dirac delta functions, the coefficients of the Legendre polynomials are the values of the Legendre polynomials (fo: (o)) 22 i.e.,
Ctm Yim (0o, Φ0), Vl, m,
(6)
the encoding procedure can be reduced to simple look-up of Yi values.
4.3 SHDD KL--Dveegencce
Before describing decoding algorithm need measure the difference between any (L + 1)-
degree SHDD representation of a spherical Dirac delta function 6, and an arbitrary (L++)) vector corresponds to certain spherical function F. We propose to use the reverse KI--diveggecce between (the normalized) F and d as the difference measure
Let P(),o) and be the normalized probability distributions corresponding to the SHDD representa- tion of (0, () and an arbitrary (L++)) vector
U Um=
P(o,s)(u,)) exp
exp
Σ
(0, (m) (u, / (FI
(7)
1=0 m=-i
(e(u, = exp
Σ
Clm Mim (u, /Z(e)
(8)
m=-
Here Z(e) a normalization constant (e)) Sor So² exp Et En Eim
Viml''i en du du'.
Then the SHDD KL-divergence is computed by
LSUDD KL. (e, PESHDDD ((,,))) x))) = /
/ 2 ge(u, u) log
vo(u,v) dudv
Pro,o) (u,
is easy to verify that the SHDD KL-divergence is a continuous difference measure. As for the property described in Equation 4 notice that by [9], the Wasserstein-2 distance W2 between P(0.)) and ge is bounded by the KL-divergence the following inequality:
W2 (P(o.)): ge) S CLSHHD-KL (c, PESHDD((O,0))
(10)
C' is finite constant. W2, being the Earth Mover's Distance, quantifies the amount of probability mass transport between two distributions. Thus, when LSUDD KL. (e, (PESuDD(()) Φ)) is small, the difference in probability mass distribution is also small, and consequently the largest-mass-regih found by the mode-seeking SHDD decoder will also remain mostly unchanged. Figure [ visualizes this with concrete examples (pretrained Sphere2 location encoder and learned neural decoder v.s.
our SHDD encoder and decoder) using heatmaps.
4.4 SHDD Decoding
KL. KI.-Divergence SHDD Decoder
Following Equation 2 the KL--Divergence SHDD Decoder is:
PDx min LSHDD-KL (e, (EESsD(((edd)) (0.6)
(11)
is impractical to compute PixxI. exactly. Luckily, Equation a natural simplification possible. Notice that minimizing reverse KL-ddvergence leads to mode-seeking behavior [29]. i.e. the (0, Φ) that satisfies Equation TT should fall within the region with the largest probability mass. Thus, we can decode by finding the center of its probability mass concentration.

---

SIDD Easeoing
3
C-Siren Module
Uneer d;
0.000
Uiter 23, 24
x
Unner di X.d,
Lneardd x
GELJ
CS-UNet
Unear dy X
Unser da
Uinarr X
SWISOOSa Emibedding
Prevvieed
6r
(8)
(b)
Figure 2: (a): The architecture of Condition SirenNet Module (C-Siren). x: is the input latent vector, x' the output latent vector, / is the scalar timestep, and is the embedding of the imput image. di is the input dimension, do is the output dimension, dr is the time embedding dimension, di the conditional embedding dimension. (b): The architecture of Conditional SirenNet- Masedd UNet (CS-UNet) and the workflow of LocDiff. d is the latent dimension. The numbered circles denote the order of training stops.
Mode-Seeking SHDD Decoder Let e = U Umm=rle [emm] the position decoder mode is defined
be an arbitrary vector in (I++)), then
mode (e; = arg max
MAP poto (Sam Jonn exp
Σ 1=0 m=-1
L
Σ
cim Vim (1, dudv) }
(12)
Ja-p
(0,6)
where p is a hyperparameter that controls the granularity of the evaluation. There is trade-off between decoding spatial resolution and decoding stability: when p is large, we only know the rough range of (0, Φ)) but the result is less sensitive to local spikes, and vice versa.
One advantage of adopting the SHDD) decoder is its learning-free property. Unlike learned neural decoders, there is no loss introduced during the decoding stage. Besides, the mapping from diffusion outputs to spherical coordinates is shown be continuous and relatively smooth. Therefore, is safe to train latent diffusion models only using the SHDD KL-divergence loss LSHDD-KL- Another critical advantage is that the SHDD decoder does not depend on specific partitions of the spherical surface or the spatial distributions of image/location galleries. This is because the SHDD representation is in effect continuous spherical function and in theory one can evaluate in arbitrary resolution. The only two constraints are the maximum degree of Legendre polynomials L which limits the spatial resolution of the spherical function itself and the computational resources (e-g., float32 or fl float64, evaluation granularity p), both being independent from other factors.
4.5
Conditional SirenNet-Based UNet (CS-UNet)
Inspired by (37. we used SirenNet 40 as the backbone of our diffusion model. The theoretical motivation behind this decision is that Spherical Harmonics coefficients are sums of sinusoidal and cosinusoidal functions (See Appendix AA3) Using sine as the activation function helps preserve gradients because the derivatives of sinusoidal/ccoiiuuoitall functions are still sinusoidal and cosinu- soidal functions. Figure 2(a) depicts the network architecture of the Conditional SirenNet (C-Siren) module. The design straightforward: inputs are the latent vector x, the image condition embedding es, and the diffusion step t. First, we use feed-forward layers to project x and into hidden vectors hz, hz- Then we use the sinusoidal embedding layer [41] and feed-forward layers to project the discrete diffusion timestep t into scale vector a and a shift vector Then, we transform hr into hr_ = (1 + (x) @ h + 6., which an unconditional denoising step. Following that, we sum the transformed ltz and the condition hy and pass the sum to feed-forward layer, which adjusts denoising step under the guidance of the condition. Finally, output the dine-activated hidden vector to the next C-Siren module. Figure 2(b) describes the architecture of CS-Unet.
7

---

4.6 LocDiff
Next, we introduce the training cycle of our LocDiff model as illusttraedd in Figure 2(b) A training
data sample includes an input image I and its associated geolocation p = (0, Φ) serving as the prediction target. First, we use frozen CLiP-based image encoder [33] encode the image I into an image embedding éイ. Then, we encode the geolocation p (0, Φ) into its SHDD representation look-up table. Following that, we perform a standard DDPM In forward pass of the latent diffusion process, the spherical Dirac delta function 6(60.00) defined by PESHDD (f, Φ) will be gradually added noise until being reduced to vector whose values in each dimension are purely generated from Gaussian noise. In a backward pass of the latent diffusion model, the CS-UNet will start with noise vector and gradually recover 000) We implement the DDPM algorithm based on the open-source Py Torch implementation. We use the SHDD KL-divergence LSHDD-KL between the ground-truth SHDD representation PESSHDD 6) and the diffusion output as the training objective, because it is more computationally stable and preserves the spatial multi -ulti-scalability than the spherical MSE (e.g. great circle distance) loss. During inferencing, we sample coefficient vectors from Gaussian noise conditioned on CLIP-based image embeddings and use PD to predict locations. There are two important implementation details worth mentioning. In practice, the integrals in Equation | and Equation 12 are approximated by summation. More specifically, we select set of N anchor points AN {00, 0;) E C) on the sphere.
PESUDD (0.c Φ) and store them
training 13 based on the proposed CS- architecture as shown in Figure
(0 di) p(e.6) (0,, 0i)
LSHDDAKI. (e, PBSSDDD (0, (0, = Σ Uo (o;, bb) log
(13)
N
(e: arg max
Σ (0.0) 1=1
(0., 0.) < p} exp Σ
Σ
Clm Yom Con.im(0r,, oi)
(14)
1=0
LSHDDAKI is used for training. thus we random sample N = 2048 anchor points over the globe for each mini-batch to avoid overfitting. As for Plmmodes the choice of AN introduces inductive bias the regions with more anchor points may have heavier impact the decoding results and higher
spatial resolutions. However, TableB]shove that LocDiff performs stably well on different AN. See Appendix A.5 for the pseudo codes of LocDiff.
5
Experiments
5.1 Main Results
We first follow the experimental setup of GeoCLIP [5). widely used image geolocalization model, for a fair comparison. The training dataset is MP16 (MediaEval Placing Tasks 2016 (19) containing 4.72 million geotagged images. The test datasets are 3 global-scale image geolocalization datasets Im2GPS3k M., YFCC26k (44), and GWS15k (6]. Note that the test datasets Im2GPS3k and YFCC-26k have similar distributions to MP16, and more importantly, their data points might overlap with those in MP16, which benefits retrieval- basee approaches [45
During model inference, we run multiple (16) samplings given different augmentations of each test image and use their geographical center as the prediction. This a successful strategy proven by GeoCLIP45) 45) and (55 For example, in our experiments, without this augmentation and averaging step, the 1km accuracy of GeoCLIP on Im2GPS3K may drop from 14% to below 10%. Then we count how many predictions fall into the neighborhoods of the ground-truth locations at different scales respectively, which are denoted as Street (1 km), City (25 km), Region (200 km), County (750 km), and Continent (2500 km). See Appendix A.4 for detailed ablation study on L. Larger L can improve LocDiff resolution, but there is a limit of L = 47 before hitting numerical stability issues (Figure We leave numerical stability issue to our future work.
Table summarizzes the performance of different models on these three datasets. In addition to the pure LocDiff model, to further increase the resolution without increasing L, we develop a hybrid approach denoted as LocDiff-H in Thhat combines the advantages of LocDiff and retrieval-basn models such as GeoCLIP we use LocDiff to generate candidate locations, and restrict the retrieval of GeoCLIP to the 200 km radius region around the candidate locations. Please see Appendix AA66 for detailed description of this hybrid model. From Table [ can that LocDiff-H can outperform
8

---

Table 1: Main evaluation results with the GeoCLIP 45 eval setup. L is the degree of SHDD representations
used in our model. LocDiff-HH stands for the hybrid approach of LocDiff plus GeoClip. "Coun." and "Cont." indicate Country and Continent scales. Bold and underline numbers denote the best and second best performance on the corresponding dataset. mcans not reported.
Tm2GPS3k
YFCC-26k
GWSLSE
Model
Street
City
Region Coun.
Cont.
Street City
Region Coun.
Cont.
Street City
Region Coun.
Cont.
km
25 km
200 km 750 km
2.5k km
km 25 km 200 km 750 km 2.5k km
km 25 km 200 km 750 km 2.5k km
[LJKNNN 17
46
7.2
19.4
26.9
38.9
55.9
PlaNet +7
8.5
24.8
34.3
48.4
64.6
4.4
11.0
16.9
28.5
47.7
CPlNNet
67
10.2
26.5
34.6
48.6
64.6
ISNs 20
10.5
28.0
36.6
49.7
66.0
5.3
12.3
19.0
31.9
50.7
0.05
0.6
4.2
15.5
38.5
Translocator
11.8
31.1
46.7
58.9
80.1
7.2
17.8
28.0
41.3
60.6
0.5
1.1
8.0
25.5
48.3
GeoDer
0
12.8
33,5
45.9
61.0
76.1
10.1
23.9
34.1
49.6
69
0.7
1.5
8.7
26,9
50.5
GcoCLIP
45
14.1
34.5
50.7
69.7
83,8
11.6
22.2
36.7
57.5
76.0
0.6
3.1
16.9
45.7
74.1
PIGEON
10
11.3
36,7
53.8
72.4
85.3
10.5
25.8
42.7
63.2
79.0
0.7
9.2
31.2
65.7
85,1
LocDiff (L=47)
10,9
34.0
72.5
85.2
9.6
22.8
58.6
76.8
12.4
33.7
67.0
85.0
LocDifT-H (L=23)
15.3
36.5
56.4
88.4
26.0
41.9
80.3
7.4
33.5
66.2
85.0
all baselines and LocDiff across almost all spatial scales on both Im2GPS3K and YFCC-26k datasets while remaining competitive for the city scale metric on Im2GPS3k and region scale metric YFCC- 26k. On the GWS15k dataset, LocDiff shows the best performance while LocDiff-H underperforms LocDiff, especially
higher spatial scale metrics (e.g., street and city scales). We hypothesize that this is because, unlike Im2GPS3k and YFCC-26k, the spatial distribution of GWS15k 5k is very different from that of the MP16 training dataset. The spatial inductive bias brought by GeoCLIP will not benefit but hurt the performance of LocDiff-H on GWS15K, especially for fine-scale metrics.
Several generative models have been developed recently for im- age geolocalization including Diff R3
Table Comparison with existing generative methods. The evaluation setup is identical to Dufour et. al 8. L is the degree of SHDD represen- tations used in our model. Bold numbers denote the best performance on the corresponding dataset. means not reported.
81. FM IR3 (8), and RFM S2 [8] Since these mod- els were evaluated in different datasets and experimental setups, in order to conduct a fair com- parison, follow the evalua- tion setup in 8 and compare our LocDiff against these gener- ative geolocalization models on two datasets OSV-5M 3 and
City
Region
Country
Continent
Dataset
Model
Density
Coverage
25 km
200 km
750 km
2500 km
VMF X
0.6
17.2
$2.7
+MFMix 17
0.3
11.1
34.2
OSV-SM
Diff R
3.6
40.9
75,9
0.752
0.568
FM R
8
4.2
40.0
74.9
0.799
0.575
RFM Sa
8
5.4
44.2
76.2
0.797
0,590
LocDiff (L=47)
11.0
46.3
77.0
88.2
0.795
0.572
vMF S
15.0
30.9
53.4
YFCC-4k [46]. The results are
vMFMix
8.8
20.9
41,0
shown in Table 2 We can see
YFCC-4k
Diff R
37.7
54.7
71.9
0.959
0.837
that LocDiff can outperform all generative geolocalization mod- els on both datasets which clearly showcases the advantages of our multi-scale latent diffusion approach compared with those diffusion models directly applied on the original coordinate space g
R
35.0
53.2
73.1
RFM 8
36.4
34.5
73.6
0.926
(10M) [8]
45.3
61.1
77.7
LocDiff (L=47)
46.7
65.2
79.9
5.2 Generalizability Experiment Results
The key advantage of generative geolocalization over traditional chassifcation/eetriaaaa-bo geolo- calization methods is that completely gets rid of predefined spatial classes and location galleries. As noted in [45). the performance of retrieval-based geolocalization methods depends heavily on the quality of the gallery- i.c., how well the candidate locations in the gallery cover the test locations. For example, GeoCLIP uses a 100k gallery with locations drawn from MP16 training data. When using this gallery for the GWS15k dataset, the performance drops due the spatial distribution mismatch between MP16 and GWS15k. As shown in Table 3 GeoCLIP's performance significantly drops when an evenly sampled grid on Earth is used instead of the default image gallery. At small scales, this is explainable because the grids are too coarse to differentiate I km to 25 km objects. However, large scales, the performance of Ge also drops significantly which is unexpected. With million grid points, the average distance between two candidates is less than 30 km. However, the

---

Table 3: Generalizability experiment. Numbers outside of the brackets denote geolocalization accuracies. Numbers in the brackets denote relative performance degradation compared to the prior knowledge gallery/anchor. Bold numbers denote the largest drop in performance with the given gallery/anchor setting.
Street
City
Reglon 200 km
Country
Continent
Model
Gallery/Anchor
Size
km
750 km
2500
1P16
100k
14.11
34.47
50.65
69.67
83.82
IM
0.03
(199.796)
9.18 (
33.47 (133.906))
55.32 (120066300)
75. (1%).
GooCLIP
Grid
5
0.03
(199. 79%)
7.17 (J79.211
29.40
([41.96%)
52.29
(124.94%%
73.11 (L12.800g
K
0.00 (1100.00%)
2.67 (192.25%)
22.39 (155.81%)
47.35
(132.08%%
68.77
(L17.94%)
21
0.00
(1100.00 %
0.87 (199..48%) %
(261.4411 %)
43.78
(1.37.17%)
64.33 (J233266%
MP16
100 k
0.57
11.1
44.42
68.35
82,50
M
0.00 (598.25%)
4.37 (_,60.633))
433.04 (13.,10%))
68.30 (100007%)
81.66 (1.1.02%))
LocDilf (L=23)
Grid
500 k|
0.07 (187.729%)
4.47 (3159.33%)
43,118 (12.799)))
68.36 (10.01%) (%)
81.65 (J (13.003%)
100
0.07 (187.725%))
4.04 (1663.60%)
42,91 (13.400%)
68.34 (10.01%) (%)
82.18 (.10.3955)
k 0.03 (194.745))
4.90 (155.861%)
43,44 (12.211%)
68.29 (10.09%)
81.68 (10.9955)
performance of GeoCLIP at the 200 km, 750 km, and 2500 km scales (way larger than 30 km) is still much lower than the performance when using 100K MP16 gallery locations. It indicates that the decline in performance is due to GeoCLIP's weak generalization to new, unseen locations. We can see that the gallery has a strong inductive bias that narrows the spatial scope, and makes the retrieval model easier to overfit, but hurts its spatial generalizability.
Our LocDiff model, while also using anchor points for decoding (training is random), is almost unaffected by the choice of anchor points. To align with GeoCLIP, we compare the same MP16 gallery and evenly sample grid points as decoding anchor points. We can see, at the smaller scales, just like GeoCLIP, introducing the MP16 gallery helps improve the accuracy because its spatial inductive bias helps offset the vagueness of decoding. However, at larger scales, the performance of Liff is almost independent of the choice of anchors- both the way how we pick the anchor points (MP16 or even grid) and the total number of anchor points (from 21k to 1M). It is strong indicator of better spatial generalizability for LocDiff.
5.3 Computational Efficiency
Though the mathematical derivation of SHDD encoding/decoding seems complicated, our method does NOT introduce much computational cost. First, the SHDD encoding/decoding operations are both deterministic, closed-form. For example, the d-dimensional (d 256, 576, 1024) SHDD encoding of point is just a parallel evaluation of a list of two-vari functions. whose time complexity is (1) and space complexity is (d). Second, during training, the SHDD encodings can be precomputed and used as an embedding lookup table and there are no further computational overheads. SHDD decoding is implemented as a matrix multiplication followed by an ar argmax operation, which is also very efficient. See Table
Encoding
DDPM
Decoding
CPU
0.0003s per location
GPU
0.056s per image (200 stops, 16 augmentations) GPU 0.0012s per batch (512)
Table 4:
Time efficiency for unit operations in LocDiff
We wish to emphasize that using an SHDD encoded multi-scale representation helps the diffusion process converge faster, because each dimension encodes information at the respective spatial scales. For example, if at a certain step, the generated SHDD representation is 500km within the ground-truth, then the dimensions with larger than 500km scales will remain mostly intact, and only the finer dimensions need to be altered. In practice, our model takes about 2 million steps to converge on YFCC, while the best results reported in g take 10 million steps.
6 Conclusion, Limitations and Future Work
In this paper, we propose a novel SHDD encoding-decoding framework that enables multi-scal latent diffusion for spherical location generation. We also propose a CS-UNet architecture to learn conditional diffusion and train a LocDiff model that addresses the image geolocalization task via location generation. It achieves the state-of-the-are geolocalization performance over all existing baselines on 5 global-scale image geolocalization datasets and demonstrates significantly better spatial generalizability. In the future, we aim to leverage the gallery during the diffusion process and explore different ways to further speed up decoding.
10

---

Acknowledgements
This work is mainly funded by the National Science Foundation under Grant No. 2521631 A Statistics-Basee Geographic Bias Quantification and Debiasing Framework for GeoAl and Foundation Models. Gengchen Mai acknowledges support from the Microsoft Research Accelerate Foundation Models Academic Research (AFMR) Initiative and University of Texas at Austin Library Map & Geospatial Collections Explorer Fellowship. Zeping Liu acknowledges the support of the Amazon Al PhDD fellowship. Any opinions, findings, conclusions, recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
References
[1] Benjamin Adams al. Frankenplace: interactive thematic mapping for ad hoc exploratory search. In Proceedings of the 24th international conference on world wide web, pages 12-22, 2015.
[2] George B Arfken, Hans J Weber, and Frank E Harris. Mathematical methods for physicists: comprehensive guide. Academic press, 2011.
[3] Guillaume Astruc, Nicolas Dufour, loannis Siglidis, Constantin Aronssohn, Nacim Bouia, Stephanie Fu, Romain Loiseau, Van Nguyen Nguyen, Charles Raude, Elliot Vincent, et al. Openstrectview-S5n The many roads to global visual geolocation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 21967-21977, 2024.
[4] Thomas Berg, Jiongxin Liu, Seung Woo Lee, Michelle L. Alexander, David W Jacobs, and Peter Belhumeur. Birdsnap: Large-scale fine-grained visual categorization of birds. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 2011- 1- 2018, 2014.
[5] Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PmLR, 2020.
[6] Brandon Clark, Alec Kerrigan, Parth Parag Kulkarni, Vicente Vivanco Cepeda, and Mubarak Shah. Where we are and what we're looking at: Query based worldwide image geo-localization using hierarchies and scenes. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 23182-23190, 2023.
[7] Elijah Cole, Grant Van Horn, Christian Lange, Alexander Shepard, Patrick Leary, Pietro Perona, Scott Loarie, and Oisin Mac Aodha. Spatial implicit neural representations for global-scale species mapping. In International Conference on Machine Learning, pages 6320-6342. PMLR, 2023.
[8] Nicolas Dufour, David Picard, Vicky Kalogeiton, and Loic Landricu. Around the world in 80 timesteps: A generative approach to global visual geolocation. arXiv preprint arXiv:2412.06771, 2024.
[9] Alison L. Gibbs and Francis Edward Su. On choosing and bounding probability metrics, 2002.
[10] Lukas Haas, Michal Skreta, Silas Alberti, and Chelsea Finn. Pigeon: Predicting image geoloca- tions. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages I 18933-12002, June 2024.
[11] James Hays and Alexei A Efros. Im2gps: estimating geographic information from a single image. In 2008 ieee conference compuler vision and putlern recognition, pages 1-8. IEEE, 2008.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33,6840-6851, 2020.

---

[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In
H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 6840-6851 Curran Associates, Inc., 2020.
[14] Yuan Hu, Zhibin Wang, Zhou Huang, and Yu Liu. Polybuilding: Polygon transformer for building extraction. ISPRS Journal of Photogrammetrry and Remote Sensing, 199:15-27, 2023.
[15] Chin-Wei Huang, Milad Aghajohari, Joey Bose, Prakash Panangaden, and Aaron C Courville. Riemannian diffusion models. Advances in Neural Information Processing Systems, 35:2750- 2761, 2022.
[16] Sinem Ince, Franz Barthelmes, Sven Reißland, Kirsten Elger. Christoph Förste, Frank Flechtner, and Harald Schuh. 1cgem-15 years of successful collection and distribution of global gravitational models, associated services, and future plans. Earth system science data, 11(2):647-677 674, 2019.
[17] Mike Izbicki, Evangelos E Papalexakis, and Vassilis Tsotras. Exploiting the earth's spherical geometry geolocate images. In Machine Learning and Knowledge Discovery in Dutabases: European Conference, ECML PKDD 2019, Wurzzurg, Germany, September 16-20, 2019, Proceedings, Part II. pages 3-19. Springer, 2020.
[18) Konstantin Klemmer, Esther Rolf, Caleb Robinson, Lester Mackey, and Marc Rußwurm. Satclip: Global, general-purposs location embeddings with satellite imagery. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 39, pages 4347-4355, 2025.
[19] Martha Larson, Mohammad Soleymani, Guillaume Gravier, Bogdan lonescu, and Gareth IF Jones. The benchmarking initiative for multimedia evaluation: Mediaeval 2016. IEEE MultiMe- dia, 24(1):93-96, 2017.
[20] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023.
[21] Zeping Liu, Fan Zhang, Junfeng Jiao, Ni Lao, and Gengchen Mai. Gair: Improving mul- timodal geo-foundation model with geo-aligned implicit representations. arXiv preprint arXiv:2503. 16683, 2025.
[22] Oisin Mac Aodha, Elijah Cole, and Pietro Perona. Presence-only geographical priors for fine- uine-graieed image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9596-9606, 2019.
[23] Gengehen Mai, Krzysztof Janowicz, Yingjie Hu, Song Gao, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao. A review of location encoding for geoai: methods and applications. International Journal of Geographical Information Science, (644):639-673, 2022.
[24] Gengehen Mai, Krzysztof Janowiez, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao. Multi-scale repre- sentation learning for spatial feature distributions using grid cells. In ICLR 2020. openreview, 2020.
[25] Gengchen Mai, Krzysztof Janowicz, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao. Multi-scale repre- sentation learning for spatial feature distributions using grid cells. In The Eighth International Conference Learning Representations. openreview, 2020.
[26] Gengchen Mai, Ni Lao, Yutong He, Jiaming Song, and Stefano Ermon. Csp: Self-supervised contrastive spatial pre-training for geospatial-visual representations, In International Conference on Machine Learning, pages 23498-23515 PMLR, 2023.
[27] Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Jiaming Song, Stefano Ermon, Krzysztof Janowicz, and Ni Lao. Sphere2vec: A general-purpose location representation learning over spherical surface for large-scale geospatial predictions. ISPRS Journal of Photogrammetry and Remote Sensing, 202:439-48 462,994-66666 2023.
12

---

[28] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. ECCV, 2020.
[29] Tom Minka et al. Divergence measures and message passing, Technical report, Technical report, Microsoft Research, 2005.
[30] Eric Muller-Budack Kader Pustu-Iren, and Ralph Ewerth. Geolocation estimation of photos using a hierarchical model and scene classification. In Proceedings of the European conference on computer vision (ECCV), pages 563-579 2018.
[31] Shraman Pramanick, Ewa M Nowara, Joshua Gleason, Carlos D Castillo, and Rama Chellappa. Where the world this image? transformer-baset geo-localization the wild. arXiv preprint arXiv:2204.1380 2022.
[32] Shraman Pramanick, Ewa M Nowara, Joshua Gleason, Carlos D Castillo, and Rama Chellappa. Where in the world is this image? transformer- based geo-localization in the wild. In European Conference Compuler Vision, pages 196-215. Springer, 2022.
[33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763. PMLR, 2021.
[34] Ali Rahimi, Benjamin Recht, et al. Random features for large-scale kernel machines. In NIPS, volume 3, page 5. Citeseer, 2007.
[35] Jinmeng Rao, Song Gao, and Sijia Zhu. Cats: Conditional adversarial trajectory synthesis for privacy- preecccpreeevvngggg trajectory data publication using deep learning approaches. International Journal of Geographical Information Science, 37(12):2538-22 2274,, 2023.
[36] Noam Rozen, Aditya Grover, Maximilian Nickel, and Yaron Lipman. Moser flow: Divergence- based generative modeling on manifolds. Advances in Neural Information Processing Systems, 34:1 14:17669-17680, 2021.
[37] Mare Rußwurm, Konstantin Klemmer, Esther Rolf, Robin Zbinden, and Devis Tuia. Geo- graphic location encoding with spherical harmonics and sinusoidal representation networks. In Proceedings of the International Conference on Learning Representations (ICLR), 2024.
[38] Paul Hongsuck Seo, Tobias Weyand, Jack Sim, and Bohyung Han. Cplanet: Enhancing image geolocalization by combinatorial partitioning of maps. In Proceedings of the European Conference Computer Vision (ECCV), pages 536-551, 2018.
[39] Yujiao Shi, Xin Yu, Dylan Campbell, and Hongdong Li. Where am i looking at? joint location and orientation estimation by cross-view matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4064 4072, 2020.
[40] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances neural information processing systems, 33:7462-7477, 2020.
[41] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[42] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020.
[43] Kevin Tang, Manohar Paluri, Li Fei-Fei, Rob Fergus, and Lubomir Bourdev. Improving image classification with location context. In Proceedings of the IEEE international conference on computer vision, pages 1008-1016 2015.
[44] Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfee100m The new data in multimedia research. Communications of the ACM, 59(2):64-73, 2016.
13

---

[45] Vicente Vivanco, Gaurav Kumar Nayak, and Mubarak Shah. Geoclip: Hlipinspired alignment between locations and images for effective worldwide geo-localization. In Advances Neural Information Processing Systems, 2023.
[46] Nam Vo, Nathan Jacobs, and James Hays. Revisiting im2gps in the deep learning era. In Proceedings of the IEEE international conference on computer vision, pages 2621 2621-2630, 2017.
[47] Tobias Weyand, Ilya Kostrikov, and James Philbin. Planet-photo geolocation with convolutional neural networks. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Purt VIII 14, pages 37-55. Springer, 2016.
[48] Nemin Wu, Qian Cao, Zhangyu Wang, Zeping Liu, Yanlin Qi, Jielu Zhang. Joshua Ni, Xiaobai Yao, Hongxu Ma, Lan Mu, Stefano Ermon, Tanuja Ganu, Akshay Nambi, Ni Lao, and Gengchen Mai. Torchspatial: A location encoding framework and benchmark for spatial representation learning. arXiv preprint arXiv:2406. 15658, 2024.
[49] Minkai Xu, Alexander $ Powers, Ron Dror, Stefano Ermon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference Machine Learning, pages 38592-38610. PMLR, 2023.
[50] Lingfeng Yang, Xiang Li, Renjie Song, Borui Zhao, Juntian Tao, Shihao Zhou, Jiajun Liang, and Jian Yang. Dynamic mip for fine-grained image classification by leveraging geographical and temporal information. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10945-10954 2022.
[51] Zhongliang Zhou, Jielu Zhang. Zihan Guan, Mengxuan Hu, Ni Lao, Lan Mu, Sheng Li, and Gengehen Mai. Img2loc: Revisiting image geolocalization using multi-modality foundation models and image-based
generation. In Proceedings of the 47th Interna- tional ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2749 2754, 2024.
[52] Sijie Zhu, Linjie Yang, Chen Chen, Mubarak Shah, Xiaohui Shen, and Heng Wang. R2former: Unified retrieval and reranking transformer for place recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19370-19380, 2023.
14

---

A Appendix
A.1 Example of Diffussinn Steps
Illustration how spherical probability mass concentration (mapped to a plane) corresponding to the SHDD encodings changes along the backward process at step=0, 10, 20, 100, 200, respectively. The more bright, the more probability mass.
Figure 3:
A.2 Sparsity of Existing Positional Encoding Methods
Almost all location encoders can be formulated as the following equation 48
Enc(O, 6) (N +)),
(15)
PE() is a position encoder that transforms the location p = (0, Φ) into a W W-dimensional vector, referred as the position embedding. The neural network NN() R.W + Re a learnable function that maps the position embedding (E((0, σ) Rw the location embedding Enc(O, o) gd
1) tile is a vanilla location encoder used by many pioneering studies 4 43 divides geographic regions into discrete global grids based on longitude and latitude and learns corresponding partition embeddings based on the grid cell indicator vectors.
2) wrap [22] is sinusoidal location encoder, normalizing latitude and longitude and processing with sinusoidal functions before feeding into NN (), which is composed of four residual blocks implemented through linear layers.
3) wrap + ffn 27 is variant of wrap that substitutes NNWTOP(() () with NN (), a simple FFN.
4) rbf 225 is a kernel-based location encoder. It randomly selects W points from the training dataset as Radial Basis Function (RBF) anchor points. It then applies Gaussian kernels to each anchor moints.Each input point E; is represented as W-dimension feature vector using these kernels, which then processed by NN ().
5) ff stands for Random Fourier Features and is another kernel-based location encoder. first encodes location 7 into W dimension vector /F (Ø) (o()). Each component of (( first projects F into random direction W; and makes a shift by b,. Then it wraps this line onto unit circle in IR2 with the cosine function. PF (ft) is further fed into NN () to get a location embedding.
6) SopaceVeecgree and Space2Vec-thees encoders on 2D Euclidean space. Both of them implement the position encoder [EE( as performing Fourier transformation on a 2D Euclidean space then fed into the NN Space2Vec-grid treats = (A, sp) as a 2D coordinate while Space2 Vec-thorry be simulated by summing three cosine grating functions oriented 60 degree apart.
25 are two versions of sinusoidal multi-scale location
7) zyz 27 a vanilla 3D location encoder, converting the lat-lon spherical coordinates into 3D Cartesian coordinates centered at the sphere center with position encoder PE*** "f)) then feeds the 3D coordinates into an MLP NN ().
NeRF can viewed as multiscale version of zyz by employing Neural Radiance Fields (NeRF) 28 as its position encoder.
Sphere2 (27) including Sphere2Vec-sphered Sphere2ppppeee1 Vec-speere+++ Sphere2Vec-spher Sphere2 Vec-sphere. and Sphere2Vec-df s, is series of multi-scale location encoders for spheri- cal surface based on Double Fourier Sphere (DFS) and 5pace2Vec. The multi-scale representation of Sphere2Ver Vec is achieved by one-to-one mapping from each point total number of scales. They are the first location encoder series that preserves the spherical surface distance between any two points to knowledge.
(As, ppx) E g with S be the
15

---

10) Siren (SH) [37 is more recently proposed spherical location encoder, which claims a learned Double Fourier Sphere location encoder. uses spherical harmonic basis functions as the position encoder PRiine (i)). (it), followed by a sinusoidal representation network (SirenNets) as the NN((.
These existing location embedding spaces suffer from sparsity issues, primarily due to the inherent correlations among the different dimensions of the position encoders. The dimensions of position embeddings are frequently interdependent. As result, many points in the position embedding space become distant or isolated from another.
A.3
Computation of Spherical Harmonics
To compute Ymm one can use the following expression in terms of associated Legendre polynomials P (x))
V p! (cos sin (mm|)) m < 0
J JP (cos 0)
0
(16)
(-1)" /2/P! (cos cos (m/))
m >> 0
21 + 1 (1- /m])!
where
and (1 is further computed by
V 4x (1 |m])
(1)(t++ k- (1122)1)/2) k
())
1)/2)
(+)x = (1-
(17)
(k m)
R=
A.4 Spatial Resolution of SHDD Encoding/Decoding
The spatial resolution of SHDD encoding/decoding (i.e., on what scales the mode-seeking decoder can accurately locate the probability mass concentration of the spherical Dirac delta functions) bound by the degree L of Legendre polynomials. For an L-degree SHDD representation. the spatial scale threshold at which it can accurately approximate spherical functions is //L in radian or approximately 20000/ in kilometers [16]. For example, for L = 15, 23, 31, the thresholds are around 1300 km, 870 km, and 640 km, respectively. At scales significantly below half this threshold, even if the diffusion model generates accurate coefficient vectors, the mode-seeking decoder can still only decode vague locations with large variances. Figure gives visual intuition.
Figure 4: Illustration of the spatial resolutions with = 15, 23 and 31. The bright regions are the probability mass concentrations and points within these regions are similarly likely to be decoded as the location predictions. The smaller the bright regions are, the lower enrors the SHDDD decoding brings.
Therefore, uplift the performance of LocDiff, one straightforward way is use larger L. We conduct an ablation study of the effect of L on image geolocalization performance on the Im2GPS3K dataset. The results are shown Figure 5(a) We can see L increases, while the model perfor- mances at larger spatial scales (e.g., 750km, 2500km) only increase slightly, the performances at smaller scales (e.g., kkm, 25km, 200km) see huge uplifting. This validates our hypothesis a larger L can make the mode-seeking decoder decoding vague locations with smaller variances, thus leading to higher image geolocalization performance. The largest we tried in Figure 5(a) is 47 which corresponds to a spatial resolution of 200km. This is why we see huge performance improvements on the 25km and 200km curves but not on the km curve since 1k is still significantly smaller than the current spatial scale threshold.
16

---

80
Log-scor scale Maximum Absolute Value
10"
60
10a
Accuracy
40
10a
20
10
k5
0-
720 ky
10
20
30
40
50
60
1000
2000
3000
4000
Legendre Polynomial Degree
SHDD Encoding Dimension
(b)
Figure 5: (a): An illustration of how the image geolocalization performance on the Im2GPS3K dataset increases as L increases. Different curves indicate performance metrics on different spatial scales. (b): A oog-scale plot of the maximum absolute values of each SHDD encoding dimension up to 64 4096 dimensions.
However, it is not recommended to unlimitedly increase L. There are two major reasons: The SHDD encoding dimension increases quadratically with L, i.c., we need quadratic space to halve the spatial resolution. It is expensive and difficult to train diffusson model on very large encodings (c.g. to achieve km spatial resolution, we theoretically need 160,000 dimensions). We find that the higher the dimension of SHDD encodings, the higher the maximum absolute values of the coefficients. Figure is a Log-scale plot of the maximum absolute values of each SHDD encoding dimension up to L 63 (i.e., in total 64 64 4096 dimensions). The absolute values below 2500 dimensions in general manageable with only a few spikes. However, dimensions beyond this threshold become unbearably large, which makes the probability computation very unstable and easy overflow. Based on these observations, we use up to = 47 in our paper because now the dimension of SHDD encoding goes to 2304, still within the manageable range.
Morcover, to address the high dimension issue when we use large L, we find that applying a low-pass filter to the dimensions is good dimension reduction solution. See Figure 5(b] Many dimensions of the SHDD encodings have very small absolute values and will not significantly influence the results of SHDD encoding/decoding. Thus, may low-pass filter analogous to Fourier transformation and signal processing, which only keeps the dimensions that have adequately large coefficient values. This can be further investigated in future works.
A.5 Pseudo-Code for LocDiff
Algorithm Training LocDiff
Input A dataset D with location-image pairs {(p, I) Each location a tuple of latitude and longitude p (0, Φ). A Gaussian noise scheduler (tt), where i is the time step. SHDD encoder PESiDD- Pretrained Image encoder Bmm. CS-UNet model M with random initialization. SHDD KL-ddvergence
loss LSHDD-KL.
Output:A trained CS-UNe model M.
For p D:
compute the SHDD encoding of PESHDD (0, +); compute the image embedding of I: eq TT,(T);
randomly draw a time step f; add Gaussian noise to the SHDD encoding (forward process): + e +N);
use the CS-UNet to denoise the SHDD encoding conditioned the image embedding (backward process): . (e'. er, compute SHDD KL-divergence loss: I LSHUD xu e); use gradient decent to minimize and update M: M +- arg min 4;
return M
17

---

Algorithm/ 2 Inferencing LocDiff
Input
Image I. Random image augmentation AUG. Ensemble number N. DDPM sampler DDPMM DDPM step I' Pretrained Image encoder Eim. Trained CS-UNet model M with random initialization. SHDD mode-seeking decoder D'Dmm Mode-seeking range hyperparameter p. An initial location p (0,0).
Output: :An ensembled location prediction p.
For i == 1,2...-N:
generate random augmentation of image I: Iung +- ATG((T;
compute the image embedding of eag Eim [mn); randomly draw an SHDD encoding e':
use the DDPM sampler and the trained CS- M to sample an SHDD encoding conditioned on the image embedding:
DDDM(( , e', Sugg. T));
use the SHDD decoder PiDdodd decode a location from the sampled SHDD encoding: FDande (ê, p):
p + p +++ P return the ensemble of location predictions: p/N
A.6 A LocDifI-H Hybrid Appoach
We develop hybrid approach which uses LocDiff's predictions to narrow down the candidate regions and then deploy GeoCLIP to generate the final predictions. More specifically, first use LocDiff to sample multiple times (e.g.. 16) and get rough distribution of candidate locations, i.c. they indicate where the true answer is highly likely to reside. Then, restrict the retrieval of GeoCLIP to the neighborhoods (200 km radius) of these candidate locations. As shown in Table [ we can see that this hybrid approach yields substantially better results than both GeoCLIP and LocDiff alone. This approach is similar to the recommendation systems' retrieve and rerank approach. This flexible hybrid approach points an interesting future research direction. We can also replace GeoCLIP with other state-of-the-art image geolocalization models such as PIGEON [0), to further improve the model performance.
A.7 Inductive Bias of Gallery
The key factor that constrains the spatial generalizability of retrieval-based geolocalization models is the inductive bias introduced by the image gallery. When the spatial distribution of the gallery's 's image locations aligns well with the image locations in the test dataset, the performance of the retrieval- based models will be boosted, especially on low-error scales. However, without such inductive bias (e.g., using evenly spaced grid points as gallery locations), the performance of the eetrieval-base models on all scales will suffer.
To better understand what the inductive bias of an image gallery is and how heavily it affects retrieval- based models, we calculate the statistics that demonstrate how spatially aligned the MP16 gallery used in GeoCLIP is with the Im2GPS3K test data. We measure how close test image locations are to the gallery image locations by counting the number of gallery locations that within 1km/25km from given test image location. Table [ shows the statistics results. We can see that the MP16 image gallery's 's locations indeed closely match the image locations in the Im2GPS3K test dataset. In contrast, when we use a set of grid locations, there are much less locations falling into the 1km or 25 km buffer of the testing image locations.
Table 5: The percentage test locations that are close (within
km/25 km) to multiple gallery locations.
Gallery
MP16
Grid
Gallery Locations
1
>>
10
50
100
>
> 10
50
100
Within km
63.5%
32.7%
14.9%
9.78%
0.1%
0.0%
0.0%
0.0%
Within
95.2%
75.7%
51.9%
42.0%
38.9%
0.0%
0.0%
Figure 6 is a set of visualizations of Table It clearly demonstrates how GeoCLIP suffers greatly from using a grid gallery without prior knowledge (i.e., without using the inductive bias brought by the MP16 image gallery), while our method remains almost unaffected on larger spatial scales (200 km, 750 km, and 2500 km) and much less affected on smaller scales (25 km). These results clearly demonstrate that the high performance of GeoCLIP smaller spatial scales is based on the fact that the 6P16 image gallery used by GeoCLIP already contains candidate locations that are close enough to true answers (i.e., test image locations). However, this is not the case for our method because
18

---

Geociar
35
Geocua
LocDiff. sion
LocDiffusion
30
Accuracy
Accuracy
10
GeoCLIP LocDiflasion
SP16
Un and
sape Cris
tdw God
au God
HP16
IM Grd
Spol Gold
100 cold
238 Grdd
MP16
Iie Gold
seth Gnid
18s Grid
ona
Gallery/Anchor Choice
Gallery/Anchor Choice
Gallery/Anchor Choice
(a) km
(b)
(c) 200
70
80.0
Accurac 66
80
Accuracy
75.0
10
70.0
GeoCLIP LocDifission
65.0
HP16
50W Gold
19sk Grld
Grld
1918
sook Gidd
Gook Grd
21x Gix
Gallery/Anchor Choice
Gallery/Anchor Choice
(d) 750 km
(c) 2500 km
Figure 6: From (a) (e): the performance changes different choices of galleries/anchor points. Different plots indicate performance metrics under different spatial scales. For each plot, X-axis indicates choice of gallery locations, starting from MP16 points, million grid points, to 2 grid points. The y-axis indicates the gcolocalization performances on the corresponding spatial scale.
our model does not rely on such an image gallery either during training or during inferencing time. Thus, our LocDiff model suffers much less when we switch to a grid location gallery. Moreover, when we decrease the number of points in the grid gallery, the performances of GeoCLIP decrease significantly while the performances of our LocDiff are almost unaffected.
A.8 Computational Complexity
We trained our model on Linux server equipped with four NVIDIA RTX 5500 GPUs, each with 24GB of memory. We report the training time and space complexity on a single GPU in Table We do not have the training times for baseline models such GcoCLIP and PIGEON because did not train them from scratch and such statistics are not reported in their papers.
It can be seen that 1024 is the maximum SHDD dimension a single GPU can handle due to GPU memory constraints. For LoDiffusion models with SHDD dimensions beyond 1024, we either use the low-pass filtering technique mentioned in Section A.4 reduce the dimension to 1024, or split the computation across multiple GPUs. Therefore their computational complexity not separately reported.
Table Training time and space complexity. Each epoch undergoes 1500 iterations.
Degree L
Hidden Dimension
Second/Epoch
Memory (MB)
15
256
130
5691
23
576
212
10599
19

---

The major factor that decides the inference time of LocDiff is the choice of the sampler. In our experiment, we use the original DDPM sampler (i.e., no DDIM acceleration) with 200 sampling steps. The inference time per image for LocDiff is 0.056s and for GeoCLIP 0.024 seconds.
A.9 Ablation Studies
A.9.1 Comparison with other location encoding/decoding techniques
As we have discussed, the superiority of using SHDD for location decoding is that its encoding space is smoother than other location encoders that use neural networks such as rbf and Sphere2 [27]. To demonstrate this, we evenly sample million locations on Earth, encode them into corresponding location embeddings by using rbf and Sphere2Vee location encoder, and train neural network decoder to map the location embeddings back to locations. We also use the learned neural decoder in the LocDiff training with weights frozen. The ablation study results are shown in Table 7 We can see that the performances of rhf and Sphere2 are much worse than SHDD, especially on smaller scales. This is because: (1) the learned decoder is not 100% accurate, i.e. it may decode encoding to wrong location, and (2) if the encoding gets a small perturbation, the decoded location may have very large drift due non-linearity.
Table 7: Comparing the performance different encodors/decoders on Im2GPS3K, The NN is a 6-layer FFN trained on million corresponding location encodings evenly spaced on Earth.
Encoder
Decader
km
25 km
200 km
750 km
2500 km
24
18.2
44.1
Veeere2Vec
SHDD (L=17)
SHDD (L=47)
10.9
34.0
53.3
72.5
85.2
To better understand the spatial drift part, Table 8 shows how much spatial drift will bring to the decoded locations when we add a small Gaussian noise (variance = 0.01) to the corresponding location encoding. We see that compared with SHDD, both pretrained rbf and Sphere2 Vec models can have much larger spatial drifts when we add a small Gaussian noise (variance = 0.01) to the corresponding location encoding. The larger the spatial drift, the less robust the encoding/decoding process is to small hidden space perturbations. Since the diffusion model will not generate perfectly noiseless encodings, such spatial drift indicates the intrinsic error of the corresponding location encoding/decoding method.
Table 8: Comparing the spatial drifts when applying small Gaussian noise (variance 0.01) to the encoding. The NN is 6-layer FFN trained 1 million corresponding location encodings evenly spaced on Earth.
Eucoder
Decoder
Perturbation Drift
fbf
NN
102.4 km
Sphene2Vec
NN
89.1 km
SHDD (L=47)
SHDD (L=47)
5.3 km
A.9.2 Comparison with other losses
Table 9: Comparing the performance of using different training losses on Im2GPS3K.
Loss
km
200
750
2500
0.0
L2
0.0
0.7
20.1
32.7
44.9
Cosine
7.5
322
53.0
71.5
84.9
SHDD (L=47) KL-divergence
10.9
34.0
53.3
72.5
85.2
Table ssoows an ablation study on the impact of different loss functions. We can see that the SHDD KL-divergence is significantly better than L1/L2 losses. Cosine distance, being similar to our SHDD KL-ddvergence in terms of mathematical formulation (SHDD KL.-divergence is the sum of exponential element wise multiplications, while cosine similarity is the sum of raw element- wise multiplications). has comparable performance especially on larger scales. It would be good approximation to reduce computational costs. We will add more thorough experiments in the camera-ready version.
20

---

A.9.3 Ablation studies on other modules
We investigate how variations in the width of the CS-UNet affect its performance (see Table TOp. In general, shrinking the bottleneck width w of the CS-UNet seems to help alleviate model overfitting (we can adopt lower dropout rate) and slightly boost performance, but make the model more difficult to train.
Table 10: The bottleneck width is the narrowest part of each C-Siren module. We report the performance when input encoding dimension is 1024 (L 31) for the sake of limited time.
Setting
km
25 km
200 km
750 km
2500 km
32,
6
27.2
50.9
71.2
84.1
128, d =
4.7
27.0
50.2
70.8
84.3
A.10 Training Hyper parameter Setup
Table Training Set-up
Dimensions
peperarraettrss
Degree
dr
batch size
Ir
epochs
beta
weight decay
dropout
anchor size
47
576, 2304
200
512
0.0001
500
[0.9.099]]
0.0005
0.3
2048
Table 11 lists the details of our training setup. We use an Adam optimizer.
21

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We accurately summarized the problem setting, the theoretical contributions and empirical results of this paper in the abstract.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
For example, we discussed how the choice of a certain hyperparameter (de- gree of Legendre polynomials) needs to a trade-off between model performance and computational efficiency.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
22

---

Answer: [Yes]
Justification: In Section and Section 4, we gave clear mathematical derivations for each equation introduced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provided model architecture illustrations, hyperparameter choices and data processing methods for reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
23

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We include the code and access to open-source data in the supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ httpppscccppullccguuudddeo Coaee///ddeSebmmmcma ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We included the details in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Statistical significance measures do not apply to the experiments conducted in this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
This is reported in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We have carefully read the NeurIPS Code of Ethics and fully complied with it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of this algorithmic work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
25

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper is an algorithmic work on long-standing open-source datasets. There is no risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All creators properly eredited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
26

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We have well documented the codes we provided.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve any crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve any crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
27

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This paper does not contain any LLM contents in any forms.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy fattps://neurigg: /2tp://///////ooeeeeeee///////// for what should or should not be described.
28


Reviews:
[{"review_id": "ufqoxm5Ozf", "reviewer": "Reviewer_Yknb", "summary": "LocDiff is a multi-scale latent diffusion model for image geolocalization, addressing limitations of existing methods that struggle with spatial generalizability and multi-scale information. The model introduces a novel Spherical Harmonics Dirac Delta (SHDD) Representation framework for encoding and decoding geolocations, which tackles issues like sparsity and non-linearity in the encoding space. A Conditional Siren-UNet (CS-UNet) architecture is proposed to learn the image-based conditional backward diffusion process within this SHDD space.", "questions": "I suggest the authors add some pseudo-code to help with implementation + understanding as the very heavy math approach can be hard to parse for a part of the community.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## Strengths\n\n* **Well-motivated from a theoretical point of view:** The paper presents a strong theoretical foundation for its approach.\n* **Strong experiments:** The experimental setup is robust, and the **generalization experiments are particularly appreciated**, demonstrating the model's performance beyond training distributions.\n\n## Weaknesses\n\n* **Prediction Averaging Strategy:** The current approach of **averaging samples for prediction might lead to incorrect predictions, especially in the presence of multiple modes**. I suggest the authors explore a strategy similar to [1] by leveraging guidance to collapse the predictions, which could also lead to faster sampling as only a single point would be needed instead of 16.\n* **Benchmark Contamination and Data Source:** As highlighted by the authors, MP16 suffers from contamination issues with the benchmarks. It would be beneficial to **train on OSV-5M, which guarantees a 1km border to test samples**, making comparisons to [1] more comparable. Furthermore, the different data source (not YFCC) makes all benchmarks inherently different from the training set.\n* **Proof of SHDD Space Efficacy:** The main claim that the **SHDD space is responsible for improved fine-grain geolocalization is not sufficiently proven**. The comparison is primarily against [1], which has entirely different design choices, training steps, and datasets. **Ablation studies are crucial here**, perhaps training SHDD space diffusion with [1]'s training setup, and vice-versa, training [1]'s architecture on LocDiff's setup.\n* **Evaluation of Generative Aspect:** The paper lacks an evaluation of the model's generative aspect. **[1] proposes specific benchmarks for probabilistic geolocalization that should be utilized**. It's important to check whether the model achieves better probabilistic modeling or if it simply collapses to improve geolocalization performance.\n* **Lack of Samples:** The paper would greatly benefit from the inclusion of **visual samples** to demonstrate the model's performance qualitatively. Could make the paper more \"fun\" to read as well.\n* **Inference Efficiency Claim:** The claim of being more inference efficient than [1] due to the Riemannian framework is questionable. The authors sample with 200 steps, whereas [1] achieves maximum performance with 16 steps. Additionally, going from 3 dimensions to 4096 dimensions should drastically impact sampling (and training) time. This claim seems incorrect. **A proper benchmark for inference and training time would be highly valuable** to substantiate this claim.\n\nI put weak accept for now as i think this paper would benefit from adressing the previous concerns, but i find this paper to be good and will likely increase my rating depending of the authors response.\n\n[1] Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation, Dufour et al, CVPR 2025", "was_revised": true, "final_justification": "Overall, i think the authors have answered most of my concerns or will answered them in the camera ready. I therefore will increase my rating to a 5 and recommend accepting this paper! The authors have been very detailed in their experiments and the idea is nice!", "rebuttal": "Dear reviewer,\n\nThank you for your comments and suggestions! We will address your concerns in the following responses, respectively.\n\n**R1. Prediction Averaging Strategy**: We wish to first clarify the way we average samples for prediction, which may be a bit unclear in the original manuscript (Section 5.1). We do **NOT** average multiple locations from one DDPM sampling; instead, we run multiple DDPM samplings given different **augmentations** of the test image, use the highest mode in each distribution as the prediction, and average the predictions. This is a successful strategy proven by GeoCLIP[2] and SimCLR[3]. For example, in our experiments, without this averaging step, the 1km accuracy of GeoCLIP on Im2GPS3K may drop from 14% to below 10%. We will improve this part of writing in our camera-ready version.\n\nThen we wish to recognize that the strategy of strengthening guidance to collapse predictions sounds very promising – especially considering it may potentially reduce the Legendre polynomial degree needed to achieve the desired spatial resolution by forcing the probability mass to concentrate (of course, trading off the global coverage). However, as the reviewer has pointed out, the training framework of our method is very different from [1] (e.g., we do not have a hyperparameter to control the guidance strength). To adopt this strategy requires re-designing the pipeline and training a diffusion model from scratch, which is unfortunately not possible within the rebuttal period. We will implement this and report the corresponding results in our camera-ready version.\n\n**R2. Benchmark Contamination and Data Source**: Sorry for the confusion. The current results reported in Table 2 are based on models trained and evaluated on OSV-5M/YFCC, respectively, following the experimental setup in [1]. Thus our results should be directly comparable to [1].\n\n**R3. Proof of SHDD Space Efficacy**: This is a very helpful suggestion. In fact, when we designed our framework, we were aware that the generative process can be latent diffusion, flow-matching, or even autoregressive models, as long as it operates in the SHDD space. As is stated in our response R1, due to time constraints, we will add this part of ablation studies in our camera-ready version.\n\n**R4. Evaluation of Generative Aspect**: As is shown in our Eq. 9 (Line 219), the training objective of our method is actually one kind of probabilistic metrics, i.e., the KL-divergence from the predicted distribution to the ground-truth. It is evidence that our method is not simply collapsing the probabilistic space to improve geolocalization performance. Since we do not have the results for iNat21, we only report the NLL metrics on OSV-5M and YFCC, following the setup of Table 2 in [1].\n\n| | OSV-5M | YFCC |\n|------|------|------|\n|Diff $\\mathbb{R}^3$| 0.58 | 0.63 |\n|FM $\\mathbb{R}^3$| -5.01 | -7.15 |\n|RFM $\\mathcal{S}^2$|-1.51|-3.71|\n|LocDiff $L=47$|-2.23 |-4.12|\n\n*Rebuttal Table 0: Probabilistic Metrics*\n\nWe thank the reviewer for pointing us to the generative metrics proposed in [1], and we will include a complete table of generative metrics in our camera-ready version.\n\n**R5. Lack of Samples**: Thank you for this suggestion. Per the rebuttal rules this year, we can not upload any PDF or links to visual examples, but we will include more illustrative examples in our camera-ready version.\n\n**R6. Inference Efficiency Claim**: First, we wish to recognize that the statement about the inference efficiency in our introduction (Line 58-60) is inaccurate. What we intended to say is that the re-projection in the non-Riemannian framework introduces extra operations in each noise-adding/removing step and it is more computationally expensive than conventional diffusion/flow-matching in the Euclidean space. We will correct this error in the camera-ready version.\n\n|Steps|Device|Time Efficiency|\n|------|------|------------|\n|Encoding|CPU|0.0003s per location|\n|DDPM|GPU|0.056s per image (200 steps, 16 augmentations)|\n|Decoding|GPU|0.0012s per batch (512)|\n\n*Rebuttal Table 1: SHDD Encoding/Decoding Time Efficiency*\n\nRebuttal Table 1 gives an intuitive sense of our time efficiency. Many of the baselines have not reported their inference time, but as an example, we know that the inference time per image of GeoCLIP is 0.024s. We wish to briefly discuss the factors that truly affect our training/inference efficiency.\n\nFirst, we wish to clarify that the 200-step sampling reported in our paper is DDPM to match our training setup, while the 16-step sampling in [1] is DDIM. It is possible to reduce our sampling steps to 16 using DDIM.\n\nThen, we wish to clarify the misunderstanding about dimensionality. Both [1] and our method use 3-$d$ (Euclidean xyz) or 2-$d$ (spherical longitude and latitude) as model input. The difference is: for [1], the width of the network is 4x512=2048 (see Figure B. in [1]); for our method, the width of the network is 2x$d$ (see Figure 2 in our paper), where $d$ is the **deterministic** SHDD encoder dimension, ranging from 256, 576 to 1024, i.e., the maximum network width is the same (2x1024=2048) as [1]. We do not scale up to 4096 dimensions, as is discussed in Appendix A.4. \n\nFinally, we wish to emphasize that using an SHDD encoded multi-scale representation helps the diffusion process to converge faster, because each dimension encodes information at the respective spatial scales. For example, if at a certain step, the generated SHDD representation is 500km within the ground-truth, then the dimensions with larger than 500km scales will remain mostly intact, and only the finer dimensions need to be altered. In practice, our model takes about 2 million steps to converge on YFCC, while the best results reported in [1] take 10 million steps. However, as [1] indicates that FM $\\mathbb{R}^3$ outperforms Diff $\\mathbb{R}^3$ at some scales, it is worth combining the flow-matching framework with our SHDD multi-scale representation, which may result in even faster convergence and better performance.\n\n**R7. Pseudo Code for Readers**: This is a very important suggestion. We will add a pseudo code block in the camera-ready version.\n\n[1] Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation, *Dufour et al, CVPR 2025*\n\n[2] GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization. *Vivanco Cepeda et al*. *NeurIPS 2023*.\n\n[3] A Simple Framework for Contrastive Learning of Visual Representations. *Chen et al.*. *ICML 2020*."}, {"review_id": "bDOBZEKTuD", "reviewer": "Reviewer_xaf8", "summary": "This paper presents a generative approach for global-scale image geolocalization, which aims to infer an image's geographic location on the Earth's surface. The authors introduce a novel location encoding-decoding strategy, termed Spherical Harmonics Dirac Delta (SHDD) representations, coupled with a UNet-based diffusion framework. This design addresses key limitations in previous methods, including distributional misalignment and insufficient multi-scale context modeling. Extensive experiments across five global-scale geolocalization datasets demonstrate the effectiveness of the proposed approach. Moreover, the method exhibits strong generalization to regions that were not seen during training.", "questions": "See comments", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Pros:\n- The motivation is clear and convincing. Representing the embedding space with Spherical Harmonics is a natural solution for tasks involving the global mapping on the Earth surface. Moreover, the problems/challenges of using the Spherical Harmonics for diffusion models are clearly identified in this paper.\n- This paper proposes a novel pipeline for image geo-localization combining the Spherical Harmonics Dirac Delta (SHDD) Representations and a latent diffusion model. Although the spherical harmonics has been explored in some geo-based tasks like the general location embedding design and geo-spatial foundation models.It is interesting to see its adaptation into the image geo-localization task.\n- Stronger performance. Experiments on 5 datasets demonstrate the stronger performance of the proposed method. Moreover, it is interesting to see that the SHDD significantly increases the generalization ability.\n\nCons:\n- My primary concern is the potential computational overhead introduced by both the spherical harmonics basis and the diffusion process. These components are likely to significantly increase the model's complexity and training cost. However, the main paper lacks a comparison of computational efficiency metrics (e.g., GFLOPs, model size, training/inference time) between the proposed method and baseline methods.\n- Given the paper’s focus on location representations, a more comprehensive comparison with alternative methods is expected. Although two methods (RBF and Sphere2Vec) are briefly compared in Appendix Table 6, the performances of other representations mentioned in Section A.2 are not reported. Furthermore, it seems that RBF and Sphere2Vec appear to collapse in performance, it is not clear if they are implemented properly. \n- The introduction of the Dirac delta function is a crucial step in adapting spherical harmonics to represent specific Earth locations. A clearer explanation of how similarity is computed (e.g., if there is any closed-form expression?), would greatly enhance the reader’s understanding of the proposed mechanism. \n- Finally, the method section is overall not easy to follow, especially considering that this paper focuses on the image geo-localization, while it is sometimes hard to connect the variables and notions in the SHDD modeling with the image geo-localization task. For example, in line 214, what do the spherical function F and \\delta denote, an image or the location embedding? and in line 216, what is p(\\theta, \\phi) and q_e respectively in the context of geo-localization?\n\nMinor issue: \n- The metrics used in Tables 1 and 2 are not defined. Are they top-1 accuracy (ACC), or some other measure? A brief description in the captions or main text would be helpful.", "was_revised": true, "final_justification": "My concerns have been addressed; overall, this is an interesting paper with solid results, so I raise my rating to 5.", "rebuttal": "Dear reviewer,\n\nThank you for your comments and suggestions! We will address your concerns in the following responses, respectively.\n\n|Steps|Device|Time Efficiency|\n|------|------|------------|\n|Encoding|CPU|0.0003s per location|\n|DDPM|GPU|0.056s per image (200 steps, 16 augmentations)|\n|Decoding|GPU|0.0012s per batch (512)|\n\n*Rebuttal Table 1: SHDD Encoding/Decoding Time Efficiency*\n\n**R1. Potential computational overhead**: The SHDD encoding/decoding are both deterministic, closed-form operators and do not incur huge computational cost. For example, the $d$-dimensional ($d=256, 576, 1024$) SHDD encoding of a point is just a *parallel* evaluation of a list of two-variable functions, whose time complexity is $O(1)$ and space complexity is $O(d)$. Not to mention that during training, the SHDD encodings can be precomputed and used as an embedding lookup table. In practice, constructing such a lookup table of 4 million training samples sequentially on CPU only takes 18 minutes in the dataloading stage, and there is no further computational overhead afterwards. SHDD decoding is implemented as a matrix multiplication followed by an argmax operation, which is also very efficient under PyTorch. Please see Rebuttal Table 1.\n\nAs for the diffusion process, we used a standard latent diffusion framework and a tailored CS-UNet model, which is much smaller than common CNN-based UNets and runs way faster. The training/inference time on a 600,000 subset of MP16 is reported in Appendix A.7. As an example of the baseline efficiency, the GeoCLIP model takes 227s per training epoch (v.s. Ours 388s) and 0.024s per image inference (v.s. Ours 0.056s). Due to the time constraints of the rebuttal, we can not reproduce all the baselines cited in our paper and report their time efficiency, but we will add these results to our camera-ready version.\n\n**R2. Other representations**: We wish to clarify that the location encoding methods introduced in A.2 and the implementations of RBF and Sphere2Vec are both based on [4], the most recent and commonly used location encoding benchmark. While not fully reported in our Appendix, RBF and Sphere2Vec are the best-performing location encoders in their respective genres (RBF is a 2-D location encoder, while Sphere2Vec is a 3-D location encoder; see Table 1 in [4]). We will extend Appendix A.8.1 Table 6 and Table 7 to include the results of all representation methods in our camera-ready version.\n\nAppendix A.8.1 Table 6 and Table 7 shows that the collapse in performance is a result of large perturbation drifts during decoding – traditional location encoding methods use non-linear layers to up-project coordinates and make decoding very inaccurate. As we clarified, the RBF and Sphere2Vec implementations both come from the official codebase of TorchSpatial, which is published in NeurIPS 2024 and widely used in other research projects. It is most likely properly implemented.\n\n**R3. Similarity of Dirac delta functions**: Spherical Dirac delta functions can be seen as probability distributions on the sphere. Thus the natural similarity measurement is the statistical distance between distributions, such as the KL-divergence that we adopt in our paper. Please refer to Eq. 9 where we give the closed-form expression the reviewer requested for.\n\n**R4. Explaining notations**: Please allow us to clarify the notations.\n\n* (1) Spherical function $F$ and $\\delta$ in Line 214. As we introduced in Section 3, any functions defined on a sphere can be uniquely represented as spherical harmonics coefficients. We denote such an arbitrary function as $F$. Among these functions, there is a special type of functions called *spherical Dirac delta functions*, which we denote as $\\delta$, and they uniquely correspond to *points* on the sphere. The two notations tightly connect to image geo-location in the following way: to identify the geo-location (i.e., a point on the sphere) can be seen as finding the spherical Dirac delta function $\\delta$ that corresponds to the point. A diffusion process is effectively altering a random guess (i.e., an arbitrary function) $F$ step by step to approximate the spherical Dirac delta function $\\delta$. In this sense, both $F$ and $\\delta$ are location embeddings.\n\n* (2) We propose to use KL-divergence to measure the similarity between $F$ (our diffusion prediction) and $\\delta$ (the ground truth), which requires both $F$ and $\\delta$ to be probability distributions. Therefore, we normalize $F$ into $q_e$ and $\\delta$ into $p(\\theta, \\phi)$, so that we can compute their KL-divergence and use it as the loss for back-propogation. In the context of geo-localization, $p(\\theta, \\phi)$ is the ground-truth probability of observing a certain image, while $q_e$ is the guess from the diffusion model.\n\n**R5. Metrics in Table 1 and Table 2**: The metrics are the threshold-based accuracy of geo-localization, i.e., the percentage of model predictions that are less than $x$ km away from the ground-truth, $x=1, 25, 200, 750, 2500$, respectively. This is the conventional metric for geo-localization [5]. Thank you for the suggestion! We will update the captions in our camera-ready version.\n\n[4] TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning. *Wu et al.*, *NeurIPS 2024*.\n\n[5] GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization. *Vivanco Cepeda et al*. *NeurIPS 2023*."}, {"review_id": "YedDAgrCjy", "reviewer": "Reviewer_qNuj", "summary": "This paper addresses the challenge of determining where a photo was taken. It introduces LocDiff, leveraging diffusion models to enhance geolocation accuracy and scalability.", "questions": "1. What is the rationale behind selecting L=47 as the maximum degree for spherical harmonics? Did you explore higher degrees beyond L=47, and if so, what were the results? How does this choice balance spatial resolution with computational feasibility?\n\n2. The paper uses the MP16 dataset for training, which may introduce a bias towards certain geographic regions or image types. How does this affect the generalizability of your model? Have you tested its performance on geographically diverse datasets?\n\n3. The SHDD encoding dimension grows quadratically with L, leading to high computational costs. How does this scale in practice? Have you evaluated the trade-offs between spatial resolution and computational efficiency for larger values of L?\n\n4. The hybrid approach (LocDiff-H) relies on a gallery of candidate locations for retrieval. How sensitive is the model's performance to the choice and distribution of these gallery points? What happens if the gallery does not contain locations similar to those in the test set?\n\n5. The ablation studies focus on different encoders and losses but do not compare against other state-of-the-art methods like PIGEON. How does your method perform in comparison to these models?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nQuality:\n\nThe paper proposed a novel encoding framework called Spherical Harmonics Dirac Delta (SHDD) with clear derivations and mathematical formulations. It also propose a novel SirenNet-based architecture (CS-UNet) to learn an image-based conditional backward process in the latent SHDD space by minimizing a latent KL-divergence loss.\n\nClarity:\n\nThe paper is generally well-written and logically structured. Key contributions, such as SHDD encoding and the hybrid LocDiff-H approach, are clearly explained.\n\nSignificance:\n\nThe introduction of the hybrid approach (LocDiff-H) is significant as it combines generative and retrieval-based methods, potentially paving the way for more robust geolocation systems.\n\nOriginality:\n\nThe introduction of SHDD KL-divergence as a loss function is innovative and aligns well with the problem's requirements.\n\nWeakness:\n\nQuality:\n\nSome hyperparameters (e.g., degree $L$) are chosen arbitrarily without a clear justification beyond empirical testing. A more systematic approach to selecting these parameters would strengthen the paper.\n\nClarity:\n\nSome sections, particularly those involving mathematical derivations (e.g., spherical harmonics computation), could be more accessible to a broader audience. Simplifying or visually representing these concepts might aid comprehension.\n\nSignificance:\n\nWhile the paper demonstrates strong empirical results, its societal impact is limited. The authors acknowledge this by stating that their work does not have direct societal implications. However, discussing potential future applications could highlight the significance of their contribution.\n\nOriginality:\n\nWhile the paper builds on existing works (e.g., Sphere2Vec), some of its extensions, such as the hybrid approach, are not thoroughly explored. A more detailed discussion of these contributions' uniqueness compared to prior work would enhance originality.", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer,\n\nThank you for your comments and suggestions! We will address your concerns in the following responses, respectively.\n\n**R1. Responses to Weaknesses**:\n* Quality. Most hyperparameters are chosen by grid search. As to the Legendre polynomial degree $L$, it is chosen by systematically analyzing the trade-off between spatial resolution and computational stability. Please see Appendix A.4, especially Figure 5(b), for a visual explanation. $L=47$ is the maximum degree where we can still stably train the model.\n\n* Clarity. Thank you for this suggestion. Per the rebuttal rules this year, we can not upload any PDF or links to visual examples, but we will include more illustrative examples and pseudo codes to help readers follow our logic in our camera-ready version.\n\n* Significance. Thank you for this suggestion. We will add more discussions on the societal impacts of geo-localization tasks in the camera-ready version. \n\n* Originality. The hybrid approach, which is effectively a hierarchical combination of our novel method and existing retrieval-based methods, is mostly chosen to reduce the computational complexity of the solution. It does not hurt the originality of our approach. In fact, Table 3 clearly shows that our method bridges the generalizability gaps of existing methods.\n\n**R2. Rationale for L=47**: The rationale for choosing the maximum degree to be $L=47$ is mostly based on computational feasibility. We find that, while increasing $L$ does increase the spatial resolution of SHDD representations (Appendix A.4 Figure 5(a)), the computational instability also increases. Appendix A.4, especially Figure 5(b) gives a clear explanation for the choice of $L=47$: when $L$ surpasses this threshold, the absolute values of the representation surge to over $10^5$, which makes computing KL-divergence (involving exponential operations) impossible. \n\n**R3. Generalizability**: In fact, the improved generalizability is one of our key contributions. Table 3 carefully discusses how the data inductive bias of MP16 affects the performance of existing methods such as GeoCLIP, whereas our method remains highly generalizable. Experiments in Table 2 on other datasets such as OSV-5M and YFCC also prove that our method can outperform baselines in more geographically diverse scenarios.\n\n**R4. Scalability**: Appendix A.4 and A.7 present a careful analysis of the trade-off between spatial resolution and computational efficiency. In fact, the improvement of spatial resolution as $L$ goes up is gradually offset by the side effect of computational instability, which we already discussed in R1. Our strategy is to constrain $L$ below 47, and to improve the finer-scale performance by hierarchically incorporating GeoCLIP, as is discussed in Appendix A.5. In this way, we can avoid high computational costs.\n\n**R5. Inductive Bias of Gallery**: This effect is discussed in detail in Section 5.2 and Appendix A.6. In summary, the hybrid approach is sensitive to the mismatch between the candidate gallery and the ground-truth. Thus we recommend to use the hybrid approach (LocDiff-H) only if we know that the gallery is a good fit, and to use the base method (LocDiff) for general cases.\n\n**R6. Comparison to Baselines**: Please see Table 1, where PIGEON is a baseline that we compared to."}, {"review_id": "wPkW7B1Y5t", "reviewer": "Reviewer_oCqZ", "summary": "The authors propose a novel framework for image geolocalization, called LocDiff, which aims to infer the geolocation where an image was captured. LocDiff addresses the lack of multiscale information in previous approaches by leveraging latent diffusion modeling in the Hilbert space of Spherical Harmonic coefficients. This representation is both dense and easier to linearly decode back into coordinate space. The framework introduces a new architecture for the diffusion process that combines a conditional SirenNet, conditioned on the image and timestamp, with a UNet. The authors compare then LocDiff to existing geolocalization methods and show that it generally outperforms them, particularly when paired with a retrieval-based approach such as GeoClip.", "questions": "I am open to increase my rating if the authors address the concerns previously raised:\n\n1. Provide a comparison of inference time of LocDiff against the baselines.\n2. Clarify the extent to which LocDiff depends on GeoCLIP when claiming superior performance over other approaches in the introduction and abstract.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper is particularly well written, with the method described in meticulous detail.\n- The approach is sound and well justified, building on components proven effective for modeling our spherical Earth.\n- When paired with GeoCLIP, the approach outperforms baselines across multiple scales.\n- Numerous ablation studies in the Appendix demonstrate the effectiveness of the modules of LocDiff.\n\nWeaknesses:\n- Both spherical harmonics and diffusion models are known to be computationally intensive at inference time. While the authors provide an insightful analysis of computational complexity during training in Appendix A.7, it would be valuable to include results on inference time, ideally with a comparison to other methods.\n- LocDiff appears to rely on GeoCLIP (or more generally, on a retrieval-based component) to outperform other baselines on the Im2GPS3k and YFCC-26k datasets, especially at fine-grained scales. This is not a big issue *per se,* but I think the authors should acknowledge this dependency more explicitly, ideally in the introduction and possibly in the abstract, to provide a fairer representation of the method's standalone capabilities.", "was_revised": true, "final_justification": "After a great rebuttal from the authors, I have decided to improve my rating from 4 to 5.", "rebuttal": "Dear reviewer,\n\nThank you for your comments and suggestions! We will address your concerns in the following responses, respectively.\n\n|Steps|Device|Time Efficiency|\n|------|------|------------|\n|Encoding|CPU|0.0003s per location|\n|DDPM|GPU|0.056s per image (200 steps, 16 augmentations)|\n|Decoding|GPU|0.0012s per batch (512)|\n\n*Rebuttal Table 1: SHDD Encoding/Decoding Time Efficiency*\n\n**R1. Comparison of Inference Time**: Thank you for the question! We will discuss the computational costs of our method with regard to the SHDD encoding/decoding and the diffusion process, respectively.\n\nFirst of all, we wish to clarify that SHDD encoding/decoding is **NOT** a computational bottleneck. While the spherical harmonics has very complex formulas and some definitions use recursive equations, we adopt the closed-form precomputation code implemented in [6], which effectively reduces the SHDD encoding into a lookup table. For example, the $d$-dimensional ($d=256, 576, 1024$) SHDD encoding of a point is just a *parallel* evaluation of a list of two-variable functions, whose time complexity is $O(1)$ and space complexity is $O(d)$. In practice, constructing such a lookup table of 4 million training samples sequentially on CPU only takes 18 minutes in the dataloading stage, and there is no further computational overhead afterwards. SHDD decoding is implemented as a matrix multiplication followed by an argmax operation, which is also very efficient under PyTorch. Please see Rebuttal Table 1. \n\nSecondly, we wish to clarify the major factors that affect the inference speed. In Appendix A.7., we reported the inference time of our method and GeoCLIP: 0.056s (ours) v.s. 0.024s (GeoCLIP). In practice, both our method and GeoCLIP used pretrained image embeddings of CLIP, i.e., we are not actively running a CLIP model to encode an image into an embedding. This step is the **true bottleneck**. On our 24GB GPU machine, processing one 1024x1024 image through CLIP takes 0.15s, way slower than the diffusion inference process. It is a similar situation with other baselines that uses pretrained image encoders. \n\nOther than that, the current inference setting is to run a 200-step DDPM sampling 16 times and get the ensemble prediction. It is possible to reduce the total inference time by (1) using DDIM to reduce the sampling steps, (2) incorporating the prediction collapsing strategy in [7] to reduce the number of ensembles. We will explore these possibilities in future work.\n\n**R2. Clarification of LocDiff-H**: Thank you for your suggestion! We will modify the abstract and the introduction to explicitly acknowledge in what scenarios and to what extent our SOTA results rely on GeoCLIP in the camera-ready version.\n\n[6] Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks. *Rußwurm et al*. *ICLR 2025*.\n[7] Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation, *Dufour et al, CVPR 2025*"}]

Author rebuttal:

Meta-review: The paper proposes LocDiff, a method for image geolocalization based on a multi-scale latent diffusion model. The authors introduce a new encoding-decoding strategy called Spherical Harmonics Dirac Delta (SHDD), addressing multi-scale context modeling and distributional misalignment. The experimental analysis is thorough and broad, including generalization tests. The reported results are solid and the rebuttal provides further clarifications and additional information that can be included in the appendices to further strengthen the content presentation. Finally, the reviewers have all recognized the quality and contributions of this work. I recommend it for publication.
Decision: Accept (poster)