Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks
Authors: Marc Rußwurm
Konstantin Klemmer
Esther Rolf
Robin Zbinden
Devis Tuia
Venue: ICLR 2024 spotlight (2024)
Keywords: Location Encoding
Positional Encoding
Implicit Neural Representations
Species Distribution Modeling
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Learning representations of geographical space is vital for any machine learning model that integrates geolocated data, spanning application domains such as remote sensing, ecology, or epidemiology. Recent work embeds coordinates using sine and cosine projections based on Double Fourier Sphere (DFS) features. These embeddings assume a rectangular data domain even on global data, which can lead to artifacts, especially at the poles. At the same time, little attention has been paid to the exact design of the neural network architectures with which these functional embeddings are combined. This work proposes a novel location encoder for globally distributed geographic data that combines spherical harmonic basis functions, natively defined on spherical surfaces, with sinusoidal representation networks (SirenNets) that can be interpreted as learned Double Fourier Sphere embedding. We systematically evaluate positional embeddings and neural network architectures across various benchmarks and synthetic evaluation datasets. In contrast to previous approaches that require the combination of both positional encoding and neural networks to learn meaningful representations, we show that both spherical harmonics and sinusoidal representation networks are competitive on their own but set state-of-the-art performances across tasks when combined. The model code and experiments are available at https://github.com/marccoru/locationencoder.

Published as a conference paper at ICLR 2024
GEOGRAPHIC LOCATION ENCODING WITH SPHERICAL HARMONICS AND SINUSOIDAL REPRESENTATION NETWORKS
Marc Rußwurm"
Konstantin Klemmer
Wageningen University
Microsoft Research New England
Laboratory of Geo-information Science and Remote Sensing
Esther Rolf
Robin Zbinden & Devis Tuia Environmental Computational Science and Earth Observation Laboratory (ECEO) École Polytechnique Fédérale de Lausanne (EPFL)
Harvard Data Science Initiative and Center for Research on Computation and Society University of Colorado, Boulder
ABSTRACT
Learning representations of geographical space is vital for any machine learn- ing model that integrates geolocated data, spanning application domains such remote sensing, ecology, epidemiology. Recent work embeds coordinates using sine and cosine projections based on Double Fourier Sphere (DFS) fea- tures. These embeddings assume rectangular data domain even on global data, which can lead to artifacts, especially at the poles. At the same time, little at- tention has been paid the exact design of the neural network architectures with which these functional embeddings are combined. This work proposes a novel location encoder for globally distributed geographic data that combines spher- ical harmonic basis functions, natively defined on spherical surfaces, with si- nusoidal representation networks (SirenNets) that can be interpreted as learned Double Fourier Sphere embedding. We systematically evaluate positional embed- dings and neural network architectures across various benchmarks and synthetic evaluation datasets. In contrast to previous approaches that require the combi- nation both positional encoding and neural networks to learn meaningful rep- resentations, show that both spherical harmonics and sinusoidal representa- tion networks are competitive on their own but set state-of-the-arr performances across tasks when combined. The model code and experiments are available wtt.a//www.........o//ooaaaoaaaa
INTRODUCTION
Location information is informative meta-data in many geospatial application,,,, ranging from species distribution modeling (Mac Aodha al., 2019 Cole et al. (023) and wildlife monitoring (Jeantet & Dufourq 2023) to solar irradiance forecasting (Boussif et al., 2023), global tree height prediction (Lang et al. 2022] or the estimation of housing prices (Klemmer et al. 2023). Coordinates also serve task task- descrritieee feature for meta- meta-learning across geographically distributed tasks (Tseng cct al.] (202)) or as input for geographic question answering (Mai et al.] 2220a).. Location encoders alone be used to learn implicit neural representation of a dataset (Stanley 2007), for example, to compress weather data (Huang & Hoefler 2023) or to learn an implicit neural representation of species distribution maps (Cole et al. 2023).
Methodologically, predictive modeling with spatial coordinates as inputs has a long history and classically done with non-parametric approaches, like Gaussian Processes (GP), i.e., Kriging, OOliver & Webster 1990] Matheron] 11699) or kernel methods in general. For instance, Berg al. (2014) proposed adaptive kernels for bird species recognition, where class-speeific kernel is learned for each species separately. Relying on spatially close training points is intuitive and
"corresponding author. Please email farc. russworm@wur.

---

Published as a conference paper at ICLR 2024
highly effective for interpolation tasks and small datasets. Still, it becomes computationally expen- sive when kernel similarities have to be computed between many test points and large training sets. And while research in scaling kernel methods to higher-dimension
data has progressed (Gardner let al. 2018), recent focus on scalable methods has shifted to using neural network weights to ap- proximate the training points (Tang et al., 2015) Mac Aodha al.) 2019 Chu al.) 2019 Huang & Hoefler 2023). Inspired by Double Fourier Sphere (DFS) (Orszag) 1974) and the success of po- sitional encodings in transformers, a series of works developed and compared different sine-cosine embedding functions at different spatial scales (frequencies) where the number of scales controls the smoothness of the interpolation (Mai al.) 2020b] 2023b). Even though these DFS-inspired approaches have proven effective, they still assume a rectangular domain of longitude and latitude coordinates, which does not accurately reflect our planet's spherical geometry.
In this work, we instead propose to use spherical harmonic basis functions as positional embeddings that are well-defined across the entire globe, including the poles. particular, the scientific use of location encoders requires good global performance, with poles being especially important for climate science. Methodologically, we build upon recent work here has been focused on learning functions general manifolds (Koestler et al. 2022) and graphs (Grattarola & Vandergheynst, 2022] in non-euclidean geometries. This work focuses on spheres, a special case of non-euclidean geometries, where utilize spherical harmonic basis functions, which are the eigenfunctions of the Laplace-Beltrann operator on the sphere (Koestler et al. 2022)
In summary, we
1. propose Spherical Harmonic (SH) coordinate embeddings that work well with all tested neural networks, and especially well when paired with Sinusoidal Representation Networks (SirenNet) (Sitzmann et al. 2020), which was untested for location encoding previously;
2. show by analytic comparison that SirenNets can be seen as learned Double Fourier Sphere embedding, which explains its good performance even with direct latitude/longitude inputs;
3. provide a comprehensive comparative study using the cross-product of positional embed- ding strategies and neural networks across diverse synthetic and real-world prediction tasks, including interpolating ERAS climate data and species distribution modeling.
2 METHOD
Nx
4 Reslayee
2.1 LOCATION ENCODING
Lin o Drup o Sin
We encode global geographic coordinate longitude
PE(A, d))
Line ReLU
Lin ReL
Dropout
Lio o KeLU
Linear
PE(A, o)
Lincar
A e [-T, π] and latitude o E |- T T/2] to predict a target variable y, which can be classification logit, a regression value, or a vector that can be integrated into another neural network, such as a vision encoder.
(a) FCNET
(b) SIRENNET
Following the convention in Mai et al. 2022). these lo- cation encoding approaches generally take the form of
Figure 1: Overview over neural networks
(SIRENNET [Sizzmann et al. (2000) and
NN (PE (X, o))
(1) FCNET Mac Aodha al. (2019)
that combines a non-parametric positional embedding (PE) with a neural network (NN). Most previ- ous work focuses on the positional encoding, typically linked together with a fully connected resid- ual network (FCNET, see fig. la This FCNET implementation with four residual layers (ResLayer) was originally proposed in Mac Aodha al. (2019) for location encoding, and has been re- us-used as-
is in follow-up work (Mai et al. 2023b) 2020b Jeanter & Dufourq 2023 Mai et al. 2023a Cole
al. 2023). However, similar networks have been proposed earlier for pose estimation Martinez ctt al. (2017). Here, we discuss the main families of positional embedding approaches, deferring detailed description to appendix [A.2 We then describe our approach for a novel global positional
encoder, linked with SIRENNET (fig. [b). network which broadly used for learning implicit neural representations of images (Mildenhall et al. 2021) but has not been tested in this context of location encoding.
DFS-based embeddings. In their foundational study, Aodha et al. (0119) use a simple WRAP positional encoding PE(A, Φ) [cos X, sin X, cos db sin Φ), which removed the discontinuity the
2

---

Published as a conference paper at ICLR 2024
LINEAR "Neural Network"
Linear(SHi SH)
output
Sir)n(SH)
Σ
1=0
L=5
L 10
L 20
(b) Left to right: increasing the number of Legendre polynomials L increases the resolveable resolution by adding additional high frequency orders and de- grees Using a more complex neural network, e.g., 2-layer SIRENNET (bottom row), further increases the model's ability to resolve fine-grained detail.
m
SPHERICAL HARMONICS
(a) visual decomposition of LINEAR (SH (SH(A))) 6)) location encoder, as weighted sum in eq.
Figure 2: SPHERICAL HARMONICS are orthogonal basis functions defined on spheres. Spherical functions can be naturally defined as the sum of peraheter-weighting basis functions, as done in a linear layer (panel (a)). In this work, we propose location encoder (PF(P Φ)) with spherical harmonic positional embedding (PE) function with the SIRENNET neural network (NN), ie.. SIREN(SH), to learn complex functions defined on the globe, as illustrated in the example of land-ocean classification in the right panel.
dateline (A x). The original wrap positional encoding has since been identified as special case of Double Fourier Sphere (DFS) (Orszag 1974) coefficients by Mai et al. (2023b):
S-1
DFS Φ)
U
sin On, cos Φn)
U m==
ssin Am cos \m
(2)
S-1 S-1
U
U
[cos Φn cos Am cos on sin Am sin on cos Am sin on sin Am
22 ==0 m=0
where the coordinates Φ, À are introduced across multiple frequencies X₂ A and م by a Y that allows users specify the minimum and maximum radii Tmin maax that can be resolved at multiple scales up to S. The union operator U indicates the concatenation of these functions into one large vector.
scale-dependent factor a, Tmin"
Tims
Pun
The full DFS embedding is computationally expensive and quantitatively less accurate than the special cases SPHEREM and SPHEREC (Mai et al. (023b)). Hence, Mai et al. (2023b) proposed SPHEREM+ and SPHEREC+, which is a combination with the GRID embedding function of Mai let al.] (2020b). In the same paper, Mai al. (2020b) proposed use hexagonal basis functions rather than rectangular grid cells, which is the THEORY embedding function that we will compare in the results section.
Baseline embeddings. DIRECT, i.e., using the identity function to encode (Ø, A), or CARTE- SIAN3D, which correponds encoding (o. A) into (x, v, coordinates, are also common baselines still used in recent work (Tseng et al. 2022).
Our Approach. We depart from DFS and instead propose use SPHERICAL HARMONIC (SH) basis functions (detailed in section (.2). which have a long tradition in the Earth sciences for rep- resenting the gravity field (Pail et al. (011) or the magnetosphere of planets (Smith et al.) 1980). The SH basis functions can be multiplied with learnable coefficient weights. Examples of SH basis functions and their corresponding learnable triangle weight matrix are shown in fig. 2a The weight matrix can be expressed as a linear layer, i.e., very simple "neural network" following the location encoding terminology of eq. (T) It can also be combined with more complex neural networks; we propose to use Sinusoidal Representation Networks (SIrENNEt) Sitzmann et al. fig. which are architecturally simpler than the FCNET (fig. Tap used in related works. We com- pare SIT with DFS encodings used in previous location encoding approaches in section 2.3
in
3

---

Published as a conference paper at ICLR 2024
2.2
WEIGHTED SPHERICAL HARMONICS
Any function on a sphere
f() = -0 Σ
w/ w Y7 (), (7m Ø)
1=0 m=-1
with surface points defined by (lat, lon) (d. A) can be expressed by the w/"h weighted sum of orthogonal spherical harmonic basis functions Y of increasingly higher-frequene degrees and orders m. In practice, we choose a maximum number of Legendre polynomials L (instead of co harmonic. To illustrate, we draw examples of harmonics up to L = 3 in fig. 2a (bottom left) alongside their coefficients (top left), which combined approximate the landmass outline of our planet (right) with only nine trainable weights. Each individual harmonic
in eq. (3) approximate spherical function up the highest-frequeet
21 + (1- |ml)! (cos X)eims
Y (A, Φ)
4x (1+ mml))
can be calculated with associated Legendre polynomial Pr This polynomial can be analytically constructed by repeated differentiation P (z) (-1)" z²) dr (P((x)) of their zero-order
(m 0) Legendre polynomials P. P. Zero-order polynomials a certain degree I are similarly
obtained by repeated differentiation P((x)= 2 de [(x² --']. To obtain associated polynomials
d'
(1-m)l PI" (a)) can be used. of negative order (m < 0), symmetries P,- mh (x) = (-1)" (1+m))
We are interested in the real form of the original complex eq.
which can be implemented as
if m < 0 (-1) /2 (cos A) sin( sin(()))), Pi (cos X), (- PIm A) cos()))) Ytm (M, 6)
if 0
(5)
if > 0
with cosine and sine functions, where P.m (cos A)
21+1 (1- ml
4m (1+ml)) P (cos X) denotes the normal-
ized associated Legendre polynomial. The polynomials can be computed iteratively in closed- form
(z) = (-1)ah 2 (1 - =²)² Ek-m A = (x) [k-m!]] However, pre-computing the polynomials analytically is more computationally efficient than using the closed form, as show empirically later in the results section. We this analytically pre-
following Green 2003
computed implementation in experiments throughout the paper.
2.3 SINUSOIDAL REPRESENTATION NETWORKS AS LEARNED DES EMBEDDINGS
Besides SPHERICAL HARMONICS, we propose to use Sinusoidal Representation Networks (SIRENNETS) (Sitzmann et al., 2020) as a preferred neural network for location encoding. We now show that SIRENNETS can be seen as a form of learned Double Fourier Sphere (DFS) positional embedding.
We use the GRID embedding function by Mai et al. (2023b) as a simple example, but the follow- ing comparison can also be made for other DFS embeddings. GRID is defined as: GRID(A, Φ)
U$-1 (cos sin As; cos dh, sin Φ.] -.51 cos( A sin( (2) cos( (.) sin( (.) where as is a scaling factor that induces increasingly high frequencies through scales from 0 to S and U indicates concatenation. With the 90-degree phase shift between sine and cosine, i.e., cos(B) = sin(f + )), we can rewrite GRID solely using sine functions
GRID(A, Φ)
S-1 U ==0
sin a2
Th "), sin( ) sin +
② + π as
") sin at
|]
(6)
a,
where every block of four feature dimensions (indexed by s) is scaled by the same constant factor and every odd feature embedding is phase shifted 3+
4

---

Published as a conference paper at ICLR 2024
Similarly, simple 1-layer SIRENNET can be written out SIRENNET(O, sin(W[ ^]T + b) finww$ + w b) with W [w*, w*] E R 1H i b H trainable weights and H hidden output dimensions. Writing out the vector-matrix multiplication as concatenated scalars results in SIRENNET(A, Φ) = U [sin( ( + " 0 + 6).. If we now set every W wh+1 - 0 and every 2 h = O, we similarly alternate between longitudinal and latitudinal scaling for each feature. we now also set bb+1 = bn+3 = O, we arrive at a GRID-like SIREN variant
H
GRIDSIREN(A,O) σ) -
U h=0,1,...
[ X + bn), sin(w, +1 A), sin(w + (h+2), ( (7)
which corresponds to eq. (6) if by bb+2 = 2 and all W = w6++ = wh+2 wh+3 Hence, a 1-layer SIRENNET with these hard-coded weights equals the GRID DFS encoding, where the longitudes and latitudes are manually scaled to produce lower or higher frequencies in each embedding dimension. Crucially, these scaling factors are learned in GRIDSIREN through gradient descent, while they are fixed for every block of four in the GRID embedding in eq. (6) This ob- servation highlights that SIRENNET blurs the distinction between Positional Embedding (PE) and Neural Network (NN) from eq. (T) and analytically explains the good performance of SIRENNET with simple DIRECT embedding functions, as we will show later.
3 DATASETS AND EXPERIMENTS
The experiments in this work span from controllable synthetic problems to real-world data. Across all datasets, the location encoder model is fitted on training data to learn an underlying signal de- fined by the training label (probability map regression value). How well the signal has been approximated in the network weights is measured on spatially different test points.
3.1 CHECKERBOARD CLASSIFICATION: CONTROLLABLE SYNTHETIC TOy DATASET
A Fibonacci lattice mathematical idcalization of natural patterns on sphere with optimal pack- ing, where the surface area represented by each point is almost identical (González) 2010) within an error of 2% (Swinbank & James Purser 2006). is calculated for 2N points (i ranges from -N to N) Φ; aresin 2N+I and A 2 with the Golden Ratio + (1+ v5) & 1.618. To make choose 100 points according the Fibonacci-latiin and assign
this a classification problem, each point one of 16 classes in a regular order (i modulo 16). In later experiments, we increase or decrease the number of grid cells to vary the spatial scale and measure the maximum resolvable resolution of location encoders. For the test dataset (fig. 3a we sample a second Fibonacci-lattice grid with 10 000 points and assign each point the class of the closest label point using the spheri- cal Haversine distance. For training and validation sets, we uniformly sample 10 000 points on the sphere and similarly assign the label of the closest labeled point.
3.2 LAND-OCEAN CLASSIFICATION: IMPLICIT NEURAL REPRESENTATION OF LAND
We also design more realistic land-ocean classification dataset shown in fig. 3b This dataset tests the encoder's s ability to learn patterns across different scales and resolutions, while providing training and evaluation points across the entire globe. For training and validation data, sample 5000 points uniformly on the sphere's 's surface and assign a positive label for land and negative label for water depending on whether they are within landmasses of the "Natural Earth Low Resolution" s For the test dataset, we generate 5000-equally spaced points using Fibonacci-latiice.
3.3 MULTI-VARIABLE ERAS INTERPOLATION
Next, we use real-world climate data. We downloaded globally distributed climate variables at Jan. 1. 2018, 23:00 UTC from the fifth-generation atmospheric reanalysis of the global cli- mate (ERA5) product of the European Centre for Medium-Rane Range Weather Forecasts. This dataset contains 6 483 600 observations corresponding to regularly gridded locations the surface of our
http:///wwwwnnttaaoooooooooaaoaa 50m land.zip

---

Published as a conference paper at ICLR 2024
(a) The Fihonacci-I Lattie Checkerhoard (test) Dataset.
(b) Land-Ocean Classification (test) Dataset.
Image encoder
InceptionV3 $
P(ul, X) P(U(T)
Location encoder
x P
NN(PE(6.A))
(c) ERA5 Dataset: near- surface temperature channel.
(d) INaturalist 2018 setup: We vary the location encoder and measure classification accuracy increase thanks to location o, A.
Figure Datasets in this work from synthetic (a) and semi-synthetic (b) to real use-cases (c,d).
planet. At each location, we obtain eight continuous climate variables: the and v components of near-surface wind speed, near-surface air temperature, surface air pressure, near-surface specific humidity, surface shortwave and longwave radiation, and total precipitation (rainfall and snowfall flux). An illustration of near surface air temperature (centered on the North pole) is shown in fig. Be We design multi-task regression in which the model learns all eight variables jointly from a small (1%) randomly sampled training set. Another 5% of the data is reserved for the validation set. Through this dataset, we approximate the continuous signal underlying the gridded climate data into the model weights of neural network and measure its ability to reproduce the signal continuously at any point without explicit interpolation of neighboring grid cells.
3.4 INATURALIST 2018: LOCATION-INFORMED IMAGE CLASSIFICATION
Finally, we test the location encoders' ability encode location as auxiliary information for fine- grained species classification with georeferenced images. Here, we use the iNaturalist (iNat2018) (Van Horn et al. 2018) dataset, which contains georeferenced images of plant and animal species. a real-world dataset with several data-inherent challenges: it presents fine-grained classification problem with 8142 individual categories in long-tailed class distribution. has a strong sampling bias toward the US and Europe due to crowd-soureing. Adding location information to an image classifier has been proven beneficial, as, e.g., similarly-looking species from different continents can be ruled out. Hence, iNaturalist 2018 has been used extensively to benchmark location encoders (Mac Aodha et al. 2019 Mai et al.] 2020b 2023a). Following these works, we used the public training split of 436 063 georeferenced images and partitioned randomly into training and valida- tion datasets in an 80:20 ratio. The public validation split of 24 343 georeferenced images was used as a test dataset. We combine the probabilities of the location encoder P po, A) and image clas- sifier II) by multiplication, as shown schematically in fig. 3d A fixed pre- tre-trained Inception V3 network from Mac Aodha et al. (2019) serves as the image classifier. The results section reports the relative accuracy increase that the location encoder adds to the base image classifier. Through this dataset, we learn a continuous species distribution map in the model weights from point observations of training points. At test time, the model predicts the probability of species presence new test locations, which informs a separate image classifier and increases the overall prediction accuracy.
4 RESULTS
The implementation details and hyperparameter tuning protocol can be found in appendix A.I
4.1 COMPARATIVE PERFORMANCE
Checkerboard and Land-Ocean Classification. tables and 1b show averaged classification curacies and standard deviations over five runs of the checkerboard and land-ocean classification datasets, respectively. We observe that the SPHERICAT. HARMONICS (SH) embedding works well

---

Published as a conference paper at ICLR 2024
Table I: Comparison of different positional embeddings (rows) and neural networks (columns). The best combination is bold, and the best positional embedding per neural network underlined.
(a) Checkerboard Results: Classification accuracy and standard deviation from runs; higher better.
(b) Land-Ocean Results: Classification accuracy and standard deviation from 5 runs; higher is better.
PE
NN + LJNEAR
FONET
SIRENNET
PE
NN
LINEAR
FcNET
SIRENNET
DIRECT
8.3
11.2
94.7 +
DIRECT
71.4 + 0.0
90.3 + 0.7
95.1 + 01.
CARTESIAN3D
0.4
85.5 = 1.2
95.7 +4.=002
CARTESIAN3D
70,5 3.5
92.7 + 0.3
92.8 : 0.3
WRAP
0.4
68.3
11.5
9..3
WRAP
74.4 0.3
93.2
95.2 0.2
0.6
93.5
GRID
81.7+
95.1
95.5
THEORY
6.1
9.3
9.3
THEORY
8..
94
95.5 0.1
SPHEREC
0.5
0.3
9.2
SPHEREC
95.0 0.2
95.2
SPHEREC+
03.
93.7 0.4
9..4++011
SPHEREC+
84.6+
95.3
95.5 0.1
SPHEREM
0.3
0.9
68.0
SPHEREM
74.0 0.0
0.11
0.4
SPHEREM+
10.8 + 0.2
87.9 + 0.3
90.0
0.2
SPHEREM+
81,9 + 0.2
92.1
93.7 0.1
SH (ours)
9.1
94.9
0.1
95.8
1.0 0.0
SH (ours)
94. 0.1
95.9 0.1
95.8 0.1
(c) Inaturalist 2018: Improved classification accuracy with location. Standard dev. from 5 runs; higher better,
4 + LIneaR
FONET
SIRENNET
PE
NN - LINEAR
FONET
SIRENNET
DIRECT
-5.9=011
+9.3 0.3
+.1.1
DIRECT
-..7
+7.1 0.2
48.8
CARTESIAN3D
=02
+11.800.1
+12.0+0.1
CARTESIAN3D
-0.8
486+01
48.7 0.1
WRAP
-0.1=011
+12.1
+12.1001
WRAP
-0.1
48
48.7
GRID
+11.2+0.1
+12.8+02
+11.6+04
GRID
-8.1
4.1
+..5
THEORY
+.0.5.0000
-10.8
+11.4+0.1
THEDRY
-8.2
+..0
+
SPHEREC
+11.2+0.1
+1200002
+.2
SPHEREC
-799 7.1
+83++01
4.1
SPHEREC+
+.2
+11.5+033
+.0.3+0.4
SPHEREC+
-7.8 0.0
+.1
+7.8 0.2
SPHEREM
-.22-0.2
+12.3:022
0.0
SPHEREM
-..
+7.5
4..
SPHEREM+
+11.6:0.11
+1..0001
0.2
SPHEREM+
0.1
+8.40.11
4..0
SH (ours)
0.11
+00
+22
SH (ours) -7.5
48.1
4..0
image-only: 59.2% top-1 accuracy with encoder NN(PE)
image-only: 77.0% top-3 accuracy with encoder NN(PB) f
for all neural networks, including the minimalist LINEAR layer. Similarly, SIRENNET (Sitzmann et al. 2020) achieves high accuracies consistently across all positional embeddings, while the FC- NET (Mac Aodha et al. (219)) shows larger variability, especially with simpler embedding functions, such as CARTESIAN3D, DIRECT, or WRAP.
ERAS. Table shows the mean squared error averaged over the eight climate vari- ables in the ERA5 dataset. Means and standard deviations are obtained from 10 runs with different random initializations. This multi-task regression is especially difficult as it requires the location encoder to learn the spatial patterns of each channel separately and their interactions. SPHERI- CAL HARMONIC embeddings consistently achieve the lowest error for all neural net- works, while the combination with Fc- NET was best with 0.58 + 0.02. This constitutes a substantial performance gain over the next best existing competitor (SPHEREC+ and FCNET) with 1.38 + 0.03.
Interpolation of 8 climate vari- ables. Averaged MSE and std. dev. from 10 runs; lower is better.
Table 2: ERA 5 dataset:
PR NN
LINEAR
FcNET
SIRENNET
DIRECT
27.19 0.08
7.83 1.06
1.62
CARTESIAN3D
24.18 + 0.00
+23++025
1.57 0.11
WRAP
13.26 + 0.02
4.13 0.25
1.89 + 0.07
GRID
9.83 0.01
1.51 0.04
2.37 + 0.13
THEORY
9.24
1.61 1 1.611.000
2.99 0.100
SPHEREC
20.03 0.02
1.92
1.95 4.009
SPHEREC+
8.50 0.02
1.38
1.97 1.008
SPHEREM
26.68 0.13
3.51
5.89 0.668
SPHEREM+
9.94 + 0.02
1.54+
2.75 +
SH (aurs)
1.39 + 0.02
0.58 + 0.02
1.19 0.04
iNaturalist. Table Ic shows the relative top-1 and top-3 accuracy improvements that location in- formation through I Φ. A) adds to an image-only classifier ()) The underlying image- only classification t and top-3 accuracies are 59.2% and 77. 77.0% respectively. All location en- coders improve the top-1 accuracy substantially between +9.3% (FCNET(DIRECC)) and +12.2% (SH)ENNE((SH)) except for the trivial LDIRERD and too simple LINEAR(WRAP) encod- ings. At 1 accuracy, the SPHERICAL HARMONIC embedding used with the SIRENNET network slightly more accurate than the other approaches but still roughly within the standard deviations (between 0.1% and 0.4%) over five evaluation runs. However, for other neural networks, other positional encoders also achieve the best column score (underlined). At top-3 accuracy, we see lower variances in general between standard deviations of 0.1 percentage. Also here, SIREN(SH) is slightly better with +9.0 + 0.1 % compared to +8.8 + 0.1% of SIREN(DIRECT) and significantly better than SPHEREM+ + (Mai et al. 2023b) with - -8.4%.
7

---

Published as a conference paper at ICLR 2024
(ANAAR(SH..-.. SIRENNET (SHL-20
SH SPHEREC+ GRID
STRENNET(SHLL-OH
FCNET
SIRENNET
70
505-70% 70"N
accuracy
0.8
mmax
L=20
10
10"N
50"N
10'S-10"N
0.4
30 1
50"-30'S
0.2
70"-50'3
90"-70"S
0.85
0.9
0.95
0.9
0.95
distance between checkerboard centers [deg]
accuracy
accuracy
(a) Resolvable resolution: Accuracy increasingly dense checkerboard patterns. Shorter distances (fur-
Latitudinal accuracy: Checkerboard accuracy in 20 bands from North to South for FCNET (left) and SIRENNET (right). Standard deviations from 5 runs.
ther left) require resolving higher-resolution signals.
FSH)
(CNED(GRID)
SIRENNET(SH) (IRENNET(SPHEREIT SPHEREM+)
(c) Predictions regular grid with the Checkerhoard (left two) and iNaturalist datasets (right two). The loca- tion encoders learn an implicit neural representation the checkerboard patterns (left two) and distribution map of Arctic Foxes P(y arctic fox 2x, Φ). Note the artifacts at North poles for the SPHEREM+ and GRID location embeddings that are not present with SPHERICAL HARMONICS (SH).
Figure 4: Experimental evaluation of properties of SPHERICAL HARMONICS paired with SIREN- NET. Fig demonstrates its ability resolve signals across different resolutions. Figures b) and c) highlight the ability to learn signals close to the poles at high latitudes quantitatively (b) and quali- tatively (c) Checkerboard and iNaturalist data.
Takeaways. The SPHERICAL HARMONIC (SH) embedding function achieved the best results across all four datasets and consistently across different neural networks for three of them. For iNaturalist, SPHERICAL. HARMONICS with STRENNET was still best (bolded), but several other positional em- beddings were better in combination with FCNET and LINEAR (underlined). We generally connect this to the location bias iNaturalist, where only a few species observations are recorded at higher latitudes, where good representations of the spherical geometry are most advantageous. Comparing neural networks: the commonly used FCNET (Mac Aodha et al.) 2019) achieved the best results in predicting ERA5 interpolation and Land- Ocdennnennnneo classification, while the SIRENNET network was best in the iNaturalist and checkerboard problems. Overall, SIRENNET performed well across all positional embedding functions, including the trivial DIRECT embedding. Similarly, SPHERICAL. HARMONICS performed reasonably well, even with simple LINEAR layer on all datasets.
4.2 PROPERTIES AND CHARACTERISTICS
Next, we investigate characteristics of SPHERICAL HARMONICS with SIRENNET with respect to key properties of location encoders: resolvable resolution, accuracy across latitudes, and computa- tional efficiency.
Resolvable Resolution. In fig. 4a we focus on resolution limitations of SPHERICAL HARMON- ICS (SH) different orders L. On the checkerboard dataset, we increase the number of grid cells in the Fibonacci lattice, which produces denser and increasingly fragmented patterns. We report the average distance between points on the x-Axis (in degree) as an indication of resolution. We test two SPHERICAL HARMONIC positional embeddings with 10 and 20 Legendre polynomials (SH,=10, 20) We primarily use a LINEAR layer as a "neural network" to highlight the reso- lution limitations of spherical harmonics alone, as it represents eq. where a single trainable pa- rameter weights one harmonic. In this case, we can analytically calculate the maximum frequency 180 that linear spherical harmonic embedding should be able resolve. In the plot, observe a decrease accuracy at resolutions higher than 9- and 18-degree resolutions, respectively. Hence, more Legendre polynomials L. allow the SH(LINEAR) model to resolve higher-frequency
frax 24
8

---

Published as a conference paper at ICLR 2024
components. Replacing the LINEAR layer with more complex SIRENNET network (dotted lines) further improves the model's resolution, indicated by a higher accuracy at smaller distances be- tween centers. Both SIRENNET(SH,, spaced 6 degrees apart with a slight edge for STRENNET(SH thanks to the larger number of spherical harmonics. Hence, both increasing the number of Legendre polynomials and the complex- ity of the neural network (i.e., SIRENNET instead of LINEAR) helps resolve high-frequency signals on the sphere. This property can also be seen qualitatively in fig. 2b on the example of land-ocean classification.
and SIRENNET(SH;, remain accurate up grid cells
Latitudinal Accuracy. In fig. 4b we compare the checkerboard classification accuracies of SPHER- ICAL HARMONICS (SH), GRID (Mai et al. (2020)) and SPHEREC+ (Mai et al.) (023b) positional embeddings, combined with the FCNET (left) and STRENNET (right) networks separated by 20" north-south spanning latitude bands. The DFS-based encodings (GRID and SPHERFC+) assume rectangular domain of latitude and longitude. This assumption is violated at the poles (90' North and 90º South) where all meridians converge to single point. This leads to polar prediction arti- facts in the checkerboard pattern, as shown in fig. 4c (left), and consequently to lower accuracies at higher latitudes. The SIRENNET network helps reduce these artifacts at the 50 to 70 bands N & S, but a performance degradation is still clearly observable close to the poles (709-90"). In contrast, no substantial degradation at higher latitudes is visible on the location encoder with the spherical harmonic embedding function with both FCNET (left) and STRENNET (right) networks, as spherical harmonics are well-defined on the entire sphere. Such artifacts also occur on real-world datasets like iNaturalist, as shown as vertical lines along the North-South meridians on the learned Arctic Fox prediction map from iNaturalist shown in fig. 4c (right) with the FcNET(GRID) location encoder.
Computational Efficiency of Positional Encoders. Figure [ shows the computational impact measured in inference runtime of 10 000 test points of the checker- board dataset with an increasing number of Legen- dre polynomials (or scales for SPHEREC+) on the checkerboard dataset. We first compare spherical harmonics implementation that uses the closed form implementation (SH-closed-fome form)) following (Green) 2003) with the SPHEREC+ embedding. The closed- form implementation (SH-closed-fomm) with large orders, as the inference time reaches 10 sec- onds with L 40 compared to 0.3s of SPHEREC+ (Mai et al. 20236). To overcome this limitation, we calculate the individual spherical harmonics analyti- cally for each harmonic with eq. (5) and simplify the equation with SymPy (Meurer et al.) 2017)) This leads to a substantial performance improvement implementation. All other experimental results in the paper use this analytic calculation, which is faster regarding runtime than the provided SPHEREC+ implementation up to L > 20 polynomials.
SH-analytic
SPHEREC+
inference time [s]
SH-closed-form
100
10-1
10
20
30
40
Legendre Polynomials L
scales poorly
Figure 5: computational efficiency of an- alytic and pre-computed SH implementa- tions with SphereC+ on the checkerboard dataset.
second for L = 40 over the closed-form
5 CONCLUSION AND OUTLOOK
Encoding geographical locations effectively is key to meeting many real-world modeling needs, and it requires considering our planet's spherical geometry. To this end, we propose to thogonal SPHERICAL HARMONIC basis functions paired with sinusoidal representation networks (SIRENNETs) learn representations of geographic location. Even though these spherical harmon- ics are not as trivial to implement as classical sine-cosine functions, they are inherently better suited to represent functions on the sphere. They can be applied at a competitive computational runtime to existing sine-cosine encodings. Regarding neural networks, we have shown that SIRENNET can be seen as a learned Double Fourier Sphere positional embedding, which provides an intuition to explain its good performance even with simple DIRECT positional embedding. This blurs the es- tablished division of positional embedding and neural networks established previous work (Mai et al. 2202)) and opens new research questions towards learning more parameter-efffcient positioned embeddings. We hope to have provided insight into best practices for location encoding of large- scale geographic data and concretely recommend SIRENNET for any problem involving geo- graphic coordinates and SPHERICAL HARMONIC embeddings for problems that involve data at a global scale or in polar regions, which not represented well by existing approaches.

---

Published as a conference paper at ICLR 2024
6 ACKNOWLEDGEMENTS
RZ and DT acknowledge financial support from the Swiss National Science Foundation on project DeepHSM (n. 200021 204057). ER acknowledges support from the Harvard Data Science Initiative and the Center for Research on Computation and Society.
REFERENCES
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama, Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pp. 2623-2631, 2019.
Thomas Berg, Jiongxin Liu, Seung Woo Lee, Michelle L Alexander, David W Jacobs, and Peter N Belhumeur. Birdsnap: Large-scale fine-grained visual categorization of birds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 201 10188, 2014.
Oussama Boussif, Ghait Boukachab, Dan Assouline, Stefano Massaroli, Tianle Yuan, Loubna Ben- abbou, and Yoshua Bengio. What if we enrich day-ahead solar irradiance time series forecasting with spatio-tempont context? arXiv preprint arXiv:2306.01 2022
Grace Chu, Brian Potetz, Weijun Wang, Andrew Howard, Yang Song, Fernando Brucher, Thomas Leung, and Hartwig Adam. Geo-aware networks for fine-grained recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 0-0, 2019.
Elijah Cole, Grant Van Horn, Christian Lange, Alexander Shepard, Patrick Leary, Pietro Perona, Scott Loarie, and Oisin Mac Aodha. Spatial implicit neural representations for global-scale species mapping. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 6342 PMLR, 23-29 Jul 2023. URL https::/prroceddnggs http://roooood.r...........e//p. html
William Falcon and The Py Torch Lightning team. PyTorch Lightning, March 2019. URL https:
Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, and Andrew Gordon Wil- son. GPyTorch: Blackbox Matrix-Matrin Gaussian Process Inference with GPU Acceleration. Advances in Neural Information Processing Systems, 2018.
Álvaro González. Measurement of areas a sphere using fibonacci and latitude-longitue lattices.
Mathematical Geosciences, 42:49-64, 2010.
Daniele Grattarola and Pierre Vandergheynst. Generalised implicit neural representations. Advances in Neural Information Processing Systems, 14:30446-3048,, 2022.
Robin Green. Spherical harmonic lighting: The gritty details. In Archives of the Game Developers Conference, volume 56, pp. 4, 2003.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016. doi: 10. 1..110/..R..226...000
Langwen Huang and Torsten Hoefler. Compressing multidimensional weather and climate data into neural networks. International Conference on Learning Representationss 2023.
Lorène Jeantet and Emmanuel Dufourq. Improving deep learning acoustic classifiers with contextual information for wildlife monitoring. Ecological Informaties, 77:102256, 2023. ISSN 1574-9541. doi: http:///eeeeeeerrreaa//////////3 com/
102256........... URL https: /twp:////wwwwwww//. www..seeeiincdddet...tt..
10

---

Published as a conference paper at ICLR 2024
Konstantin Klemmer, Nathan S Safir, and Daniel B Neill. Positional encoder graph neural networks for geographic data. In International Conference on Artificial Intelligence and Statistics, pp- 1379-1389. PMLR, 2023.
Lukas Koestler, Daniel Grittner, Michael Moeller, Daniel Cremers, and Zorah Lähner. Intrinsic neural fields: Learning functions manifolds. In European Conference Computer Vision, pp- 622-639. Springer, 2022.
Nico Lang. Walter Jetz, Konrad Schindler, and Jan Dirk Wegner. A high-resolution canopy height model of the Earth. arXiv preprint arXiv:2204.08322, 2022.
Oisin Mac Aodha, Elijah Cole, and Pietro Perona. Presence-only geographical priors for fine- grained image classification. Proceedings the IEEE/CVF International Conference on Com- puter Vision, pp. 9596-9608 2019.
Gengchen Mai, Krzysztof Janowicz, Ling Cai, Rui Zhu, Blake Regalia, Bo Yan, Meilin Shi, and Ni Lao. Se-kge: A location-aware knowledge graph embedding model for geographic question answering and spatial semantic lifting. Transactions in GIS, 24(3):233-655, 2020a.
Gengchen Mai, Krzysztof Janowicz, Bo Yan, Rui Zhu, Ling Cai, and Ni Lao. Multi-scale repre- sentation learning for spatial feature distributions using grid cells. International Conference on Learning Representations, 2020b.
Gengchen Mai, Krzysztof Janowiez, Yingjie Hu, Song Gao, Yan, Rui Zhu, Ling Cai, and Ni Lao. A review of location encoding for GeoAl: methods and applications. International Journal of Geographical Information Science, 36(4):639-61 2022.
Gengchen Mai, Ni Lao, Yutong He, Jiaming Song, and Stefano Ermon. Csp: Self-supervised con- trastive spatial pre-training for geospatial-visual representations. 2023a.
Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Stefano Ermon, Jiamiig Song. Krzysztof Janowicz, and Ni Lao. Sphere2vec: Self-supervised location representation learning on spherical surfaces. ISPRS Photogrammetry and Remote Sensing Journal., 2023b.
Julieta Martinez, Rayat Hossain, Javier Romero, and James Little. A simple yet effective baseline for 3d human pose estimation. In Proceedings of the IEEE international conference on computer vision, pp. 2640-266600 26409 2017.
Georges Matheron. Le krigeage universel (Universal kriging). Cahiers du Centre de Morphologie Mathematique, 1:83, 1969.
Philip F. Merilees. The pseudospectral approximation applied to the shallow water equations on a sphere. Armosphere, 11(1):13-20, 1973. doi: 10. 11011000000693311973333 96489494 URL https:
wod.i.or//....... 0ttp:/1111110000000455550000 1.T.S.997..98888 9648342
9733
Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B. Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rath- nayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, Stěpán Roučka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Scopatz. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e 103, January 2017. ISSN 2376-5992. doi: 10.7717/peerj-cs. cs. 103. URL
http:///tiooogggo//////oo 10. 7 7717 7ww./pe177epp111 wttp//pprrjjjpp cs. 103
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
Margaret A Oliver and Richard Webster. Kriging: A method of interpolation for geographical information systems. International Journal of Geographical Information System, 4(3):313-332, 1990.
Steven A Orszag. Fourier series on spheres. Monthly Weather Review, 102(1):56-75. 1974.
11

---

Published as a conference paper at ICLR 2024
Roland Pail, Sean Bruinsma, Federica Migliaccio, Christoph Förste, Helmut Goiginger, Wolf-Dieter Schuh, Eduard Höck, Mirko Reguzzoni, Jan Martin Brockmann, Oleg Abrikosov, Martin Ve- icherts, Thomas Fecher, Reinhard Mayrhofer, Ina Krasbutter, Fernando Sansò, and Carl Chris- tian Tscherning. First goee gravity field models derived by three different approaches. Jour- nal of Geodexy, 85(11):819-843 843, 2011. doi: 0010007/000100-011-0447
URL https:
/ttg/110.100 10o...rgg10.100111100
0467-x
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33:7462-7473, 2020.
EJ Smith, L Davis Jr, DE Jones, P Jo Coleman Jr, DS Colburn, P Dyal, and CP Sonett. Saturn's magnetic field and magnetosphere. Science, 207(4429):407-410, 1980.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929-1995 1958, 2014. URL http: http:///mm...ccccppeeesss
srivastaval4a.htm.
Kenneth O Stanley.
Compositional pattern producing networks: novel abstraction of development. Genetic programming and evolvable machines, 8:131-162, 2007.
Richard Swinbank and R James Purser. Fibonacci grids: A novel approach to global modelling.
Quarterly Journal of the Royal Meteorological Sociery: A journal of the atmospheric sciences, applied meteorology and physical oceanography, 132(619):1799-1799, 2006.
Kevin Tang, Manohar Paluri, Li Fei-Fei, Rob Fergus, and Lubomir Bourdev. Improving image classification with location context. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1008-1016, 2015.
Gabriel Tseng, Hannah Kerner, and David Rolnick. TIML: Task-informed meta-learning for agri- culture. arXiv preprint ar arXiv:2202.021124, 2022.
Gabriel Tseng, Ivan Zvonkov, Mirali Purohit, David Rolnick, and Hannah Kerner. Lightweight, pre-trained transformers for remote sensing timeseries. arXiv preprint arXiv:2304. J4065, 2023.
Grant Van Hom, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun. Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The iNaturalist species classification and detection dataset. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp- 9769-8778 2018
12

---

Published as a conference paper at ICLR 2024
A SUPPLEMENTARY INFORMATION
A.1 IMPLEMENTATION DETAILS AND HYPERPARAMETER TUNING
We use the same protocol across all experiments. To find optimal hyperparameters, minimize validation loss in 100 iterations (30 iterations for iNaturalist 2018) with the Optuna framework (Akiba et al. 2019) in combination with Pytorch Lightning (Falcon & The Py Torch Lightning team 2019) for each positional embedding and neural network combination separately. For the DFS-based positional embeddings (GRID, THEORY, SPHEREC, SPHEREC++ SPIEREM, and SPHEREM+), we tune the minimum radius Tmin between and 90 degrees in 9-degree steps. We keep the maximum radius mmax fixed at 360 degrees. as all problems have global or continental scale (sea-ice thickness) scale. We further tune the number of frequencies S between 16 and 64 with steps of size 16 (32 for ERA5). For the spherical harmonics embeddings, we tune the number of Legendre polynomials I between 10 and 30 in steps of 5 polynomials. In terms of neural networks, we vary the number of hidden dimensions between 32 and 128 in 32-dimension steps both for SIREN (Sitzmann al.] 2020) and FCNET (Mac Aodha et al., 2019) and vary the number of layers between one and three for Siren. For all combinations, we tune the learning rate on a logarithmic scale between 10 and 10 and the weight decay between 10- and 10-!.
Annotations provided in the iNaturalist 2018 dataset are presence-only, which means that the pres- ence of species is not informative of the absence of another. This makes the commonly used cross-entropy loss sub-optimal, as the classes are not mutually exclusive. Instead, we use the full "assume negative" loss function proposed recently by (Cole al. 2023), defined
8 Σ 2 (Im (omgo))) + [vo=1] log(1 ú.) + log(1 - -1%)]
Cly. () - S
(8)
where 8142 is the number of species, À weighs the positive term of the loss, and g' is prediction randomly sampled location across the globe.
During fitting/traiiing the model, we monitor the validation loss and perform early stopping with a patience of 30 epochs (10 epochs for iNaturalis)) if the loss has not decreased. We do this for all NN-PE combinations consistently.
A.2 COMPARISON METHODS: LOCATION ENCODINGS
All encoding approaches take the form of
y == NN (PE (A, Φ))
where non-parametric positional embedding (PE) projects the raw coordinates longitude E [-n, л and latitude o E [- a 7/2) into higher-dimensional encoding space. A small neural network (NN) then projects the encoding to typically lower-dimensional output such as clas- sification logit.
A.2.1 NEURAL NETWORKS
LINEAR simply consists of a linear layer top of the positional embedding. It serves baseline to ascertain the quality of the representation produced by the positional embedding alone.
FCNET is multi-layer perceptron with dropout (Srivastava et al. 2014) and residual connections between layers (He et al. 2016). as illustrated in fig. la Proposed by Mac Aodha et al. (2019) is the default architecture used by the follow-up work.
SIRENNET, the sinusoidal representation network, is a recent architecture proposed by Sitzmann al. (2020) represent complex natural signals with the use of periodic activation functions. As shown in fig. [b] layer of SIREN comprises a linear projection, dropout, and the sine activation function. It is the first time that this network is used for location encoding.
A.2.2 POSITIONAL EMBEDDINGS
DIRECT, i.e., identity, serves as an often-used baseline
PE(A, Φ) (A, Φ))
(10)
13

---

Published as a conference paper at ICLR 2024
CARTESIAN3D encodes the spherical X, e into 3D-Cartesian coordinates z, y, 2 is popular alter- native Tseng et al. [2023 2022)
PE(A, x) cos cos X, cos @ sin X, sin Φ
(11)
WRAP (Mac Aodha et al. 2019) embeds the spherical coordinates (A, Φ) into sine and cosine func- tions avoid discontinuities on the dateline:
PE(X, Φ) = [cos X, sin X, cos Φ, sin Φ)
(12)
GRID Mai al. 2020b enhances WRAP by adding multiple frequencies X a, and o, with scale-dependent factor O2 Tmin maximum radii min maax that can be resolved multiple scales up to S:
[mss)
3 that allows to specify the minimum and
Free
S-1
PE(A, Φ)
U
[cos As sin )a cos o., sin cs.)
(13)
8=0
THEORY Mai et al. (2020b) extends GRID by projecting the locally-rectangular X, coordinates
into a hexagonal grid with three 2sp-ced basis functions aı [1,0]T, a2 = [- 2 v and
33
ag = 2
8-1 U
([A, Φb, ()) ax
.in (1A, Φ0. as) O²
}
PE(A, ②)
(14)
0=0 1=1
SPHERFC Mai al.
2023b) is simplified version the Double Fourier Sphere (DFS) (Merilees) Orszag 1974):
1973
S-1
PE(A, Φ)
U
[sin 6s, cos o, cos Xg cos Φ. sin As]
(15)
s=0
SPHEREC+ Mai al. 2023b) combines SPHEREC and GRID;
PE(A, Φ)
ssin d.,, cos os cos A, cos o. sin Aa]
(16)
s=0
U
[cos Ag, sin As cos c.. sin 6.]
(17)
SPHEREM Mai et al.
2023b) is a simplified version of the Double Fourier Sphere (DFS) MMerlles
1973
Orszag 1974):
PE(A, Φ)
U ssn ds, cos cos X, cos di cos >,, cos d,, sin X, cos sin A.]
(18)
SPHEREM+
Mai et al.
2023b combines SPHEREM and GRID:
PE(A, Φ)
U
[sin Φs. cos os cos A, cos & cos As, cos de sin X, cos cos Ag)
(19)
S-1
[cos Xo sin As cos os, sin Φ+)
(20)
SPHERICAL HARMONICS is detailed in section 2.2
14


Reviews:
[{"review_id": "tL5LuyGKnt", "reviewer": "Reviewer_K9PX", "summary": "This work considers the problem of encoding geographical locations to facilitate downstream geospatial prediction tasks. The paper proposes a new approach (spherical harmonics / SH) to geographical location encoding and compares it to other approaches on a collection of real and synthetic downstream tasks. A key focus of the paper is separately evaluating the contributions of the location encoding and the network into which the location encoding is passed. \n\nGenerally, I think this paper has value but there are a few concerns about the experimental results that need to be cleared up. \n\nEDIT: Increase from marginally below to marginally above after discussion. \n\n# References\n\n@inproceedings{martinez2017simple,\n  title={A simple yet effective baseline for 3d human pose estimation},\n  author={Martinez, Julieta and Hossain, Rayat and Romero, Javier and Little, James J},\n  booktitle={Proceedings of the IEEE international conference on computer vision},\n  pages={2640--2649},\n  year={2017}\n}\n\n@inproceedings{mac2019presence,\n  title={Presence-only geographical priors for fine-grained image classification},\n  author={Mac Aodha, Oisin and Cole, Elijah and Perona, Pietro},\n  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},\n  pages={9596--9606},\n  year={2019}\n}", "questions": "See the questions/headings in \"weaknesses\". \n\n# Misc. Comments\n* I suggest underlining the second-best value in each column in Table 1 to make it easier for the reader to see the magnitude of the change between the best and second-best method. \n* For context, I believe that \"FCNet\" in [mac2019presence] was based on an architecture found in earlier work in pose estimation [martinez2017simple].\n* It's not clear that the set union notation in Eq. (2) is being used to build a vector - please clarify in the text. \n* Is there any intuition for why DFS don't perform well empirically? \n* I think the reader would benefit from a discussion of how the test sets were chosen, and why they are appropriate in a spatial prediction context.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "* The paper is generally well-written, with few typos or unclear passages. \n* The problem under consideration is important and timely. \n* Results are presented with error bars. \n* The figures are well-made and informative. \n* The evaluation tasks are reasonable.\n* The detailed comparison of different combinations of location encodings and network is useful for the community - as the authors point out, these have not often been studied as separate modules.  \n* The performance improvements (with a linear encoder) are impressive. \n* Solid hyperparameter tuning procedures.", "weaknesses": "# When are the performance gains from SH worth it?\n\nWhen using FCNet or SirenNet (which is not much of a burden in practice), the gains from SH are fairly small (<1%). The gains from SH are only large when using a low-capacity network. Similarly, in Figure 4(a) we see that the difference between $L=10$ and $L=20$ is only significant when using low-capacity networks. To me, this seems to suggest that \"expressiveness\" is the main issue. From a user's perspective, does it matter whether the expressiveness comes from the input encoding or the network? What's the difference in computational efficiency between SH + SirenNet and X + SirenNet where X is another high-performing input encoding? What's the best argument (combining computational efficiency and performance) that SH is worthwhile to use as a component in a real system compared to the alternatives? \n\n# What is the trade-off between performance and efficiency? \n\nGenerally speaking, the \"take home\" efficiencies of the different methods are not clear. Figure 5 shows the computation time required to compute SH under different implementations, but the important thing is the trade-off between performance and computational efficiency. E.g. is SH + Linear faster than SH + FCNet? This would be very helpful for evaluating the practical usefulness of SH. \n\n# Isn't network capacity and degree of fit a significant confounder in the experiments?\n\nFollowing on from the previous point, FCNet and SirenNet are \"typically\" implemented with a certain number of layers. I might have missed it, but how many layers does SirenNet have in this work? Are FCNet and SirenNet matched in this sense? Do they behave differently as the capacity of the networks increases? What is the effect of training duration / how well the network has fit to the training data? These factors seem very important for comparing the results for these two networks, but I do not see them discussed in the text. This makes it difficult to evaluate key claims in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vq4Nx5pNQO", "reviewer": "Reviewer_BVXQ", "summary": "In this paper, authors propose a ML model for learning feature representation for geographical space, based on spherical harmonic basis functions as positional embeddings that are well-defined across the entire globe, including the poles. Specifically, author propose ropose Spherical Harmonic (SH) coordinate embeddings that work well with a few tested neural networks, especially well when paired with Sinusoidal Representation Networks (SirenNet) .\n\nAuthors test the proposed embeddings with a few state-of-the-art models and demonstrate improved performance with their method in comparison to previous research.", "questions": "Please see the previous section: part 3 is too dense, please insert more explanations on what was done and descriptions. Alternatively, you can include mere details as appendices for better paper understanding", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written, the experiments include some key datasets, used in climate science (such as ERA5). the results are well-presented and sound. The paper meets all the requirements for the ICLR publication.", "weaknesses": "The paper is quite dense, with prevalence of abbreviations and mathematical notation over the description, which makes it difficult to read, especially in  sec.3 Datasets and experiments. Readers without the knowledge of the specific datasets won't be able to understand the application. I would suggest reduce the number of the use-cases to 3 (g.g ERA5 and iNaturalist) and describe in details what exactly did authors do.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DEsqGnaiC6", "reviewer": "Reviewer_g3zK", "summary": "This paper proposes a location encoder which combines spherical harmonics (SH) coordinate embeddings with sinusoidal representation networks (SirenNets) and argues that SH encodings enable accounting better for the geometry of the Earth than existing methods using Double Fourier Sphere which project coordinates to rectangular domain. \n\nSeveral positional embedding methods in combination with different neural networks to obtain location encoders are compared across 4 tasks. They claim that SH with SirenNets are an effective way of encoding geographical location, in particular for tasks where there is data in polar regions or at a global scale. \nThey also find that SirenNets perform competitively on their own (without any encoding of the latitude and longitude) which they explain by showing that SirenNets can written out as DFS embeddings.", "questions": "- I am curious to hear the authors' intuition about the combination of SirenNet and some DFS based encodings performing worse than SirenNet alone? (e.g. SphereC+ + SirenNet on INaturalist 2018)\n- I may have missed this information but what is the number of Legendre Polynomials in the experiments results reported in the tables? I would also be curious to have the performance of the different models of Figure 5 with the number of polynomials alongside the computational efficiency. As in, is the extra computational cost associated with using SH worth bearing with a higher number of polynomials in terms of gain in performance for L > 20? It seems that not (from looking at Fig4a) but it would be helpful to compare the performance of the different location encoding methods. \n-  I am surprised that SphereM performs so poorly in comparison to the other positional embeddings (the difference in performance is not as striking in the Sphere2vec paper on their tasks). What is your take on this poor performance?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is clear and well-written.\n- While neither SH nor SirenNets are novel, their combination appears as a well-motivated approach to location encoding, and has never been used in this context. \n- The proposed method attempts at addressing challenges around the poles, an issue which seems to have been overlooked with methods assuming a rectangular data space. \n- The results obtained on the 4 datasets suggest that the proposed combination of SH and SirenNet is an effective way of encoding location.", "weaknesses": "- While we acknowledge the work put into designing tasks to showcase challenges specific to geographic location encoding, it would have been interesting to see more comparisons on datasets that have been used by previous work on this topic, besides the iNaturalist task (e.g. the fMoW dataset used in the Sphere2Vec paper cited in this work). \n- The task on the ERA5 dataset was designed specifically for this work. It is difficult to assess whether the gain in using SH would also be significant in real-life climate science tasks, which is a domain of application that the paper puts forward. Would it be possible to integrate this method other tasks more common with ERA5 such as downscaling?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GVjhpdHPYI", "reviewer": "Reviewer_YTje", "summary": "A method to learn features representations of geographical data observed on the Earth's surface is presented.  The method is based on intrinsic neural fields, using spherical harmonic functions for the positional encoding, combined with SIRENs which adopt sine non-linearities as periodic activation functions (the spherical harmonic embedding can of course also be combined with other neural networks).  It is also shown that SIRENs can be see as a form of learned Double Fourier Sphere (DFS) positional embedding.  A number of experiments are presented, across synthetic and real-world problems, demonstrating the effectiveness of the representation for machine learning tasks, typcially classification.  The proposed method demonstrates an improvement in performance over existing techniques.", "questions": "- Could the authors elaborate how their work fits into the context of prior work by Grattarola & Vandergheynst (2022) and [Koestler et al. (2022)](https://arxiv.org/abs/2203.07967).\n\n- It seems combined spherical harmonic encoding with SIRENs is generally best.  However, for the ERA 5 dataset it seems a fully connected network was significantly better than combing with SIREN.  Do the authors have any idea why a fully connected network was best here?  \n\n- Do the authors have any thoughts on how they could extend the method to higher degrees L?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The method is straightforward and flexible since it simply involves a spherical harmonic encoding coupled with a neural network (e.g. fully-connected, SIREN).  The poles on the sphere can be handled directly since the continuous spherical harmonic functions can be evalued for any coordinates, including the poles, whereas alternative approaches based on equirectangular sampling of the sphere often have large numbers of samples near the poles, which can induce artefacts.", "weaknesses": "Positional encoding for implicit neural representations on manifolds, such as the sphere, have been considered previously in Grattarola & Vandergheynst (2022), which is cited, and also in [Koestler et al. (2022)](https://arxiv.org/abs/2203.07967), which is not cited but should be.  Grattarola & Vandergheynst do not specifically consider the sphere and spherical harmonics, instead focusing on graph representations, but consider an emedding based on the eigenfunctions of the graph Laplacian.  Koestler et al. consider general manifolds, with an embedding based on the eignenfunctions of the Laplace-Beltrami operator.  On the sphere, these eignerfunctions are specifically the spherical harmonics.  So an embedding based on spherical harmonics as presented in this article is not new, although in this work the embedding is combined with SIRENs and extensive experiments on the sphere are performed.\n\nOne limitation of the proposed approach is that it is limited to very low degrees L on the sphere, typically L of 20 or at most 40.  The spherical harmonics are precomputed analytically, which while fine for these very low degrees will not scale to higher degrees.\n\nIt is indeed the case that combining the spherical harmonic encoding with SIRENS typically gives the best performance (see Table 1).  However, in many cases the improvement is not that substantial, compared to the next-best method.  The improvements for the ERA 5 dataset appear to be more marked.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)