Attention on the Sphere
Authors: Boris Bonev
Max Rietmann
Andrea Paris
Alberto Carpentieri
Thorsten Kurth
Venue: NeurIPS 2025 poster ()
Keywords: Attention
neighborhood attention
transformers
sphere
geometric machine learning
equivariance
scientific machine learning
spherical signal processing
CUDA
Primary area: deep_learning
TL;DR: We generalize attention and neighborhood attention to the two-dimensional sphere in an equivariant manner.

Abstract: We introduce a generalized attention mechanism for spherical domains, enabling Transformer architectures to natively process data defined on the two-dimensional sphere - a critical need in fields such as atmospheric physics, cosmology, and robotics, where preserving spherical symmetries and topology is essential for physical accuracy. By integrating numerical quadrature weights into the attention mechanism, we obtain a geometrically faithful spherical attention that is approximately rotationally equivariant, providing strong inductive biases and leading to better performance than Cartesian approaches.
To further enhance both scalability and model performance, we propose neighborhood attention on the sphere, which confines interactions to geodesic neighborhoods. This approach reduces computational complexity and introduces the additional inductive bias for locality, while retaining the symmetry properties of our method. We provide optimized CUDA kernels and memory-efficient implementations to ensure practical applicability.
The method is validated on three diverse tasks: simulating shallow water equations on the rotating sphere, spherical image segmentation, and spherical depth estimation. Across all tasks, our spherical Transformers consistently outperform their planar counterparts, highlighting the advantage of geometric priors for learning on spherical domains.

Attention on the Sphere
Boris Bonev", Max Rietmann"", Andrea Paris, Alberto Carpentieri, Thorsten Kurth NVIDIA Corporation, 95051 Santa Clara, CA, USA
{bbonev, mrietmann, aparis, acarpentieri, ckurth)/nvidia..
Abstract
We introduce generalized attention mechanism for spherical domains, enabling Transformer architectures to natively process data defined on the two-dimensional sphere critical need in fields such atmospheric physics, cosmology, and robotics, where preserving spherical symmetries and topology is essential for physical accuracy. By integrating numerical quadrature weights into the attention mechanism, we obtain geometrically faithful spherical attention that is approxi- mately rotationally equivariant, providing strong inductive biases and leading better performance than Cartesian approaches. To further enhance both scalability and model performance, we propose neighborhood attention on the sphere, which confines interactions to geodesic neighborhoods. This approach reduces computa- tional complexity and introduces the additional inductive bias for locality, while retaining the symmetry properties of our method. We provide optimized CUDA kernels and memory-efficien implementations to ensure practical upplicability. The method is validated on three diverse tasks: simulating shallow water equations on the rotating sphere, spherical image segmentation, and spherical depth estima- tion. Across all tasks, our spherical Transformers consistently outperform their planar counterparts, highlighting the advantage of geometric priors for learning on spherical domains.
1 Introduction
The mro-dimensional sphere embedded in three dimensions (S²) plays crucial role in variety of scientific and engineering domains such as geophysics, cosmology, chemistry, and computer graphics. All of these domains require the processing of functions defined on the surface of the sphere. With the advent and increased success of machine-learning in these domains, there a growing demand This is particularly
to generalize sate-of-theeart architectures to spherical data [13 18 19 34 6
important in scientific applications which require the solution of time-dependent partial differential equations which are often approximated with auto-regressive models. These models are particularly sensitive to spurious artifacts which arise from a non non-geometrical treatment as these tend to build up and compromise stability [6]
While spherical convolutional neural networks (CNNs) and other geometric deep learning models have advanced the state of the art for learning on the sphere by respecting its symmetries and topology, these architectures are primarily designed for capturing local interactions and equivariant representations
30 However, many scientific and engineering problems on the sphere, such as weather and climate modeling. 360 perrpetcn,,ion, and cortical surface analysis, require modeling complex, long-range dependencies that are not efficiently handled by convolutional approaches
13
7 3 9]. Transformers, with their global attention mechanism, have revolutionized learning in Euclidean domains by enabling flexible, data-driven modeling of both local and global relationships
"Corresponging author. 'qqual contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

24).. Extending Transformers to spherical domains promises to combine the geometric fidelity of spherical models with the expressive power of attention, enabling new capabilities for spherical data analysis.
15 38 4
Recent works have begun explore this direction, demonstrating that Transformer-basee models tailored to spherical geometry can outperform traditional methods on tasks such as depth estimation or semantic segmentation by directly leveraging spherical geometry their design [3] [0] However, due to their reliance on permutation equivariance on icosahedral grids, they fail to capture the full rotational equivariance associated with the sphere. Other works have successfully applied Euclidean Transformer architectures to weather forecasting on the sphere [4], however, they show deterioration in longer auto-regressive rollouts [26). This motivates the development of spherical attention mechanisms that operate natively on the sphere, unlocking the full potential of Transformer architectures for scientific and engineering applications involving spherical data.
Our contribution We generalize the attention mechanism to the spherical domain. To respect rotational symmetry, we derive a continuous formulation on the sphere for both global attention and neighborhood attention mechanisms. A discrete formulation is then derived using quadrature rules to approximate kernel integrals, thus ensuring approximate equivariance w.r.t. three diree-dimensional rotations. Both spherical attention variants are implemented in PyTorch, including custom CUDA extensions of the spherical neighborhood attention. We demonstrate the effectiveness of our approach by generalizing standard Transformer architectures to the spherical domain using our formulation and scoring them against their Euclidean counterparts in three different spherical benchmark problems.
Our implementation and code to reproduce experiments are available in the open-source library torch-harmonics
2 Related work
Transformers Attention mechanism and Transformer architectures were initially developed for sequence modeling applications such as language models (37] and later applied to vision tasks in the Euclidean domain [5]. This has proven to be an effective method despite the lack of inductive
biases such as locality 4 24 The attention mechanism is equivariant with respect to permutations
of the input tokens and therefore lacks any notion of the underlying topology of the domain. This makes position embeddings necessary, which introduce the notion of locality. A notable advantage of the attention mechanism is its ability to capture long-range interactions. However, this makes prohibitively expensive for large resolutions due to its ((N)) asymptotic complexity, especially for multi-dimeesional data where the number of pixels N grows exponentially with dimension limiting the resolution.
Neighborhood Transformers Hassani et al. [22) address this by simultaneously introducing the notion and inductive bias of locality into the architecture. This is achieved by using the neighborhood attention mechanism. In this approach, the attention layer is constrained to a neighborhood of grid points around the query point. This has great computational benefits as lowers the asymptotic complexity of the attention mechanism to O(kN) where k is the kernel size of the neighborhood. While this method highly efficient (23,, it is generally only formulated for equidistant Cartesian grids. For spherical problems, this leads to distorted neighborhoods on spherical geometries.
Equivariant architectures Several convolutional architectures address this problem and capture the geometric properties alongside SO(3) rotational symmetry [12] been addressed by Transformer architectures yet. Finally, Assaad et al. 2 introduce a rotationally equivariant Transformer architecture in the context of vector neurons using the Frobenius norm.
17 13
6). it has not
Graph equivariant Transformers Various equivariant transformer architectures have been pro- posed in the literature. Notable examples are the architecture [29], and related architectures [39 g). However, these architectures focus on vector- valued features as inputs, rather than entire functions defines over a subdomain. Examples are positions of atoms in molecules and LiDAR point clouds. As such, the attention mechanisms in these architectures aim to preserve both distances of positions with each other and the relative orientation of the vectors.
(20 and the Equiformer
2

---

(a) Euclidean
(b) naive spherical
(c) weighted spherical
Figure I: Neighborhood attention on the sphere: (a) Euclidean neighborhoods use planar distances, distorting receptive fields near poles. (b) Naive spherical attention uses geodesic neighborhoods but equally weights points, oversampling points and breaking SO(3) symmetry. (c) Our method introduces quadrature weights to account for non-une uniffrmf sampling density, preserving rotational equivariance regardless of grid alignment through proper discretization of the continuous formulation.
Graph transformers applied to spherical functions Several graph-transformer models have been introduced that operate icosahedral discretizations of the sphere [T0]
enabling the application of transformer architectures to function-valued data spherical domains. While this approach has shown strong performance for spherical signals, it relies on the standard attention mechanism, which ensures only permutation equivariance with respect to the underlying graph. Consequently, these architectures exploit discrete graph symmetries rather than the continuous rotational symmetry the sphere itself.
3
3 Method
3.1 Attention mechanism and kernel regression
We revisit the motivation behind the original attention mechanism. For two one-dimensional signals q. R + I, we can compute the correlation function Clg. (xx,xx') = corr(q(z). k(z²)) using the Pearson correlation corr(..-). which measures the statistical dependence between signal pairs as
function of the point-pair x'). Many stochastic processes, images and other spatial signals have distinct auto-correlation functions, which can be modeled using a kernel function (zx
This analogy between correlation functions and kernel functions is exploited in nonparametric regression techniques such kernel regression. For query/key signals k IR+ - Rd. and value signal v I + IR, the kernel regressor is given by
a(((z).k(z')
r(g(x)) JR JR+ k(z()) de
v(x') dx',
where a(-.)) -) is suitable kernel function. For instance, by choosing ((x) x, k(x')((x=)))))))))) x' and the Gaussian kernel (xx,x') exp(-(x x²)² /h?) of width h, recover the radial-bast regressor.
The attention mechanism [37] generalizes this concept further through learnable kernel transforma- tions. The dominn attention mechanism
Attulg,k, k ul(x) / Alg, (a( v(x') dr',
(2a)
Alg, (((xzx))===))))) exp(q (x))))(())//Vl J&+ exp(( (TT(T) ((1x\///]])) dr'
(2b)
corresponds to exponential kemne smoothing, where the exponential kernel a(g. k) poo[n"k/VV measures similarity between query-key pairs while the denominator normalizes attention weights as probability distributions. The linearized variant [36] omits softmax normalization.
3

---

analogous to using unnormalized kernel weights in regression numerators. Modern attention mecha- nisms extend this kernel method by learning adaptive correlation patterns which are parameterized by the query and key weights, rather than relying on predefined kernel functions.
By discretizing the continuous attention mechanism on the sequence {.:) standard attention mechanism
Ma we recover the
Man
exp qt (1.) k(z)\/V)) exp (T k(zi)/V))
exp(d"
Attulg, k u2(2)
Σ
v(zy),
New
commonly found in sequence modeling [37 and other applications. This formulation is permutation- equivariant, in the sense that a permutation of the tokens g(z;), k(x;) and v(x;) will result in the same output but permuted.
3.2 Attention on the sphere
The generalization the attention mechanisms for spherical data requires a proper treatment of spherical geometry and topology. To do so, change the domain in the continuous formulation to the sphere and obtain
/ Asa (1, ki x²) o(x') du(a'),
(4a)
exp(q (x) k(z)\/V)
As=ly, (1,,,))
(4b)
SSa exp q" (x) k(z"//Vd) du(x"))
for inputs q, 52 RRd
52 R 11 52_, J denotes the invariant Haar measure the
sphere and ensures that the integrals in 4 remain unchanged under dimennionn rotations
R E SO(3) of the variables of integration. This makes the attention mechanism (4) equivariant, i.e. Attns= gg k u x) = Attns= y (x) for rotated signals (((x) = q(R xx) k(x)
=2) o (z) ((R-1z)
local, neighborhood attention variant of can derived by choosing a compactly supported kernel a. We define
D()) exp
Nsa ig, (z,
(5)
Ssa 1 D(u) (x") exp qt (x) k(x")////////) v/d du(x"))
where 1 Dz) (x') is the indicator function of the spherical disk D(x) {x' S2|d(z, z²) S Ocunott centered at x. This disk determined by the geodesic distance (great circle/Haversine distance) d(;;)s and the hyperparameter Ocunotr which controls the size of the neighborhood. By replacing the attention operator Aga with Nga, we obtain the spherical neighborhood attention
[ Ns= kl(, 20) v(x') du(x'),
NAttng (g, k, (x())
(6)
Jsz
which retains the equivariance property of (4).
To obtain discrete attention mechanisms, discretize the spherical domain with set of grid points {z, E 52 for = 2. Ngraa and apply a suitable quadrature formula w; == wxxx), such that
Nas u(x;) Wi.
J u(x) dr
This allows numerical evaluation of the integrands in and (6) giving the discrete full and
neighborhood attention (see Appendix B.3 for additional detail).
Figure I visualizes our approach to neighborhood attention on spherical data. By deriving the continuous formulation through quadrature rules and rigorously incorporating geodesic distances, establish our architecture neural operator. This formulation enables consistent evaluation

---

across arbitrary discretizations while preserving geometric structure. Crucially, the quadrature- implementation ensures approximate SO(3) equivariance, maintaining transformation consistency under 3D rotations. To validate the equivariance of our implementation, we conduct convergence test in Appendix that demonstrates convergence of the approximate equivariance error relation to the interpolation error as resolution is increased.
3.3 Implementation
We implement both spherical attention mechanisms for equiangular and Gaussian grids and make them publicly available in forch-harmonics, library for machine learning and signal processing on the sphere. The global spherical attention may efficiently implemented using the PyTorch scaled dot product attention implementation by leveraging the attention mask to incorporate the quadrature weights. The neighborhood variant, on the other hand, requires a custom CUDA implementation due to the non-uniform memory access patterns implied by the sparse attention support. Details regarding the implementation and expressions of input gradients are provided in Appendix B.4
3.4 Spherical Transformer architecture
We apply the spherical attention mechanisms derived in Section in the context of popular Trans- former architectures. To this end, we modify the Vision Transformer (ViT) 15 to obtain a fully spherical architecture that attains equivariance properties. To do so, we replace non-spherical opera- tions with spherical counterparts and derive the 5d Transformer and S neighborhood Transformer architectures (see Figure 2 similar approach can be taken to obtain spherical variants of the SegFormer architecture 38 ; i.e. the 52 SegFormer and 5 neighborhood SegFormers, respectively. Due to their continuous formulations, all of these architectures are neural operators [28), allowing them to be realized on arbitrary discretizations of the sphere, that permit a suitable quadrature rule.
In the following, outline the methodology for deriving the spherical Transformers.
macro architecture
spherical operation
attention block
upcamplo
spherical decoder
MLP
attendion
norm
nx
spherical attention block
spherical
multi-head attention
convatutio
position embedding
spherical encoder
norm
Figure 2: Illustration of the spherical Transformer architecture.
Encoding Vision Transformers typically use patch embeddings which can be regarded convolu- tions with special choice of pre-set, sparse weights. As such, we employ a convolutional embedding scheme based on the discrete- cocrrtte-ooniuuuus spherical convolutions [34]. Unlike standard patch em- beddings, our approach leverages spherical convolutions to ensure that the resulting representation remains function defined on 5 preserving equivariance properties of the model with respect to rotations. Moreover, the discrete-continuous formulation makes these convolutions neural operators 30), enabling flexible discretization.
5

---

Spherical attention blocks The latent representation is refined by sequence of spherical attention blocks (Fig. 2 right). Each block applies spherical multi-head attention followed by a multilayer perceptron (MLP), both with residual connections and preceded by instance normalization, forming a pre-norm architecture. Positional embeddings (defined in Section (3.4) are added before the multi-head attention step. The spherical attention can operate either globally across the entire sphere, as defined in Eq. or locally, as shown in Eq. (6).
The final latent is passed through a decoder, which mirrors the encoding step. Bilinear spherical interpolation is performed to upsample the signal to the full resolution. This is followed by a spherical discrete-continuout convolution, which produces the output.
Decoding
Position embeddings Transformer architectures typically incorporate positional embeddings to provide information about the location of each input token. This is necessary because the attention mechanism (3) is inherently permutation-invarian and thus unaware of spatial structure. We experi- mented with several possibilities including learnable [5]. sequence- based [37], and spectral position
embeddings [35] In the end, we chose the spectral embedding, where the k-th channel is determined by the real-valued spherical harmonics Y (z), with l = Lvk] and m = k E(e + +).
The resulting spherical Transformer architecture is naturally neural operator, all spatial operations can be formulated in the continuous domain, as well as for arbitrary grid-resolutions which permit the accurate numerical computation of integrals over the sphere. Finally, the same design modifications are applied to the SegFormer architecture (38) to obtain the spherical SegFormer and spherical neighborhood Former... These architectures are made available as part of torch-harmoniis
4 Experiments
We evaluate our method by adapting the popular Vision Transformer (ViT) and SegFormer architecture to equirectangular grids the spherical domain. We follow the procedure described in Section [3.]] replacing the attention and convolution layers with their spherical counterparts while maintaining equivalent architectural configurations, such depth and embedding size. In particular, receptive fields of neighborhood attention mechanisms and convolutions are carefully chosen to match the covered areas. This allows for fair comparison of models, as models retain similar parameter counts, as listed in in the appendix. comprehensive study comparing spherical Transformers to their corresponding Euclidean baseline is carried out for both global and local (neighborhood) attention variants. Implementation details are provided in Appendix C.T
4.1 Segmentation on the sphere
The Stanford 2D3DS Dataset I provides spherical images taken indoors in a university setting. provides a total of 1621 RGB images along with number of output pairs including segmentation data containing 14 classes, such as clutter, ceiling, window, etc. For the purpose of training, we downsample the data to a resolution of 128x256 and split the dataset into 95% train, 2.5% test and 2.5% validation parts. Training carried using uniform weighted cross entropy loss for 200 epochs using the ADAMW optimizer (31] a learning rate of 0.5 x 10-4 and cosine scheduler. Furthermore, to reduce overfitting, weight decay and dropout path rate are both set to 0.1. On single NVIDIA RTX 6000 Ada, training took between 12 and 215 minutes depending on the respective models. The results are validated using the Accuracy (Acc) and Intersection over Union (IoU) metrics (25) on the validation dataset which contains 35 samples. For a detailed discussion refer to Appendix A.2
Table reports the results of our experiments. For both transformer and SegFormer models, we see the spherical variants outperforming their Euclidean counterparts, where the Transformer models have compatible hyperparameters across spherical and euclidean variants. We train EGFormer [40] and SphericalCNN [1]] to serve as additional geometric baselines. We note that the EGFormer seemss to overfit and may require different hyperparameters than the other architectures. Moreover, we report results for the SphereUFormer [3] alongside high-resolution variants of our neighborhood SegFormers with double the embedding dimension. We observe that our S SegFormer with locally supported spherical attention mechanism outperforms the other models in the validation metrics.

---

RGB input
R SegFormer (local)
S SegFormer (local)
Ground truth
C
Cier
BORA
C
Figure 3: Samples from the segmentation dataset with Euclidean and S2 neighborhood SegFormers.
Table I: Numerical results the segmentation dataset. Cross entropy training and validation loss alongside nntersection-ovee--niioo (IoU) and Accuracy (Acc) metries are reported.
Model
Training loss & Validation loss
IoU
Acc
Results on 128 256 data
EGFormer [40]
0. 0.203
1.29 = 0.487
0.586
0.106
0.962 = 0.012
SphericalCNN 14
0.382
0.775 + 0.300
0.622 = 0.099
0.966 =
2 Transformer
0.325
1.041 + 0.812
0.593 0.151
0.962 0.011
S2
Transformer
0.312
1.016 0.499
0.588 0.111
IR
Transformer (local)
0.344
0.854 = 0.352
0.619 = 0.103
0.061
S
Transformer (local)
0.298
0.852 + 0.369
0.622 + 0.106
2
Segformer
0.296
0.796 0.363
0.636 0.109
0.967 + 0.012
22
SegFormer
0.307
0.693 + 0.303
0.657 + 0.106
0.970 + 0.011
R Segformer (local)
0.288
0.782 + 0.334
0.637 + 0.101
0.968 + 0.011
22 FegFormer (local)
0.320
0.667 0.309
0.667 0.110
0.971 + 0.012
Results
256x512 data
SphereUFormerr [3]
0.722
0.886
IR2 Segformer (local, large)
0.184
0.740 0.858
0.712 + 0.153
0.974 0.017
2 SegFormer (local, large)
0.189
0.554 + 0.550
0.748 + 0.126
0.979 + 0.013
Figure 3 depicts results from the best performing local 52 SegFormer architecture alongside its Euclidean counterpart. It demonstrates improvement in the quality of segmentation masks, when compared to the ground truth target.
4.2 Depth estimation on the sphere
The Stanford 2D3DS dataset provides depth maps alongside segmentation data, which our models on the depth estimation task. Models are trained for 100 epochs using the Leept objective functions (6) which combines L, and Sobolev 11 losses, to better preserve image sharpness. The top and bottom 15% masked out there is valid target data in this area. Training times vary between and 220 minutes on a single NVIDIA RTX6000 Ada GPU depending on the architecture.
train
The numerical results alongside individual samples from the Transformer architectures are provided in Table and Figure 4 We observe improved validation losses and W1.1 error for spherical architectures (see Appendix A.2 for detail on the losses and norms).

---

RGB input
RC Transformer (local)
S Transformer (local)
Ground truth
Figure 4: Sample predictions from transformers alongside ground truth data from the spherical depth estimation task.
Table Numerical results on the depth estimation dataset. Training and validation errors, alongside errors for various cartesian Transformer variants and their spherical counterparts.
Model
Training loss
Validation loss 4
L1 error
W111 error |
R
Transformer
0.954
0.982 + 0.386
0.215 + 0.084
7.67 + 3.22
5
Transformer
0.978
0.993 + 0.381
0.234 + 0.086
7.59 + 3.23
R2
Transformer (local)
1.014
1.023 0.370
0.237 0.079
7.86 + 3.17
S2
Transformer (local)
0.978
0.976 + 0.354
0.209 + 0.068
7.68 + 3.12
R22
SegFormer
1.037
0.980 + 0.371
0.169 + 0.058
8.11 + 3.22
SegFormer
1.005
0.935 + 0.369
0.165 0.057
7.70 + 3.23
R2 SegFormer (local)
1.055
1.002 + 0.374
0.174 0.061
8.27
3.25
2
SegFormer (local)
1.014
0.948 + 0.368
0.174 + 0.061
7.75 + 3.21
additional baselines
Spherical CNN [14]
1.019
1.098 + 0.412
0.315 0.123
7.83
3.21
LSNO 30 EGFormer
0.994
1.020 + 0.387
0.248 + 0.095
7.72 + 3.19
0.831
0.945 = 0.386
0.182 0.084
7.63
3.26
4.3 Shallow water equations on the rotating sphere
The shallow water equations are a system of partial differential equations that describe the dynamics of thin fluid layer on rotating sphere. As such, this system is useful model for geophysical fluid flows such as atmospheric dynamics or ocean dynamics 5 We use the dataset as presented [ Initial conditions are sampled from Gaussian random process and reference solutions are generated on the fly using a spectral solver. The solutions are computed at a resolution Gaussian grid, on sphere which matches the Earth's 's radius and angular velocity. The target solution set at lead time of 30 minutes, which requires 12 time-steps of the numerical solver to compute the target due to time-stepping restrictions. A detailed explanation is provided in Appendix C.2
128 x 256 on
All architectures are trained using solutions output by a traditional spectral solver, with 12 classical time-steps steps to provide the single prediction step at a learning rate of 10-3 All of the above carried out using the ADAM optimizer [27] and using ReduceL_ROnPl ROnPLeeatea learning rate scheduler. On an NVIDIA A6000 GPU, training took approximately between 24 and 34 minutes depending on the model. To evaluate the models, we compared the classically-solved solution with model predictions after single prediction step and five autoregressive prediction steps.

---

Table Numerical results on the shallow water equations on the rotating sphere. Validation errors are reported after a single and after five autoregressive steps.
Model
Training loss
Validation loss 4
L1 error 4
Lg error +
step
I24 Transformer
0.051
0.051 + 0.002
0.148 + 0.003
0.196 + 0.005
8
Transformer
0.003
0.003 + 0.000
0.042 + 0.001
0.054 + 0.001
I22
Transformer (local)
0.015
0.016 0.001
0.075 0.002
0.115 0.004
8
Transformer (local)
0.003
0.003 0.000
0.040
0.001
0.050 + 0.001
I
SegFormer
0.087
0.085 = 0.003
0.214 0.004
0.281 + 0.005
S
SegFormer
0.039
0.038 0.001
0.147 = 0.003
0,193 0.004
222
SegFormrr (local)
0.052
0.050 + 0.002
0.163 + 0.003
0.219 + 0.005
8
SegFormer (local)
0.034
0.033 + 0.001 5 steps
0.138 + 0.002
0.181 + 0.004
12
Transformer
36.986 9.390
3.282 0.367
5.237 0.605
8
Transformer
0.062 + 0.006
0.190 = 0.009
0.239 + 0.011
2
Transformer (local)
5.667 + 0.566
0.933 + 0.044
1.987 + 0.088
Transformer (local)
0.033 0.005
0.137 0.010
0.172 0.012
R2
SegFormer
0.680 = 0.025
0.618 = 0.012
0.781 + 0.015
S
SegFormer
0,154 + 0.013
0.304 0.013
0.386 + 0.017
R22 SegFormer (local)
0.546 0.024
0.544 0.013
0.692 0.016
2 SegFormer (local)
0.104 + 0.008
0.250 0.010
0.319 0.013
Based the outcomes of this experiment reported in Table reduced validation loss of the spherical architectures compared to their Huclidean counterparts. This is reflected in the single- and five-step results in the table and visualized in Figure 5 which depicts a drastic reduction distortion errors towards the poles for the 52 neighborhood Transformer its Euclidean counterparts. We that training on larger time-steps caused all transformer models (including the S2 models) to be unstable on autoregressive roll-outs, which non-trans non-transformer 8 op-opiime convolutional, neural-operator models reported 6 30
highlight the significantly
do not seem suffer from.
2 Transformer, step
2 Transformer, step
Ground truth, step
Initial condition
122 Transformer, 5 steps
S Transformer, 5 steps
Ground truth, 5 steps
Figure 5: Comparison of single step and five-step autoregressive predictions of both local R2 and 52 Transformers ground truth solution of the shallow water equations.
4.4
Ablation study
We conduct an ablation study on the shallow water equations setting to assess the contribution of individual architectural choices. Starting from the general-purpose vision transformer (15), we incrementally introduce spherical convolution, spherical attention, and local attention to evaluate

---

their individual and combined effects. Training losses and validation metrics after 5 autoregressive steps are reported Table The results imply that incorporating the inductive biases spherical geometry and locality into encoding and attention layers is highly effective in obtaining competitive performance on the given task and other geophysical settings. We remark that the changed from learned position embeddings to spectral embeddings decreases the skill as it constitutes significant reduction in trainable parameters. Moreover, some improvements are only notable as you after multiple autoregressive steps, such as the changes from global IR* to local S attention.
Table 4: Ablation results on the shallow water equations on the rotating sphere. Validation reports are reported for autoregressive steps.
Model
Training loss
Validation loss +
L1 error +
L2 error + 1.013 + 0.017
VIT 11
0.463
1.202 + 0.043
0.805 + 0.014
patch embeddings
0.042
6.753 + 0.948
1.384 0.060
2.210 + 0.137
I conv. encoder
learned pos. embed.
0.053
40.361 = 8.379
3.275 0.310
5.285 + 0.529
spectral pos. embed.
I conv. encoder
0.003
0.079 + 0.011
0.208 + 0.013
0.261 + 0.016
+ 52 conv. encoder
IR22 attention
0.003
0.062 + 0.007
0.193 + 0.011
0.242 + 0.013
S attention
5 global attention
0.003
0.034 + 0.006
0.141
0.012
0.177 + 0.014
Sa local attention
Limitations and future work
The unstructured nature the neighborhoods in our S method requires custom CUDA kernels that are well optimized, and show increased performance over the full attention methods using heavily optimized SDPA kernels available within PyTorch. The NATTEN X baseline comparisons provides optimized neighborhood attention kernels (for a fixed, rectangular neighborhood size), including support for lower precision data types, such as Float16 and BFloat16. Our custom kernel employs vectorization over features and optimized memory access but utilizes 32 bit floating point type. We will investigate if our kernel can benefit from lower precision data types as well. Furthermore, we aim implementing distributed memory support for both global and neighborhood spherical attention layers, similar to other layers implemented in sarch-harmonics
package used in our
Conclusion
We have generalized the attention mechanism in its continuous formulation the two-dimensional sphere, obtaining an equivariant formulation. Moreover, a spherical variant of neighborhood attention using geodesic distance has been derived. These approaches are implemented as neural operators via quadrature rules for practical spherical discretizations.
In conjunction with other spherical signal- iignal-processigg techniques, obtain spherical counterparts of popular Vision Transformer architectures such as the ViT, Neighborhood Transformer, and SegFormer. The effectiveness of our method has been demonstrated for three distinet learning tasks. The new spherical Transformers consistently outperform their Euclidean counterparts, demonstrating the importance of geometrically faithful treatment of such problems. Considering the growing importance of transformer architectures in scientific and engineering domains, we foresee this method becoming valuable approach for researchers and practitioners seeking to advance the state of the art.
Acknowledgments
We are especially grateful to our colleague Mauro Bisson for helping us write and optimize CUDA kernels. Moreover, we are grateful Alexander Keller, J.P. Lewis, and Vishal Mehta for fruitful discussions and proofreading the manuscript.
10

---

References
[1] Iro Armeni, Sasha Sax, Amir R. Zamir, and Silvio Savarese. Joint 2d-3d-semantic data for indoor scene understanding. arXiv preprint arXiv:1702.01105, 2017.
[2] Serge Assaad, Carlton Downey, Rami Al-Rfou, Nigamaa Nayakanti, and Ben Sapp. Vn- transformer: cotation-equivarian attention for vector neurons. 2022. URL http://aaaa /arxiv .41766
[3] Yaniv Benny and Lior Wolf. SphereUFormer: A u-shaped transformer for spherical 360 perception. 12 2024. URL http::/arxiv... http://axxvvvv..aaaaa. a412 06968
[4] Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619:533-5388 2023. ISSN 14764687. doi:
12--06
[5] Boris Bonev, Jan S. Hesthaven, Francis X. Giraldo, and Michal A. Kopera. Discontinuous galerkin scheme for the spherical shallow water equations with applications to tsumami modeling and prediction. Journal of Computational Physics, 2018. ISSN 10902716. doi: 10.1016/j.jcp. 8.0.002.
[6] Boris Bonev, Thorsten Kurth, Christian Hundt, Jaideep Pathak, Maximilian Baust, Karthik Kashinath, and Anima Anandkumar. Spherical fourier neural operators: Learning stable dynamics on the sphere. Proceedings of the 40th International Conference on Machine Learning. 012:2806-282.) 6 2023. URL //txxx/vvvv/.//////////////awav. org/ ha///bb//22000 2206. 03838
[7] Salva Rühling Cachay, Peetak Mitra, Haruki Hirasawa, Sookyung Kim, Subhashis Hazarika, Dipti Hingmire, Phil Rasch, Hansi Singh, and Kalai Ramea. Climformer-a spherical transformer model for long-term climate projections. In Proceedings of the Machine Learning and the Physical Sciences Workshop, NeurIPS 2022, 2022.
[8] Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Edgar Dobriban, and Kostas Daniilidis. Se(uv-cquivariant attention networks for shape reconstruction in function space. 2 2023. URL
http://arxiv.. wwtp//p/////....222222/ 02394
[9] Jiale Cheng, Xin Zhang, Fenqiang Zhao, Zhengwang Wu, Xinrui Yuan, John H. Gilmore, Li Wang. Weili Lin, and Gang Li. Spherical transformer cortical surfaces. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 13583 LNCS, pages 406-415. Springer Science and Business Media Deutschland GmbH, 2022. ISBN 9783031210138 doi: 10. 00.1007/978-30011-210
[10] Sungmin Cho, Raehyuk Jung. and Junseok Kwon. Spherical transformer. 2022. URL
http://arxiv.org. wtt.:r/g/.baaa..aaa/.////// 04942
[11] Oliver J. Cobb, Christopher G. R. Wallis, Augustine N. Mavor-Parker, Augustin Marignier, Matthew A. Price, Mayeul d' Avezac, and Jason D. McEwen. Efficient generalized spherical
CNNs. 10 2020.
org/ abs/ 11661
[12] Taco S. Cohen and Max Welling. Group equivariant convolutional networks. 2016. URL
[13] Taco S. Cohen, Mario Geiger, Jonas Kochler, and Max Welling. Spherical cnns. International Conference on Learning Representations, 2018. URL http://arxiv.orgg/bbe
10130
[14] Michaël Defferrard, Martino Milani, Frédérick Gusset, and Nathanaël Perraudin. DeepSphere: a graph-based spherical CNN. In International Conference on Learning Representations (ICLR), 2020. httphtttts:://oeeeeeet
net. het//forum/.d-B1e3011.5
[15] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. International Conference on Learning Representations, 2020. URL. http://arxtt www//rr///////ooooooooo / 11929

---

[16] J.R. Driscoll and D.M. Healy. Computing fourier transforms and convolutions on the 2- sphere. Advances in Applied Mathematics, 25:202-250, 1994. ISSN 01968858. doi: 10. 10.1006/aama.1999 1994. 1008.. URL httpp::/ www..iersreer..... wttp://re.././/eerree///////////
$0196885884710096
[17] Carlos Esteves, Christine Alen-Blanchhete, Ameesh Makadia, and Kostas Daniilidis, Learning so(3) equivariant representations with spherical cnns. 11 2017. URL http::ttt//aaaa //tpxv/awxvvvv//a org/ abs/1711.06711
[18] Carlos Esteves, Ameesh Makadia, and Kostas Danilidis. Spin-weighted spherical cnns. Ad- vances in Neural Information Processing Systems, pages 8614 8625, 2020. URL http:
//arxiv.org/ wwtriv..rrgabb//0000 aas.ab/00 10731
[19] Carlos Esteves, Jean--ejJean-aaqquuee Slotine, and Ameesh Makadia. Scaling spherical cnns. Proceedings of the 40th International Conference on Machine Learning, pages 9396-9411, 6 2023. URL attp://arae http://aaxiv...o org/ http:/bb//2//55 05420
[20] Fabian B. Fuchs, Daniel E. Worrall, Volker Fischer, and Max Welling. Sea))-transformmes:: 3d roto-translation equivariant attention networks. 11 2020. http:://arrxxww.r///aabbba 2006. 10503.
[21] Francis X Giraldo. A spectral element shallow water model on spherical geodesic grids. Technical report, 2001.
[22] Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. Neighborhood atten- tion transformer. Proceedings of the IEEF/CVF conference computer vision and pultern recognition, pages 6185-6194, 2023. URL
[23] Ali Hassani, Wen-Mei Hwu, and Humphrey Shi. Faster neighborhood attention: Reducing the o(n2) cost of self attention the threadblock level. Neural Information Processing Systems, 2024. htttltttp///axat , org/ abs/ 2203 04690
abs//403,,0.
[24] Maximilian Herde, Bogdan Raonić, Tobias Rohner, Roger Käppeli, Roberto Molinaro, Em- manuel de Bézenac, and Siddhartha Mishra. Poseidon: Efficient foundation models for PDEs.
5 2024. URL UtL/tttp://arxi www./rg/./.ooo.o.......o.oee 2atss/2405..111
[25] Pavel Iakubovskii. Segmentation models. models. pytorch 2019.
[26) Matthias Karlbauer, Nathaniel Cresswell-Clay, Dale R. Durran, Raul A. Moreno, Thorsten Kurth, Boris Bonev, Noah Brenowitz, and Martin V. Butz. Advancing parsimonious deep learning weather prediction using the healpix mesh. Journal of Advances Modeling Earth Systems, 16, 8 2024. ISSN 19422466. doi: 00.10292023MS00422...
[27] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. 12 2014. URL httt:tttt//xiii org/ 1412. 6980
[28] Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces. 2021. URL
awwaabb//...../ 2108. 08481
[29] Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs. 2 2023. http:////// org/ wwt.:orrg/abbaeeeeeee 11990
[30] Miguel Liu-Schiaffini, Julius Berner, Boris Bonev, Thorsten Kurth, Kamyar Azizzadenesheli, and Anima Anandkumar, Neural operators with localized integral and differential kernels. 2024. URL htttp::/aarxv..rrrgg//
16845
[31] llya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. utt.httpp:/arrii.. wwwpwwwwwww.i.......o//aaaabaaaa arg/abs/ 1711. 011001
[32] Jason. D. McEwen and Yves Wiaux. A novel sampling theorem on the sphere. 10 2011. doi: 10.1109/TSP.201122266 1.2666944
URL http:://arxx..rr//bbb///////// http://11...020ttt:////o
doi. ori/o1 1TPP 2011 2166394
12

---

[33] Ramachandran D. Nair, Stephen J. Thomas, and Richard D. Loft. A discontinuous Galerkin global shallow water model. Monthly Weather Review, 133:876-888 888, 4 2005. ISSN 0027- 0644. doi: 10. N0.1175/WR220..11
httpttps::/// h/ooo/////// ametsoc org/doi/00.1175/
MWR2903.1
[34] Jeremy Ocampo, Matthew A. Price, and Jason D. McEwen. Scalable and equivariant spherical cnns by discrete-continuuus (disco) convolutions. 9 2022. URL http::/arxii .rgz///..g////o. abs/
2209. 13603
[35] Marc RuBwurm, Konstantin Klemmer, Esther Rolf, Robin Zbinden, and Devis Tuia. Geographic location encoding with spherical harmonics and sinusoidal representation networks. Interna- lional Conference on Learning Representations, 10 2023. URL http::/arxiv.. wrgg abs/
2310.06743
[36] Imanol Schlag, Kazuki Irie, and Jürgen Schmidhuber. Linear transformers are secretly fast weight programmers. In International conference on machine learning, pages 9355-9366. PMLR, 2021.
[37] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 6 2017. URL http::/tta/iiiitit/t// org/ absa 1706 0766103766.77222
[38] Enze Xie, Wenhai Wang. Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo. SegFormer: Simple and efficient design for semantic segmentation with transformers. 5 2021. URL httt::///r/xii org/ abs// /ta.//2/5 15203
[39] Yinshuang Xu, Dian Chen, Katherine Liu, Sergey Zakharov, Rares Ambrus, Kostas Danilidis, and Vitor Guizilini. se(3) equivariant ray embeddings for implicit multi-view depth estimation. 11 2024. URLpttp::/ ptttt:://arxiie org/ herggabb/24411 07326
[40] Zhihui Yang, Qingyong Zhang, Wanfeng Chang. Peng Xiao, and Minglong Li. Egformer: an enhanced transformer model with efficient attention mechanism for traffic flow forecasting. Vehicles, 6(1): 2024.
13

---

A Signals on the sphere
A.1 Grids and quadrature rules
Our method deals with real-valued signals a S² - IR" defined on the unit sphere
sin 1 cos yp sin 0 sin cos 0
x x00, ()
(8)
Processing these signals requires discretizations by means of a sampling scheme. While there are various sampling theorems on the sphere [16] 32), restrict our discussion sampling schemes motivated by quadrature rules. Many of the operations discussed in this chapter require evaluating integrals of the form
/ u(z) dp(x)) (a(x))/[a 2T J. u(x(0, (le(0,,p)) 4)) sin 2 de dip, S2
(9)
where a E (S") is an integrable function on the sphere and p(z) denotes the Haar measure on the sphere. We approximate this integral using a quadrature rule
I u(x) dp(x)
Pipal
u(s,) Qi.
(10)
which is characterized by choice of grid points {x, {r and corresponding quadrature weights {ge Veyy While our methods are applicable to many types of grids, we typically use the equiangular grid (x(0, Ao {1:}. {4,}} defined by
d. wi/Nat
for i 0, 1, ....., Taat 1, 1,
(I
Pj = 2nj/naa
for j = 0, 1,
(11b)
where aaat and Mon denote the number of grid points in latitude and longitude. The associated quadrature weights are
2x
sin Di, Tea and approximately sums to 4x. This choice of quadrature weights correspond to trapezoidal quadra- ture weights in spherical coordinates.
45
(12)
A.2 Loss functions and metrics
For two real-valued signals on the sphere u, u² S R, we define loss functions for the purpose of regression and classification tasks. All of the losses are integrated over the sphere which numerically evaluated using a quadrature rule.
Lp norms
The Lı distance on the sphere is defined via the (S(2) norm follows:
/ Iu u²| du(x))
L [u, w²] ||u u" (9.. 4t
(13)
In the same fashion, the squared L2 distance is defined as
1
C u, u'| = ||u u" (12,(s9))
(u - u²)² du(x).
(14)
4x
Sa
Sobolev norm The spherical Sobolev I semi-norm is defined as
||u u ---u"llwaasss
1 4m
s2 sin 0s(u un) + 18g u²)l da(x).
(15)
The loss Ldepth be composed as combination of the Sobolev WI. semi-norm and the L distance:
Lasph |/u == (1)())))) A||u u x'Ilw-.(s?))
(16)
with À 0.1 to approximately match the magnitudes of both components the depth estimation task.
14

---

Cross entropy loss
The cross entropy loss Les can be defined as follows
1 -
/
LCE
CrossEntropy
(17)
4T
for C' classes, and the predicted and ground truth functions a and u². The point-wise cross-entropy is given by
exp(logits(ue))
CrossEntropy(u u²)
Σ c=1
u
log
(18)
i= exp(logits(u:)
and the logits operator is given by
logits(w) u)) = log
(19)
1 1
Intersection over Union (IoU) IoU is a commonly used metric for segmentation quality that captures the correctly guessed segmentation classes normalized by the sum of the correct predictions and all false predictions. The loU is given by
Tp
(e)
ToU
(20)
E T + ( (0)
where for given class T/ The) 1/o) F(()) denote area fraction associated with true positives, true negatives, false positives, and false negatives, respectively. Those can be computed from the predictions u, and ground-truth values u via
The 1
/
up n nut du(x))
(21a)
4m
Ssa
1
F(c)
nu du(x))
(21b)
4T
S
1
P(o)
u nue du(x),
(21c)
4T
s2
1
T(c)
u-c nu" du(x).
(21d)
4
5²
(21e)
For each class c, those quantities satisfy
T(e) + F(e) + ((0) T( =1
(22)
Accuracy (Ace) For segmentation evaluation, the segmentation accuracy
C T(e)
Acc = / M [(c) P
(23)
the fraction of true predictions in the overall domain, averaged over all classes
A.3 Spherical harmonics
The spherical harmonic functions (1, define a set of orthogonal polynomials on the sphere. They are defined as
(m () (-1)m²) cm (cos tmm = -Pm (cos 2 imp.
(24a)
21 (1 m)!
cf
(24b)
4m (1 + m)!
'Note that 200 sometimes referred to as numerator and denominator separately. This definition is more stable than it's avero-averaged counterpart, where the average is computed the ratio instead. For this work. the micro-averaged expression.
where the class averages are computed over
15

---

where (I 1) are the associated Legendre polynomials. The normalization factor of" is chosen to normalize the spherical harmonics w.r.t. the (52) inner product, s.t.
(Tm Yo (2) dp(x) mm.m
(25)
ga
B Implementation details
B.1
Discrete realization
With an appropriate quadrature rule applied to the integrands of the continuous formulation, we get the discrete attention
Nys
exp (T
(x) k(x))/Vd) v(Iz) wg.
st (g, vz)
(26)
Niia 11 exp (q=(x) k(x1///d) wi
followed by the discrete neighborhood attention
Ngra
1 (xz) exp (x) m(2z) wi
Attng= ka, =
(27)
Ei=l (x)) exp q" kmii/vV) w7
Naii
B.2 Global spherical attention implementation
Our global spherical attention mechanism leverages Py native scaled dot product attention (SDPA), where quadrature weights are incorporated via the framework's additive attention mask. SDPA applies additive attention masking, i.e. by adding the mask tensor the q k logits prior to computing the softmax. We can translate this into multiplicative weighting of the exponential functions by encoding log(w:: directly into the mask-a valid approach for strictly positive weights
(w, > 0).
B.3 Spherical neighborhood attention implementation
Our implementation for spherical neighborhood attention follows the ideas used to derived discrete continuous convolutions the sphere (DISCO) by (34) In this work, spherical convolution be described by a contraction of sparse matrix vb. which encodes the geometry of the problem, with dense input tensor of shape B C H x W:
alat-1 nlon-1
yib, k c, h = Σ Σ
vik h h, w/] C, w/ w]
(28)
Here, 0 k K and K is the dimension of a chosen set basis functions which support as well as coefficient values encoded in v. The final result of the convolution is obtained by contracting the channels e and basis indices k with a learnable weight tensor.
Note that 1h only depends on the input and output grids (via it's sparsity structure and quadrature weights for the input grid wx) as well as the number and type of basis functions. However, does not depend on the number of channels or any learnable parameters. Therefore, vá can typically be re-used across many different layers in contemporary neural networks. Additionally, d does not depend on w because all grids considered in this work are translationally invariant in longitude direction. This again reduces the memory footprint of DISCO convolution layers.
In order to implement spherical neighborhood attention, we modify equation (8). In this specific case, we can choose = and do not need to store any basis coefficient values, as out attention kernel (5) computes the appropriate neighbor weights dynamically. In this case, 1h is basically just an indicator function [h, W/, w/). which is equal to one for all Jyy (h, 0) - x (h', w w²)ls² zero otherwise. We can write:
Ocnea and
16

---

nlat- 1 nlon-1
ylb, (,,,cho w. = Σ
ulb c, Wi, w/ + w)
=0=0 w
h h', w/l exp -l= glb, 1, h, w| 1lb,i, W, w/ + ww// Vd) w|h]
x
(29)
Inlat-1 -n"=0 1h,, h2, w"] exp Ei=o q(b, 1, h, w] k|b,, 1, h², w" + w// vd) w w[h"]
As noted, the desire for weighted spherical neighborhood of our attention model 6 comes with the downside that we cannot leverage pre-existing optimized attention implementations, which do not parameterize quadrature weights the shape of the neighborhood.
B.4 Spherical neighborhood attention gradients
Beyond the CUDA implementation of forward attention A, have derive and implement the backward passes of the model. If we let qi = g(z;), Ky == k(xz) and ouj == exp (g: - ky) wg, starting with the Jacobian w.r.t. qo which requires use of the quotient rule, we define the terms
s = Σ U3
S' = Σ
aighi
9 = Σ osj
g' = Σ Kiykij
which combine to form the the Jacobian
0A dy = (f 02)9 f(dy . u,)g')
Dq:
Σ
(digk((dy) uz)) Σ aig -
C(sss - uz)) > (aigki))
3
-2
Σ
(ai))
(30)
Next we will show the Jacobian for ky. which uses the softmax derivative identity exp(p:)
da
(p): Ej exp(p,)'
p.òie
P.Pp.
dp,
With this identity, we can derive the Jacobian for ky
0A dy
Σjqau
dyy dy E, (dys U1))
(31)
0k, Ej anj
Eg Osi
The Jacobian for is simply
0A 0ug
Ejandy E,au ais
dy:
(32)
B.5 Spherical neighborhood attention CUDA implementation
With the discrete neighborhood attention formulation, we implement the necessary forward and backward routines in a custom CUDA extension to ensure reasonable performance for our S neighborhood experiments. The implementation is thread parallel over the neighborhood points ky, with output points (g:) parallelized over thread blocks. Due to this choice of parallelism, the softmax is computed in two steps, computing the necessary max(q k) required to avoid overflow in the exponential first, and finalizing the softmax sum per query point in the second step. This pattern is repeated for the gradient kernel, which additionally fuses y, k; v gradients into a single kernel. This parallelization strategy diverges from traditional optimized attention kernels, and we leave further optimizations and restructuring of this kernel to future work.
17

---

C Experiments
C.1
Experimental setup
Standard convolutional networks and Vision Transformers are designed for planar data and struggle with spherical inputs due distortions introduced by equirectangular projections. These distortions, particularly near the poles, break translation equivariance and lead to inconsistent feature extraction, limiting performance on tasks requiring spherical understanding. To address these limitations, we implement spherical generalizations of these architectures.
Baselines We compare against standard Euclidean models applied equirectangular projections, including the Vision Transformer (ViT) [[]] and SegFormer (3). These models operate on regular 2D grids and perform attention or convolution uniformly.
VIT employs convolutional patch embedding with kernel and stride size 2 x as well as global atten- tion. To allow flexibility with input sizes not divisible by the patch stride, we also evaluate modified Transformer that uses overlapping 3 x 3 patch embeddings and replaces the decoder's reshaping with bilinear interpolation. Both Transformers 128, GELU activation, and either layer or instance normalization. The Transformer neighborhood variant uses localized attention with 7 kernel. ViT uses learnable positional embeddings, while Transformer makes use of spectral positional embeddings. The standard Transformer has 531,968 parameters, while the neighborhood variant has 532,480 parameters.
4 attention layers, embedding dimension of
SegFormer follows a hierarchical architecture with four stages. It uses embedding dimensions [16, 32, 64, 128), attention heads [1, 2, 4, 8), and layer depths [3, 4. 6, 6.3). Convolutions use 4 4 kernels, and models include MLP ratios 4.0, dropout rates of 0.5, and drop path rates of 0.1. Neighborhood versions of SegFormer apply local attention using 7 x 7 attention kernel. The resulting standard SegFormer has 688,321 parameters, and its neighborhood variant has 689,265 parameters.
Spherical counterparts To evaluate the effectiveness of the spherical counterparts, we compare against spherical models that mirror their respective Fuclidean baselines, with matching embedding dimensions and parameter counts. The Spherical Transformer follows the ViT structure but replaces patch embeddings and global attention with discrete-continuous convolutions and attention on the sphere. The localized variant restricts the attention window to neighborhoods with a geodesic cutoff radius of 7 / ( T(a)) radians, where Tiat is the latitudinal dimension, matching the effective receptive field of the planar 7 x 7 attention window at the equator. Both the global and localized spherical Transformer models have 531,968 parameters.
Spherical SegFormer models replicate the hierarchical encoder-decoded structure the Fuclidean Seg- Former, with identical stage-wise dimensions, heads, and depths. They employ discrete-continunn convolutions for the encoding-decoding steps. Down-sampling and up-sampling use overlap-basee patch merging and spherical convolutions to preserve structure without distortion. The spherical SegFormer has 606, 113 parameters, consistent across both standard and neighborhood versions.
All spherical models employ GELU activation, instance normalization, and spectral positional embeddings. For the spherical linear basis functions equivalent to the parameter count of their Euclidean counterparts kernels (e.g..
contreee-ootiuuouss convolutions, we set the number of piecewise
9 basis functions for a x 3 kernel).
C.2 Datasets
Segmentation of spherical data To facilitate future usage of the Stanford 2D3DS Dataset M. we report additional details regarding its usage. The semantic labels is provided as json file in the GitHub repository, while the image data itself is hosted at /https://cvg data. inf ethz ch/ ch/2d3ds/no. /t///oo_xx no_xy// Each picture's 's segmentation information chair_9 wwt__onffeeccceeeeeeeeeee 1_4 labeled to account for both the type of object (e-g., chair, ceiling) and the room information (e.g., conferenceRoom, 1_4), where we drop the additional room information leaving us with only 14 classes. We additionally note that the semantic labels were built such that one label 855309 overflowed to 3341 (due to uint8 handling of integers), which is ignored when using NumPy L.x.
18

---

Table 5: Overview of spherical models and Euclidean counterparts.
Model
Encoder/Decoder type
Attention type
Parameter count
R2
Transformer
2D convolution
Global
531,968
R²
Transformer (local)
2D convolution
7 x 7 window
532,480
S
Transformer
5a convolution
Global
531,968
S
Transformer (local)
5
convolution
Geodesic window
531,968
2
SegFormer
2D
convolution
Hierarchical
688,321
R²
SegFormer (local)
2D
convolution
7 x 7 window
689,265
S SegFormrr
5
convolution
Hierarchical
606,113
2 SegFormer (local)
Sa convolution
Geodesic window
606,113
but as of NumPy 2.x, this overflow caught, which we fixed in our implementation, but needed to account for the assumed overflow in the semantic labels json file.
Depth estimation on spherical data We adapt our approach to predict per-pixel depth values from single-view panoramic images. We leverage the Stanford 2D3DS dataset's panoramic depth maps. which are stored as 16-bit PNGs with a maximum measurable distance of 128 m and a sensitivity of 1/512 m. Each depth value represents the Euclidean distance from the camera center point in the scene. To recover the metric depth from the RGB channels of the PNG images, we use the following formula:
256G + 256
(33)
512
where R, G. and denote the 8-bit values of the red, green, and blue channels, respectively. This linear transformation preserves the original 16-bit precision while accommodating PNG storage constraints. To address invalid or distorted regions inherent in the equirectangular projection we apply a mask covering 15% of the latitude at both poles to climinate areas with severe distortion. The remaining invalid pixels, indicated by the value 2 1, are filtered out. Additionally, the data is standardized to have zero-mean, unit-variance distributions across all batches.
Shallow water equations on the rotating sphere The shallow water equations on the rotating 2-sphere model thin layer of fluid covering rotating sphere. They are typically derived from the three-dimensional Navier-Stokes equations, assuming incompressibility and integrating over the depth of the fluid layer. They formulated system of hyperbolic partial differential equations
O.p + V. (pp) (pp))) + V T S
in S2 (0)
in 52 x (0, 0o) on 52 x {t 0}. on 52 { 0}.
(34)
0 P0
uo
The state (4p, you") contains the geopotential layer depth & (mass) and the tangential momentum vector you (discharge). In curvilinear coordinates, the flux tensor T can be written using the outer product as you @ u. The right-hand side contains flux terms such as the Coriolis force. A detailed treatment of the SWE equations can be found in e.g. 21 5 33).
Training data for the SWE generated by randomly generating initial conditions and advancing them in time using classical numerical solver. The initial geopotential height and velocity fields are realized as Gaussian random fields on the sphere. The initial layer depth has an average of iPag 103 g with standard deviation of 120 9. The initial velocity components have zero mean and a standard deviation of 0.2 * Payg. The parameters of the PDE, such as gravity, radius of the sphere and angular velocity, choose the parameters of the Earth. Training data is generated on the fly by using a spectral method to numerically solve the PDE on an equiangular grid with a spatial resolution of 256 512 and timesteps of 150 seconds. Time-stapping is performed using the thirr-order Adams-Bashford scheme. The numerical method then computes geopotential height, vorticity and divergence output.
This data is z-score normalized and the modes are trained using epochs containing 256 samples each. To optimize the weights, we use the popular Adam optimizer with a learning rate of 2 10
19

---

Table 6: Numerical results for the equivariance test. The test performed on equiangular grids of
various resolutions.
global S attention
local S attention
Resolution
Interpolation error
Equivariance error
Interpolation error
Equivariance error
13 x 24
0.321 = 0.096
1.097 = 0.658
0.360 = 0.040
0.879 = 0.119
25 x 48
0.190 0.047
0.674 0.372
0.212 + 0.018
0.541 0.063
49 96
0.105 + 0.040
0.220 0.118
0.106 + 0.010
0.222 + 0.030
97 x 192
0.052 0.028
0.070 0.038
0.045 0.007
0.086 0.014
193 x 384
0.018 + 0.015
0.021 0.015
0.014 + 0.003
0.028 0.005
385 X 768
0.003 + 0.001
0.008 + 0.002
D Equivariance test
To verify the approximate equivariance of the spherical attention formulation we study the equivari- ance error
V R- Attuga [ V V ui (x) - Attnsalu k, e)(x))
(35)
Attniaa (g, ki, v²[(2) ((
where R is the representation corresponding to the rotation R E SO(3). passively rotating a function LS*)),, such that V ]nf(x) = Equation 33) measures the approximate equivariance by comparing the result of applying the attention mechanism to a rotated signal and rotating it back to the result of directly applying it the original signal. In the perfectly equivariant case, this error is exactly O.
To perform this test numerically, we sample Gaussian process the sphere, which is defined by random, normally distributed spectral harmonic coefficients up to degrees m = 12. This process is repeated three times to generate random inputs g(x),, k(x) and v(z). The resulting functions are in turn rotated numerically by applying the rotation R to the grid on which the functions are represented and subsequently interpolating the result onto the original grid. We use cubic interpolation, while making sure that the periodicity at the boundary is respected. The spherical attention mechanism is applied and the result is rotated back using the to the attention mechanism directly applied to the original signals. As this procedure incurs an interpolation error due the numerical application of the rotation, compare the results to the interpolation error | [ R-1 V a(x) - a(x)ll (5a) (=ss))/|a((z)]] [||a(=)]] (2(5)) where the same rotation and it's inverse are numerically applied to the output of the attention mechanism a(x) = Attn (g. k, (1)).
method. The outcome is then compared
We perform the for the rotation R defined by the Euler angles a T/3, 3 T/5.7 x/2 7/2, but find these results remain qualitatively the same for other choices of R. The results are averaged over 32 different random initializations of q, k and and both mean and standard deviation are reported in Table 6) We observe that the equivariance error decays roughly at the same rate as the interpolation eтror incurred when rotating the signals, which indicates that the method is indeed approximately equivariant. Moreover, for the global S attention, we find that the equivariance error is only slightly larger than the interpolation error and roughly twice as large as the interpolation error the case of the local 52 attention. While it is not possible to clearly separate the contribution of the interpolation error to the equivariance error, this indicates that the equivariance error worst twice that of the interpolation error.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly articulate the main claims and contribu- tions of the paper. These sections accurately summarize both the theoretical and experimental results, and appropriately outline the scope, assumptions, and limitations of our work. The claims presented are consistent with the findings reported in the main body of the paper and do not overstate the generalizability of the results.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The limitations our work are explicitly discussed in the paper, in the context of presenting individual results. We address the main assumptions underlying our approach, discuss the robustness of our results to potential violations of these assumptions, and outline the scope and generalizability of our claims. Additionally, we comment on factors that may influence performance, and computational efficiency.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
21

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
This work focuses on empirical and methodological contributions rather than theoretical analysis. While the paper includes algebraic derivations to explain its approach, it does not present formal theorems, lemmas, or proofs that would fall under the category of theoretical results, All claims are substantiated through experimental validation rather than mathematical proofs.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed description of the experimental setup, including datasets, model architectures, hyperparameters, and evaluation protocols, ensuring that all key aspects necessary for reproducibility are covered. Additionally, the code used to conduct the experiments publicly available on through an anonymized repo (and eventually GitHuub,, further facilitating independent verification and replication of the results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
22

---

(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code necessary to reproduce all main experimental results is openly available on an anonymous library (and eventually GitHub), accompanied by detailed installation and usage instructions. These instructions specify the required environment, dependencies, and exact commands run the experiments. Additionally, we provide guidance on accessing and preparing the datasets used in our study.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips wtwp//ublccgggiiecocceooccooeeee guwdes/deeeeee /ooeeSe/mooeeemiaaaac wwwmiisssonnooo ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides a thorough description of the experimental setup, including details on data splits, selected hyperparameters, optimization methods, and the procedures used for hyperparameter selection. These details presented in the main text and sup- plemented with additional information in the appendix and the publicly available code repository.
Guidelines:
The answer NA means that the paper does not include experiments.
23

---

The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: All evaluation metries and their corresponding error measurements are pre- sented with appropriate explanations and referenced in the relevant tables and figures.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper specifies the hardware used for all experiments, including details on the type of compute resources, and execution times for key experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: :ttp::::///euiis.....o http://urrppcccppppca/hcceeeeeae
Answer: [Yes]

---

The research fully adheres to the NeurlPS Code of Ethics. The authors have carefully reviewed the guidelines and ensured compliance regarding data privacy, consent, copyright, fairness, and transparency. All datasets were used in accordance with their licenses. No aspect of the work was found violate the ethical standards set by NeurIPS.
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [NA]
Justification: This paper presents methodological improvement to attention mechanisms for machine learning tasks on spherical data. As the contribution is foundational and not tied to any specific application or deployment scenario, we do not foresee direct negative societal impacts arising from this work. The method is broadly applicable and does not introduce any new risks or harms beyond those already associated with general-purpose machine learning algorithms.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairesss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The methods and resources presented in this paper do not involve the release of data models that carry high risk of misuse. As such, specific safeguards necessary. The work focuses on foundational algorithmic improvements and does not introduce new risks beyond those already present in standard machine learning research.
Guidelines:
25

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external assets used this paper, including the Stanford 2D3Ds dataset, are properly eredited and cited in the relevant sections. We explicitly mention the dataset version, provide appropriate references, and acknowledge the original creators. The license and terms of use for all utilized assets are clearly stated and strictly respected throughout the work.
Guidelines:
The answer NA means that the paper does use existing
The authors should cite the original paper that produced the code package dataset.
The authors should state which version asset used and, possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not introduce new assets. The code used an extension of the publicly available anonymous library) which is published with clear documentation and instructions for reproducing the experiments. All datasets used are publicly available and properly cited, with appropriate credit given to their original creators.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
26

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs were not used an important, original, non-standard component in the core methodology of this research. Their use was limited to writing, editing, or formatting the manuscript.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy for what should or should be described.
27


Reviews:
[{"review_id": "uDma81bVi8", "reviewer": "Reviewer_dPKj", "summary": "This paper revisits attention but on spherical data. Attention mechanisms are derived clearly and implemented successfully on spherical data. The method outperforms planar counter parts in clear evaluation of the method.", "questions": "Minor additional ablation study could improve the paper.\nFuture work may investigate alternative ways to compute convolutions of the transformer.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "+ The paper has clear theoretical contribution to attention with spheres\n+ An implementation is made available and outperforms planar transformers in evaluation\n+ Computational runtime is evaluated\n\n- while the evaluation is fairly complete, the paper would benefit from ablation study which tests equivariant transformations of the sphere\n- other spherical convolution method could enhance the experiments. For example 2D3DS is frequently reported in spherical convolution works.\n- I am wondering if an icosahedron mesh could help with efficiency issues. See\n\n[1] Chao Zhang, Stephan Liwicki, Sen He, William Smith, Roberto Cipolla:\nHexNet: An Orientation-Aware Deep Learning Framework for Omni-Directional Input. IEEE Trans. Pattern Anal. Mach. Intell. 45(12): 14665-14681 (2023)\n[2]  Y. Lee, J. Jeong, J. Yun, W. Cho, and K.-J. Yoon, “Spherephd: Applying cnns on a spherical polyhedron representation of 360deg\nimages,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9181–9189.", "was_revised": true, "final_justification": "I am satisfied with the rebuttal.", "rebuttal": "First of all, we would like to thank the reviewer for taking their time and providing us with this review. In the following we address the reviewers concerns:\n\n**W1 - equivariant experiments**\n\nAs suggested by the reviewer, we have added an equivariance test to the manuscript. The convergence study is reported in Table 1, which shows equivariance and interpolation convergence for both the full and neighborhood transformer as resolution is increased. We see that the error is dominated by the interpolation error incurred when rotating the signal numerically.\n\n*Table 1 - Equivariance test*\nUpon suggestion from multiple reviewers , we have added an equivariance test. This test takes a Gaussian random linear combination of spherical Harmonics to create a vector-valued noisy signal on the sphere, which is cut off at l=12, m=12.Equivariance errors are reported alongside interpolation errors that arise from applying the rotations discretely. All errors are reported in terms of the L2 loss on the sphere.\n|                  |            S2 Transformer        |          |      Neighborhood S2 Transformer |    |\n|-------------|-------------------|-------------------|-------------------|-------------------|\n| Grid Size   |   Interpolation   |   Equivariance   |   Interpolation   |   Equivariance   |               |\n| 8x16        |            53.913 |           74.934 |            29.588 |           61.374 |\n| 16x32       |            28.630 |           47.374 |            23.750 |           40.656 |\n| 32x64       |            18.268 |           32.687 |            14.621 |           29.052 |\n| 64x128      |            11.470 |           18.350 |             8.852 |           15.690 |\n| 128x256     |             6.354 |            9.617 |             4.763 |            8.678 |\n| 256x512     |                 - |                - |             2.397 |            4.123 |\n\n\n**W2 - spherical convolution & other baselines**\n\nWe were able to run the suggested SphericalCNN[1] (icosohedral projection) model on our depth and segmentation datasets, with the results presented in the revised Tables 2 & 3. We additionally implemented the EGFormer model for the segmentation dataset with results also presented in Tables 2 & 3. That said, with the limited time for review, it is likely that both models could be improved. For example, we noted that our EGFormer[2] model experienced significant overfitting when training.\n\n*Table 2 - Segmentation*\nFor segmentation, we added two new baselines (EGFormer & Spherical CNN) and reran several S2 and R2 attention-based models to ensure comparability across the models.\n| Model                                    | Training loss ↓ | Validation loss ↓    | IoU ↑           | Acc ↑           |\n|-------------------------------------------|-----------------|----------------------|------------------|------------------|\n| **Results on 128×256 data**               |                 |                      |                  |                  |\n| S² Transformer                            | 0.32939         | 1.02 ± 0.572         | 0.578 ± 0.126    | 0.961 ± 0.015    |\n| S² SegFormer                              | 0.306           | **0.705 ± 0.307**    | 0.662 ± 0.102    | 0.970 ± 0.011    |\n| ℝ² SegFormer (local)                      | 0.289           | 0.817 ± 0.339        | 0.624 ± 0.103    | 0.966 ± 0.011    |\n| **S² SegFormer (local)**                  | 0.321           | 0.706 ± 0.303        | **0.658 ± 0.100**| **0.970 ± 0.011**|\n| EGFormer                                  | **0.203**       | 1.29 ± 0.487         | 0.586 ± 0.106    | 0.962 ± 0.012    |\n| **Results on 256×512 data**               |                 |                      |                  |                  |\n| Spherical CNN                             | 0.605           | 0.809 ± 0.287        | 0.590 ± 0.099    | 0.963 ± 0.012    |\n| ℝ² SegFormer (local, large)               | 0.626           | 0.812 ± 0.891        | 0.692 ± 0.158    | 0.972 ± 0.018    |\n| **S² SegFormer (local, large)**           | **0.515**       | **0.656 ± 0.719**    | **0.728 ± 0.136**| **0.976 ± 0.015**|\n\n*Table 3 - Depth Estimation*\nWe have updated the depth estimation data with new baselines (EGFormer & SphericalCNN). Please find the updated values below:\n\n| Model                        | Training loss ↓ | Validation loss ↓     | L₁ error ↓          | W¹,¹ error ↓         |\n|-----------------------------|-----------------|------------------------|---------------------|----------------------|\n| ℝ² Transformer              | 0.954           | 0.982 ± 0.386          | 0.215 ± 0.084       | 7.67 ± 3.22          |\n| S² Transformer              | 0.978           | 0.993 ± 0.381          | 0.234 ± 0.086       |**7.59 ± 3.23**      |\n| ℝ² Transformer (local)      | 1.014           | 1.023 ± 0.370          | 0.237 ± 0.079       | 7.86 ± 3.17          |\n| S² Transformer (local)      | **0.978**       | 0.976 ± 0.354          | 0.209 ± 0.068       | 7.68 ± 3.12          |\n| ℝ² SegFormer                | 1.037           | 0.980 ± 0.371          | 0.169 ± 0.058       | 8.11 ± 3.22          |\n| S² SegFormer                | 1.005           | **0.935 ± 0.369**      | **0.165 ± 0.057**   | 7.70 ± 3.23          |\n| ℝ² SegFormer (local)        | 1.055           | 1.002 ± 0.374          | 0.174 ± 0.061       | 8.27 ± 3.25          |\n| S² SegFormer (local)        | 1.014           | 0.948 ± 0.368          | 0.174 ± 0.061       | 7.75 ± 3.21          |\n| **Additional baselines**    |                 |                        |                     |                      |\n| Spherical CNN               | 1.019           | 1.098 ± 0.412          | 0.315 ± 0.123       | 7.83 ± 3.21          |\n| LSNO                        | 0.994           | 1.020 ± 0.387          | 0.248 ± 0.095       | 7.72 ± 3.19          |\n| SUNet                       | 1.006           | 0.985 ± 0.352          | 0.220 ± 0.061       | 7.65 ± 3.11          |\n| EGFormer                    | 0.831           | 0.945 ± 0.386          | 0.182 ± 0.084       | 7.63 ± 3.26          |\n\n\n**W3 - icosohedral mesh**\n\nThe choice of an equiangular discretization has several advantages, including allowing direct comparison against standard R2 attention methods, ease of visualization, and direct use of the data, which is frequently given in an equiangular format. Since submission, we have worked on our CUDA implementation with an improved parallelization strategy that enabled online softmax and coalesced channel memory accesses, which have dramatically improved the forward and backward kernel performance. That said, nothing rules out the use of an icosahedron mesh except for finding an appropriate quadrature rule to enable the discretized spatial integration, and adapting the implementation to use indirect addressing to gather the mesh elements within the geodesic neighborhood. The lower density of points at the poles of the sphere could offset the performance losses due to indirect addressing lookups.\n\n**Q - Alternate ways of convolutions of the transformer**\n\nIf the reviewer is referring to the choice of the encoding / decoding approaches in the spherical transformer, they were chosen to ensure equivariance throughout the architecture. In particular, because spherical convolutional patch embeddings can be seen as sparse convolution on the sphere.\n\n**References**\n\n[1] Chao Zhang, Stephan Liwicki, Sen He, William Smith, Roberto Cipolla: HexNet: An Orientation-Aware Deep Learning Framework for Omni-Directional Input. IEEE Trans. Pattern Anal. Mach. Intell. 45(12): 14665-14681 (2023) [2] Y. Lee, J. Jeong, J. Yun, W. Cho, and K.-J. Yoon, “Spherephd: Applying cnns on a spherical polyhedron representation of 360deg \nimages,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9181–9189.\n\n[2] Zelin Zhang, Tao Zhang, KediLI, Xu Zheng. EGFormer. arXiv preprint arXiv:2505.14014, 2025"}, {"review_id": "kVFoneRjqn", "reviewer": "Reviewer_qKvE", "summary": "The paper proposes an extension of the attention mechanism for spherical domains which is geometry aware and approximately rotationally equi-variant, enabling transformer architectures to process data natively defined on the 2D sphere. The authors derive continuous formulations for both global and neighborhood attention mechanisms on spherical domains, then discretize them using quadrature rules to ensure approximate SO(3) rotational equivariance. The method is implemented as neural operators that can handle arbitrary discretizations of the sphere. \n\nThe authors evaluate their approach on three diverse tasks: shallow water equation simulation, spherical image segmentation, and spherical depth estimation, demonstrating consistent improvements over euclidean counterparts. Popular architectures like ViT and SegFormer are systematically adapted for spherical domains by replacing standard attention mechanisms with either global spherical attention or neighborhood spherical attention. Additionally, the encoding and decoding laters are modified to handle spherical data by substituting conventional CNN operations with spherical convolutions, etc.", "questions": "- In Table 1, why do S2 Transformers perform worse than R2 transformers for both, local and neighbourhood variants?\n- For comparison with SphereUFormer, why double the embedding dimension? - to equate for number of parameters? \n- How does the quadrature-based approximation quality degrade with grid resolution? Can you provide empirical evidence of the approximate SO(3) equivariance?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper provides a rigorous maths derivation by extending attention mechanisms to spherical domains through proper geometric treatment. The continuous formulation followed by quadrature-based discretization is mathematically sound.\n- The method achieves approximate SO(3) rotational equivariance, a crucial inductive bias for spherical data that standard Transformers lack. This addresses key limitations when applying planar methods to spherical domains.\n- Custom CUDA kernel implementations for both variants, full and local spherical attention will be made available, making the approach practically applicable.\n- The continuous formulation makes these architectures neural operators, enabling evaluation on arbitrary sphere discretizations while preserving geometric structure.\n- The evaluation maintains similar parameter counts and hyperparameters (embedding dimensions, layer depths) between spherical and Euclidean variants, ensuring fair comparisons that isolate the impact of geometric considerations.\n- The inclusion of sample outputs across all three tasks provides valuable visual evidence of improvements, particularly highlighting reduced distortions near poles and better preservation of spherical structure.\n- In Table 3 and Fig 4 - all S2 variants beat their R2 counterparts for shallow water equations on the rotating sphere\n\n**Weaknesses**\n- The paper lacks thorough computational complexity analysis and scalability comparisons, especially for the custom CUDA implementations versus optimized baselines. \n- In some experiments (e.g., segmentation), the improvements over Euclidean baselines are modest. The IoU improvements are often within error margins, raising questions about practical significance. The paper claims that “Across all models, we see the spherical variants outperforming their Euclidean counterparts” - this is not exactly true as S2 Transformers are worse in most metrics in Table 1.\n- The evaluation primarily compares against naive euclidean applications to spherical data. Beyond brief mentions of SphereUFormer, comprehensive comparisons with other geometric deep learning approaches would strengthen the claims.\n- The experiments are conducted on relatively small datasets (e.g., 1,621 images for segmentation with only 35-40 test samples), limiting the ability to assess generalisation and robustness. Larger-scale evaluations would provide more convincing evidence of the method's effectiveness.", "was_revised": true, "final_justification": "I am satisfied with the authors' response and would recommend Accept.", "rebuttal": "First of all we would like to thank the reviewer for taking their time to review our manuscript and providing us with feedback.\n\n**W1 - Complexity & performance analysis**\nAt time of submission, the CUDA implementation was still too immature for serious complexity and performance experiments, particularly against optimized baselines like NATTEN or PyTorch’s full attention implementation. Since the submission, we have reworked the parallelization strategy to enable inline softmax and much improved coalesced memory access patterns such that the complexity benefit compared to full attention can be realized. Experiments done for this review show that updated local vs full S2 segformer is about 1.5x faster in our tests. Our implementation is made publicly available and we aim to provide timing information in the revised version of the manuscript.\n\n**W2 - Discuss segmentation and depth similarity across image models**\nWe have softened the language for the performance of the S2 models on the segmentation and depth datasets, which are within the experimental bounds of the R2 models. Moreover, we have discovered a bug in the implementation of the standard S2 transformer, which has made the results more consistent. Given the out-performance of the S2 models on the shallow water equations, we assume this similar performance is a result of the lack of detail at the poles of the spherical images, which doesn’t allow the S2 models to outperform the R2 models on depth and segmentation particularly at the poles.\n\n**W3 - More comparisons against other spherical techniques**\nWhile the focus of the paper is to prove the effectiveness of the spherical attention formulation, we agree that more baselines could be added. Upon suggestion we have added an implementation of Spherical CNNs and EGFormer to our segmentation and depth estimation datasets.\n\n*Table1*: Updated table for the segmentation problem\n| Model | Training loss ↓ | Validation loss ↓ | IoU ↑ | Acc ↑ |\n|--|--|--|--|--|\n| **Results on 128×256 data** | | | | |\n| ℝ² Transformer | 0.325  | 0.985 ± 0.390 | 0.588 ± 0.105 | 0.962 ± 0.012 |\n| S² Transformer   | 0.330 | 1.02 ± 0.572  | 0.578 ± 0.126 | 0.961 ± 0.015 |\n| S² SegFormer  | 0.306 | **0.705 ± 0.307** | 0.662 ± 0.102 | 0.970 ± 0.011 |\n| ℝ² SegFormer (local)  | 0.289  | 0.817 ± 0.339 | 0.624 ± 0.103 | 0.966 ± 0.011 |\n| **S² SegFormer (local)**  | 0.321  | 0.706 ± 0.303  | **0.658 ± 0.100**| **0.970 ± 0.011**|\n| EGFormer   | **0.203**       | 1.29 ± 0.487   | 0.586 ± 0.106  | 0.962 ± 0.012 |\n| **Results on 256×512 data**  | | | | |\n| Spherical CNN | 0.605 | 0.809 ± 0.287 | 0.590 ± 0.099  | 0.963 ± 0.012    |\n| ℝ² SegFormer (local, large) | 0.626 | 0.812 ± 0.891 | 0.692 ± 0.158 | 0.972 ± 0.018 |\n| **S² SegFormer (local, large)** | **0.515** | **0.656 ± 0.719**    | **0.728 ± 0.136**| **0.976 ± 0.015**|\n\n*Table 2 - Depth Estimation*\nWe have updated the depth estimation data with new baselines (EGFormer & SphericalCNN). Please find the updated values below:\n\n| Model | Training loss ↓ | Validation loss ↓ | L₁ error ↓  | W¹,¹ error ↓ |\n|---|----|--|---|--|\n| ℝ² Transformer | 0.954 | 0.982 ± 0.386 | 0.215 ± 0.084 | 7.67 ± 3.22 |\n| S² Transformer | 0.978 | 0.993 ± 0.381 | 0.234 ± 0.086 | **7.59 ± 3.23** |\n| ℝ² Transformer (local) | 1.014 | 1.023 ± 0.370  | 0.237 ± 0.079 | 7.86 ± 3.17 |\n| S² Transformer (local) | **0.978** | 0.976 ± 0.354   | 0.209 ± 0.068       | 7.68 ± 3.12 |\n| ℝ² SegFormer | 1.037 | 0.980 ± 0.371 | 0.169 ± 0.058       | 8.11 ± 3.22 |\n| S² SegFormer | 1.005 | **0.935 ± 0.369**  | **0.165 ± 0.057** | 7.70 ± 3.23 |\n| ℝ² SegFormer (local)        | 1.055   | 1.002 ± 0.374          | 0.174 ± 0.061       | 8.27 ± 3.25          |\n| S² SegFormer (local)        | 1.014           | 0.948 ± 0.368          | 0.174 ± 0.061       | 7.75 ± 3.21          |\n| **Additional baselines**    |  | | |  |\n| Spherical CNN   | 1.019| 1.098 ± 0.412          | 0.315 ± 0.123       | 7.83 ± 3.21          |\n| LSNO  | 0.994  | 1.020 ± 0.387 | 0.248 ± 0.095       | 7.72 ± 3.19          |\n| SUNet  | 1.006 | 0.985 ± 0.352 | 0.220 ± 0.061       | 7.65 ± 3.11          |\n| EGFormer | 0.831 | 0.945 ± 0.386 | 0.182 ± 0.084       | 7.63 ± 3.26          |\n\n\n**W4 - 2D3DS dataset size**\nThe lack of images is definitely a weakness of the 2D3DS dataset. However, we find that numerous works in the literature use this dataset to evaluate the effectiveness of spherical architectures. We chose to keep our architectures small as the goal of the paper was to evaluate the effectiveness of the spherical attention formulation over the euclidean counterparts, and not to develop a SOTA architecture. We believe that for this limited setting, this is sufficient.\n\n**Q1 - why do S2 transformers perform worse than R2 transformers**\nSince the submission of the manuscript, we have discovered a bug in the implementation of the S2 transformer, which has been fixed. Updated results are shown in Table 1. Both S2 and R2 perform very similarly, especially considering the standard deviation. \n\n\n**Q2 - Why double the embedding dimension in the SphereUformer comparison**\nThe results for SphereUFormer were taken directly from their paper and they were carried out on double the resolution. To bring our architecture into a similar parameter count, we doubled the embedding dimension.\n\n\n**Q3 - How does quadrature scale with mesh resolution, and empirical test of SO(3) approximate equivariance**\n\nWe thank the reviewer for this excellent suggestion. On equiangular grids such as for 2D3DS, we use the Clenshaw-Curtis quadrature rule, whereas on the Gauss grids (Shallow Water equations), we use Gauss quadrature. Theory suggests that both of these methods have an error that bounded by $\\mathcal{O}((2 N )^{-k} / k)$ for a k-times differentiable integrand. We have added a test to estimate the approximate equivariance in Table 3. In this test, equivariance is tested by rotating smooth signals (degree up to l=12, m=12) discretely and applying the attention mechanism to the rotated signal before rotating it back. The result is compared to a direct application of the attention mechanism. We find that the error is dominated by the interpolation error which is incurred when the signal is rotated discretely and observe that the total error is within 1.5x of the interpolation error incurred by the rotation. We have added a more detailed explanation of this test alongside the results to the manuscript.\n\n|                  |            S2 Transformer        |          |      Neighborhood S2 Transformer |    |\n|-------------|-------------------|-------------------|-------------------|-------------------|\n| Grid Size   |   Interpolation   |   Equivariance   |   Interpolation   |   Equivariance   |               |\n|-------------|-------------------|------------------|-------------------|------------------|\n| 8x16        |            53.913 |           74.934 |            29.588 |           61.374 |\n| 16x32       |            28.630 |           47.374 |            23.750 |           40.656 |\n| 32x64       |            18.268 |           32.687 |            14.621 |           29.052 |\n| 64x128      |            11.470 |           18.350 |             8.852 |           15.690 |\n| 128x256     |             6.354 |            9.617 |             4.763 |            8.678 |\n| 256x512     |                 - |                - |             2.397 |            4.123 |"}, {"review_id": "ZzCTZ7KecM", "reviewer": "Reviewer_WwPf", "summary": "The authors start with a continuous formulation of attention for the entire sphere as well as for neighborhoods defined by the geodesic distance. The continuous kernel integrals are discretized using quadrature and neighborhoods defined as above. The architecture is a modified ViT (but a Segformer can be used as well) having spherical convolutions for patch encoding, spherical multi-head attention, and a decoder with a bilinear spherical interpolation. Position encoding is spectral using the spherical harmonic basis. \nResults on the author's spherical version of ViT and Segformer outperform transformers defined on the regular grid of the sphere in segmentation, depth estimation, and the shallow water problem.", "questions": "Q1 (and W3): The authors do not mention what they do with the positional encodings. If they add them or concatenate them to the features then the invariance formula in line 117 does not hold anymore because $q$ and $k$ do not behave like scalars (except the $L=0$ order harmonics). The rest have to be multiplied with a $W_q$ that is the product of a radial function times the spherical harmonics basis as described, in the references above and here: \nhttps://alchemybio.substack.com/p/spherical-equivariant-graph-transformer\n\nQ2 (W4): Clarify the notion of continuous.\n\nAddress all other weaknesses.", "limitations": "none identified.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength: The paper handles in a correct way the discretization of the sphere by using neighborhoods defined by the geodesic thershold. Such a representation does not suffer from the singularity of the equirectangular grid or icosahedral domains.  \n\nWeaknesses: The paper has several weaknesses which I will list here and I would be eager to see the authors' responses and how the paper could be improved. \n\nW1: The authors claim as their main contribution the continuous formulation of the transformer. However, their implementation is the discrete sum of neighboorhood $k$ based on the geodesic distance. This is nothing else than a graph transformer with edges only between points where the geodesic distance is less than the $\\theta_{cutoff}$. \n\nW2: Related to W1, the authors misrepresent the literature in lines 81-84. All SO(3) equivariant transformers like the ones below have the spherical attention as a subcase, I will explain below.\n\nFuchs, F., Worrall, D., Fischer, V., & Welling, M. (2020). SE(3)-transformers: 3d roto-translation equivariant attention networks. Advances in neural information processing systems, 33, 1970-1981.\n\nand their implementation in Alphafold2.\n\nLiao, Yi-Lun, and Tess Smidt. \"Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs.\" The Eleventh International Conference on Learning Representations.\n\nW3: The authors ignore the processing of the positional encoding in the implementation.  The paper addresses only the scalar-valued version of an equivariant graph transformer. There are papers which combine scalar features (like the features in this paper) and spherical encodings like the equivariant perceiver I/O, for example, see 3.3.2 in:\n\nXu, Y., Chen, D., Liu, K., Zakharov, S., Ambrus, R. A., Daniilidis, K., & Guizilini, V. C. (2024). SE(3) Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation. The Thirty-Eighth Annual Conference on Neural Information Processing Systems. \n\nW4: The notion of continuous as used in the paper it is not in the sense of a neural operator as cited in [25]. Requirement of a neural operator (see Table 1 in [25]) is to to \"query the output at any point\". That would be achieved by a decoder that has as an input a query spherical coordinate vector and conditioned on the attention output compute the segmentation or depth value at this point on the sphere. This is, for example, done in the equivariant attention decoder in \n\nChatzipantazis, E., Pertigkiozoglou, S., Dobriban, E., & Daniilidis, K. (2023). SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space. Int. Conference Learning Representations ICLR. \n\nThe current paper though uses a bilinear interpolator in the decoder keeping a discrete structure. \n\nW5: The paper is limited to scalar valued functions failing to address applications like vector field prediction on the earth or even optical flow on images. In contrast, the SO(3) equivariant transformer papers can handle both vector  valued functions on any graph defined by the geodesic threshold as well as the processing of the spherical embedding. \n\nW7: The paper claims that the icosahedral discretization \"do not take into account the rotational symmetry of the sphere, but rather discrete permutations on the resulting graph\". If the icosahedral approaches would have used the same positional encoding as the authors here they would also account for rotation symmetry.\n\nW8: While the authors compare with ShapeUformer in the segmentation task, they do not do in the depth prediction although the ShapeUformer paper contains results (with different metrics).\n\nW9: ShapeUFormer contains depth estimation comparisons with several other approaches like Panoformer, EGformer, SFSS, HexRUnet, HealSWIN, Elite360D.\n\nIn summary, the paper lacks in novelty and technical soundness. It needs a significant enhancement to be elevated from a subcase of other papers to a paper competing in scope and properties with the state of the art.", "was_revised": true, "final_justification": "The authors addressed all my concerns. They recognized that they cannot handle higher than scalar types in inputs and outputs in their layers. They convinced me that the community needs such an approach. Their rebuttal and final remarks are exemplary in their honesty and style. I recommend accept.", "rebuttal": "First of all, we would like to thank the reviewer for taking their time and providing us with this review. In the following we address the reviewers concerns:\n\n**W1 - Lack of novelty, only a graph transformer, no equivariance for vector-valued**\n\nThe reviewer argues that this is only a graph transformer where edges are restricted to nodes within a certain geodesic distance from each other. This is not the case, as a graph transformer does not impose a quadrature rule on the summation imposed from a continuous formulation. A consequence of this is that a graph transformer will have permutation equivariance within the nodes inside this cutoff radius. This is not the case here - we break this permutational equivariance and get SO(3) equivariance - a notable difference and a strong inductive bias. We have emphasized this point in the manuscript to make it clearer.\n\nMoreover, the reviewer cites a number of works that construct SE(3) equivariant transformers based on the equivariant SE(3) Graph attention. While the equivariant Graph attention does indeed ensure SE(3) and therefore SO(3) equivariance, a fundamental difference is that the Equiformer and related approaches mainly consider features/point clouds as inputs whereas we consider entire functions that are densely sampled on the sphere. Moreover the argument that this is a special case of aforementioned architectures does not hold. For instance, their depth-wise product (DTP) ensures equivariance by introducing a weight parameterized by the pairwise distance of features. In contrast , we make use of the proper discretization of the invariant Haar measure to obtain an equivariant formulation. This results in a pre-determined integration weight dependent on the absolute location. As such this formulation cannot be obtained as a special casse from the equiformer.\n\nFinally, we would like to emphasize that there is significant value in realizing special cases of architectures and motivating them from a different theoretical perspective. Moreover, we provide efficient implementations and experimental evaluations that this approach is valid. As such this work presents a valuable addition to the literature, regardless of whether it can be derived from another architecture.\n\n**W2 - misrepresented related works, S2 attention as subcase of SE(3)**\n\nWe thank the reviewer for bringing these works to our attention. We believe these works are relevant and should have been added to the manuscript. We have added [1,2,3,4] to the related work section of our manuscript.\n\n**W3 - Ignoring positional embedding, scalar-valued equivariant graph transformer**\n\nIn contrast to equivariant graph transformers, we consider functions on the sphere and not features. A natural way of processing the positional embeddings in an equivariant manner is to treat them as inputs, therefore rotating them alongside the input if a rotation is applied. We have clarified this in the text. Regarding vector-valued functions see W5.\n\n**W4 - Clarify notion of continuous**\n\nThe notion of continuous refers to all blocks being defined in the continuous domain, such that discretizations of the architecture can be derived at any resolution/grid that do not alter the parameterization. As such, the interpolation is not problematic as it is well-defined for any resolution, given that the discretizations are reasonable. There exist other Neural Operator architectures such as the Convolutional Neural Operator [5] which make heavy use of interpolation. Moreover, the decoder does not require this operation - it is added to avoid aliasing as is common in many modern convolutional architectures. We have clarified this in the manuscript. \n\n**W5 - Equivariance of vector-valued inputs**\n\nWe do consider signals defined on the surface of the sphere, rather than features which denote positions or directions in euclidean space. As such rotations are applied as passive rotations to the input signals rather than to the outputs of the functions as well (as would be required). Upon popular request, we have added an equivariance test which demonstrates the approximate equivariance (see Table 3). This test considers vector-valued signals on the sphere which are rotated before the attention is applied and then rotated back. The L2 errors remain within a factor 2 of the interpolation incurred by rotating the signals and interpolating them onto the new grid and then reversing this operation. We have added this equivariance test alongside plots to visually demonstrate the equivariance of the attention mechanism (including vector-valued signals).\n\n*Table 3*: Upon suggestion from multiple reviewers , we have added an equivariance test. This test takes a Gaussian random linear combination of spherical Harmonics to create a vector-valued noisy signal on the sphere, which is cut off at l=12, m=12.Equivariance errors are reported alongside interpolation errors that arise from applying the rotations discretely. All errors are reported in terms of the L2 loss on the sphere.\n\n|  | S2 Transformer | | Neighborhood S2 Transformer | |\n|---|----|----|----|-----|\n| Grid Size | Interpolation |   Equivariance   | Interpolation   | Equivariance   |   |\n| 8x16 | 53.913 | 74.934 | 29.588 | 61.374 |\n| 16x32 | 28.630 | 47.374 | 23.750 | 40.656 |\n| 32x64 | 18.268 | 32.687 | 14.621 | 29.052 |\n| 64x128 | 11.470 | 18.350 | 8.852 | 15.690 |\n| 128x256 | 6.354 | 9.617 | 4.763 | 8.678 |\n| 256x512 |  - |  - | 2.397 | 4.123 |\n\n**W6 - Positional embeddings could fix the icosahedral approaches**\n\nThe icosahedral approaches that are referenced in the paper do use graph-based attention and rely on its properties to achieve equivariance[6]. However, this is limited to discrete rotations which map nodes of the icosahedron back to nodes of the icosahedron, irrespective of the positional embeddings. This is stated in their work [6]. In contrast, our method only achieves approximate equivariance (see Table 3), however for arbitrary rotations in SO(3).\n\n**W7 - SphereUFormer not applied to depth**\n\nWe took the results for SphereUFormer directly from their paper and used only metrics which are comparable. However, we have added two additional baselines to the depth example.\n\n**W9 - Lack of baselines compared to SphereUFormer**\n\nUpon request from the reviewers we have added SphericalCNN [2] alongside EGFormer [3] to the results section (See Table 2 and Table 1 in he rebuttal for reviewer nUh3). We would like to emphasize that the goal of the paper is not to introduce a novel architecture but rather demonstrate how the attention mechanism is correctly adapted to spherical data. This is why in our experiments we use the same transformer architectures formulated on $R^2$ and demonstrate improvements that can be achieved by formulating the attention mechanism correctly. We expect that this “building block” may find its way into other architectures dealing with spherical data.\n\n*Table 2 - Depth Estimation*\nWe have updated the depth estimation data with new baselines (EGFormer & SphericalCNN). Please find the updated values below:\n\n| Model | Training loss ↓ | Validation loss ↓ | L₁ error ↓  | W¹,¹ error ↓ |\n|---|----|--|---|--|\n| ℝ² Transformer | 0.954 | 0.982 ± 0.386 | 0.215 ± 0.084 | 7.67 ± 3.22 |\n| S² Transformer | 0.978 | 0.993 ± 0.381 | 0.234 ± 0.086 | **7.59 ± 3.23** |\n| ℝ² Transformer (local) | 1.014 | 1.023 ± 0.370  | 0.237 ± 0.079 | 7.86 ± 3.17 |\n| S² Transformer (local) | **0.978** | 0.976 ± 0.354   | 0.209 ± 0.068       | 7.68 ± 3.12 |\n| ℝ² SegFormer | 1.037 | 0.980 ± 0.371 | 0.169 ± 0.058       | 8.11 ± 3.22 |\n| S² SegFormer | 1.005 | **0.935 ± 0.369**  | **0.165 ± 0.057** | 7.70 ± 3.23 |\n| ℝ² SegFormer (local)        | 1.055   | 1.002 ± 0.374          | 0.174 ± 0.061       | 8.27 ± 3.25          |\n| S² SegFormer (local)        | 1.014           | 0.948 ± 0.368          | 0.174 ± 0.061       | 7.75 ± 3.21          |\n| **Additional baselines**    |  | | |  |\n| Spherical CNN   | 1.019| 1.098 ± 0.412          | 0.315 ± 0.123       | 7.83 ± 3.21          |\n| LSNO  | 0.994  | 1.020 ± 0.387 | 0.248 ± 0.095       | 7.72 ± 3.19          |\n| SUNet  | 1.006 | 0.985 ± 0.352 | 0.220 ± 0.061       | 7.65 ± 3.11          |\n| EGFormer | 0.831 | 0.945 ± 0.386 | 0.182 ± 0.084       | 7.63 ± 3.26          |\n\n\n**References**\n\n[1] Fuchs, F., Worrall, D., Fischer, V., & Welling, M. (2020). SE(3)-transformers: 3d roto-translation equivariant attention networks. Advances in neural information processing systems, 33, 1970-1981.\n\n[2] Liao, Yi-Lun, and Tess Smidt. \"Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs.\" The Eleventh International Conference on Learning Representations.\n\n[3] Xu, Y., Chen, D., Liu, K., Zakharov, S., Ambrus, R. A., Daniilidis, K., & Guizilini, V. C. (2024). SE(3) Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation. The Thirty-Eighth Annual Conference on Neural Information Processing Systems. \n\n[4] Chatzipantazis, E., Pertigkiozoglou, S., Dobriban, E., & Daniilidis, K. (2023). SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space. Int. Conference Learning Representations ICLR. \n\n[5] Bogdan Raonić, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, Emmanuel de Bézenac. Convolutional Neural Operators for robust and accurate learning of PDEs. Advances in Neural Information Processing Systems, 2023\n\n[6] Sungmin Cho, Raehyuk Jung, Junseok Kwon. Spherical Transformer . arXiv preprint arXiv:2202.04942, 2022•arxiv.org\n\n[7] Taco S. Cohen, Mario Geiger, Jonas Koehler, Max Welling. Spherical CNNs. arXiv preprint arXiv:1801.10130, 2018\n\n[8] Zelin Zhang, Tao Zhang, KediLI, Xu Zheng."}, {"review_id": "0sO7ypDJ7b", "reviewer": "Reviewer_nUh3", "summary": "The authors introduce a generalized attention mechanism for spherical domains, enabling Transformers to effectively process data on the 2D sphere. By incorporating numerical quadrature weights into the attention mechanism, an approximately rotationally equivariant and geometrically faithful spherical attention is obtained, offering stronger inductive biases and better performance than Cartesian methods. To improve scalability and locality, the authors additionally propose neighborhood attention that limits interactions to geodesic neighborhoods, reducing computational cost while maintaining symmetry. They provide optimized CUDA implementations and demonstrate their method’s effectiveness across three tasks: shallow water simulation, spherical image segmentation, and depth estimation. The spherical Transformers consistently outperform planar baselines, emphasizing the benefits of geometric priors for learning on spherical domains.", "questions": "1. Are the $\\mathbb{R}^{2}$ and $S^{2}$ versions of Transformer, Transformer (local), SegFormer, SegFormer (local), and SphereUFormer trained using the same input data? If yes, is this input appropriate and optimal for validating the advantages of the proposed method?\n2. Can the authors clarify the input representation used for the Stanford 2D3DS dataset? Specifically, how is the data preprocessed or adapted to fit the spherical model’s input requirements?\n3. Given the abstract and introduction, there is an expectation to see validation on diverse domains such as atmospheric physics, cosmology, and robotics. Can the authors provide additional experimental results or validation on datasets from these or other relevant domains?\n4. How does the proposed method compare quantitatively and qualitatively to existing spherical CNNs and geometric deep learning models?\n5. How generalizable is the model to data with varying resolutions or noise characteristics?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The motivation is clear, and the problem is important.\n2. The paper is well written and clearly organized.\n3. Validation is done on three different tasks.\n\nWeaknesses:\n\n1. The literature review in the related work section is insufficient. It is mentioned in the introduction that spherical convolutional neural networks and geometric deep learning models have been used for learning on the sphere, the paper does not provide a detailed review or comparison of these prior works. Additionally, the relationship between the proposed method and neighborhood Transformers is not clear.\n2. The suitability of the Stanford 2D3DS dataset for the proposed approach is questionable. It is not clear what input representation was used during training with this dataset, raising concerns about whether the data aligns well with the model’s intended application.", "was_revised": true, "final_justification": "While I appreciate the authors’ effort to generalize local and global attention mechanisms to the spherical domain, I remain unconvinced by their justification for not providing stronger empirical validation.\n\nThe authors state that their goal is not to present a new state-of-the-art architecture for spherical data, but rather to offer a generalization of attention mechanisms to the spherical setting. However, I believe that a method aiming to generalize such foundational building blocks should be accompanied by more substantial experimental evidence to demonstrate its effectiveness and practical relevance on more tasks like classification.\n\nMoreover, the authors’ assertion that it is “impossible to say” whether their method performs better than others in certain settings seems evasive. While it may not be possible to make universal claims, it is certainly reasonable to expect a focused empirical analysis that explores strengths and limitations across representative tasks. Without this, the contribution risks being overly theoretical or speculative.\n\nI encourage the authors to strengthen their work by including:\n1. Comparative evaluations against relevant baselines (e.g., Spherical CNNs, existing transformer variants for spherical data) on more tasks.\n2. A more grounded discussion of the method’s limitations and applicable scenarios.\n\nThis would significantly improve the clarity and impact of the contribution.", "rebuttal": "**W1 - Review of Prior work**\nWe agree with the reviewer that our review of prior work was incomplete and could have been more extensive. Since the review, we have added additional references to equivariant graph transformers as suggested by reviewer WwPf ([1,2,3,4]) along side additional baselines (SphericalCNN[5] and EGFormer[6]) to the related work sections and results sections.\n\nOur focus in the manuscript is on extending the popular attention mechanism used by transformer & segformer architectures to the spherical domain. Given this goal, we did not set out to beat the state of the art, but instead ensure that our technique is applicable across several image and simulation applications, with benefits over naive euclidean approaches. Despite this, we agree that baselines are helpful in estimating the performance of the approach and have added SphericalCNN[5] and EGFormer[6] to the segmentation and depth estimation problems.\n\n**W2 - Suitability of the 2D3DS dataset**\nThe 2D3DS dataset provides problems which require mapping spherical signals to other spherical signals, in contrast to other datasets which often deal with estimating pose or other features. This makes depth estimation and segmentation a natural choice given that the architecture is aimed at operator learning (i.e. mapping spherical signals to other spherical signals). Moreover, the 2D3DS dataset has been a popular choice for manuy related works such as [7]. \n\nFor both segmentation and depth estimation, the input is an RGB image that was downsampled to 128x256 and 256x512 to reduce the computational cost and enable both non-local and non-spherical attention mechanisms as baselines, which otherwise become intractable. We agree that this could have been exposed better in the text and have edited the manuscript accordingly.\n\n**Q1 - Are all models trained with the same input/outputs. Is this appropriate to validate the method?**\nYes - all of the architectures are trained in the same manner to ensure comparability. Moreover the same loss formulation is used for all architectures, and architectures are kept as close to eachother in terms which makes this a the good test to evaluate the advantage of the spherical attention formulation.\n\nOne weakness of the dataset is that the segmentation and depth content closer to the poles was less variable. For the shallow water equations, a large source of visually noticeable artifacts arise at the poles, where our equivariant methods have no issues. Despite this we see advantages with our methods for all three datasets, which validates the approach.\n\nFinally, the dataset has been used by many previous works giving a good comparison, at both a visual and quantitative (e.g., IoU) level for the applicability of our method. Metrics have been added to verify the validity of the method beyond just the validation loss.\n\n**Q2 - 2D3DS data preprocessing**\nYes - The dataset is downsampled from it’s original 4K representation to a more suitable resolution of 128x256 on an equiangular grid on the sphere. The target in the depth estimation case is a logarithm of the depth whereas in the segmentation case it is a class label enumerated by an integer number. The input data is z-score normalized to ensure proper training of the architectures. We agree that this should have been clarified better and have edited the manuscript accordingly\n\n**Q3 - Other application areas**\nThe depth estimation and segmentation datasets were chosen due to their relevance in robotics, whereas the Shallow Water Equation example was chosen due to the relevance of the dataset in geophysics and especially atmospheric physics. Regarding cosmology, we are not aware of a suitable image-based dataset, however there is a big body of work that deals with signal processing of spherical data in cosmology. Given the short period of time of the rebuttal phase we chose to focus on adding additional baselines and an equivariance test (see other reviews). We adapted the language in the manuscript to only mention cosmology in the context of possible application domains.\n\n**Q4 - Comparison against spherical CNNs**\nWe have implemented the SphericalCNN [5] architecture and the EGFormer [6] and added them as additional baselines to the tables provided below (Table 1 and 2). In our initial, time-limited, attempt to implement the baselines, the segmentation IoU performance is worse, which is visually notable in the segmentation results. Our technique relies on a suitable quadrature rule to enable numerical integration on the given input grid, and to use an icosohedral representation would require numerical integration rules for the different grid as well as updates to our neighborhood implementation to enable gathering of the icosohedron within the geodesic neighborhood. That said, one could imagine plugging into an adapted neighborhood attention mechanism into the SphericalCNN architecture after the input grid is projected to the icoshedral grid.\n\n*Table1*: Updated table for the segmentation problem\n| Model | Training loss ↓ | Validation loss ↓ | IoU ↑ | Acc ↑ |\n|--|--|--|--|--|\n| **Results on 128×256 data** | | | | |\n| S² Transformer   | 0.330 | 1.02 ± 0.572  | 0.578 ± 0.126 | 0.961 ± 0.015 |\n| S² SegFormer  | 0.306 | **0.705 ± 0.307** | 0.662 ± 0.102 | 0.970 ± 0.011 |\n| ℝ² SegFormer (local)  | 0.289  | 0.817 ± 0.339 | 0.624 ± 0.103 | 0.966 ± 0.011 |\n| **S² SegFormer (local)**  | 0.321  | 0.706 ± 0.303  | **0.658 ± 0.100**| **0.970 ± 0.011**|\n| EGFormer   | **0.203**       | 1.29 ± 0.487   | 0.586 ± 0.106  | 0.962 ± 0.012 |\n| **Results on 256×512 data**  | | | | |\n| Spherical CNN | 0.605 | 0.809 ± 0.287 | 0.590 ± 0.099  | 0.963 ± 0.012    |\n| ℝ² SegFormer (local, large) | 0.626 | 0.812 ± 0.891 | 0.692 ± 0.158 | 0.972 ± 0.018 |\n| **S² SegFormer (local, large)** | **0.515** | **0.656 ± 0.719**    | **0.728 ± 0.136**| **0.976 ± 0.015**|\n\n*Table 2 - Depth Estimation*\nWe have updated the depth estimation data with new baselines (EGFormer & SphericalCNN). Please find the updated values below:\n\n| Model | Training loss ↓ | Validation loss ↓ | L₁ error ↓  | W¹,¹ error ↓ |\n|---|----|--|---|--|\n| ℝ² Transformer | 0.954 | 0.982 ± 0.386 | 0.215 ± 0.084 | 7.67 ± 3.22 |\n| S² Transformer | 0.978 | 0.993 ± 0.381 | 0.234 ± 0.086 | **7.59 ± 3.23** |\n| ℝ² Transformer (local) | 1.014 | 1.023 ± 0.370  | 0.237 ± 0.079 | 7.86 ± 3.17 |\n| S² Transformer (local) | **0.978** | 0.976 ± 0.354   | 0.209 ± 0.068       | 7.68 ± 3.12 |\n| ℝ² SegFormer | 1.037 | 0.980 ± 0.371 | 0.169 ± 0.058       | 8.11 ± 3.22 |\n| S² SegFormer | 1.005 | **0.935 ± 0.369**  | **0.165 ± 0.057** | 7.70 ± 3.23 |\n| ℝ² SegFormer (local)        | 1.055   | 1.002 ± 0.374          | 0.174 ± 0.061       | 8.27 ± 3.25          |\n| S² SegFormer (local)        | 1.014           | 0.948 ± 0.368          | 0.174 ± 0.061       | 7.75 ± 3.21          |\n| **Additional baselines**    |  | | |  |\n| Spherical CNN   | 1.019| 1.098 ± 0.412          | 0.315 ± 0.123       | 7.83 ± 3.21          |\n| LSNO  | 0.994  | 1.020 ± 0.387 | 0.248 ± 0.095       | 7.72 ± 3.19          |\n| SUNet  | 1.006 | 0.985 ± 0.352 | 0.220 ± 0.061       | 7.65 ± 3.11          |\n| EGFormer | 0.831 | 0.945 ± 0.386 | 0.182 ± 0.084       | 7.63 ± 3.26          |\n\n**Q5 - Varying input resolutions and noise**\nAs noted in Table 1 (segmentation results) we tried two resolutions with the local S2 segformer against the local R2 segformer, and both resolutions had similar evaluation scores to their lower resolution counterparts. The continuous formulation of the architecture makes the method applicable to any reasonable resolution and sampling scheme which allows accurate approximation of the integrals in the attention formulation. Moreover, our implementation of sparse attention (geodesic circle) on the sphere is equivalent with standard cartesian blocked neighborhood attention from a memory and computational complexity standpoint and thus scales favorably compared to full attention as resolution grows. Regarding performance with noise: The 2D3DS dataset is composed of RGB images of the real world and shows that our models are robust against noise found in non-synthetic images. That said, explicitly evaluating noise performance, was beyond the scope of our work, although we have no reason to expect it to perform differently than a traditional Transformer or Segformer model on euclidean data.\n\n[1] Fuchs, F., Worrall, D., Fischer, V., & Welling, M. (2020). SE(3)-transformers: 3d roto-translation equivariant attention networks. Advances in neural information processing systems, 33, 1970-1981.\n\n[2] Liao, Yi-Lun, and Tess Smidt. \"Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs.\" The Eleventh International Conference on Learning Representations.\n\n[3] Xu, Y., Chen, D., Liu, K., Zakharov, S., Ambrus, R. A., Daniilidis, K., & Guizilini, V. C. (2024). SE(3) Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation. The Thirty-Eighth Annual Conference on Neural Information Processing Systems. \n\n[4] Chatzipantazis, E., Pertigkiozoglou, S., Dobriban, E., & Daniilidis, K. (2023). SE(3)-Equivariant Attention Networks for Shape Reconstruction in Function Space. Int. Conference Learning Representations ICLR. \n\n[5] Chao Zhang, Stephan Liwicki, Sen He, William Smith, Roberto Cipolla: HexNet: An Orientation-Aware Deep Learning Framework for Omni-Directional Input. IEEE Trans. Pattern Anal. Mach. Intell. 45(12): 14665-14681 (2023) [2] Y. Lee, J. Jeong, J. Yun, W. Cho, and K.-J. Yoon, “Spherephd: Applying cnns on a spherical polyhedron representation of 360deg images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 9181–9189.\n\n[6] Zelin Zhang, Tao Zhang, KediLI, Xu Zheng. EGFormer. arXiv preprint arXiv:2505.14014, 2025\n\n[7] Jeremy Ocampo, Matthew A. Price, Jason D. McEwen. Scalable and Equivariant Spherical CNNs by Discrete-Continuous (DISCO) Convolutions. arXiv preprint arXiv:2209.13603, 2022•arxiv.org"}]

Author rebuttal:

Meta-review: The reviewer discussion around Attention on the Sphere was divided, with scores ranging from reject to clear accept. Reviewers highlighted that the paper makes a principled and technically sound extension of attention and neighborhood attention mechanisms to spherical domains, with rigorous derivations, a continuous formulation discretized via quadrature, and efficient CUDA implementations. Empirical results across three diverse tasks consistently show benefits of spherical over Euclidean attention, particularly in the shallow water equation benchmark where geometric fidelity is crucial. Multiple reviewers praised the clarity of the paper, the motivation, and the provision of open-source resources, and were ultimately convinced by the authors’ thorough rebuttal and clarifications on novelty, positional encodings, and additional baselines (including Spherical CNNs and EGFormer).

The main disagreements arose around scope and completeness. One reviewer (initially skeptical) argued the work lacked novelty relative to SE(3)-equivariant transformers and questioned whether the “continuous” claim was justified, though after discussion they recommended acceptance. Another reviewer remained concerned about the limited breadth of experimental validation and the lack of deeper comparative analysis against established spherical architectures, leaving a borderline score. Overall, however, the rebuttal addressed most key concerns, clarified misunderstandings, and added substantive evidence, leading the majority of reviewers to converge on acceptance.
Decision: Accept (poster)