SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The points  $\pmb{x}$  on the ray  $x = (d_x, m_x)$  can be uniquely expressed as  $\pmb{x} = s_b(x) + t_x d_x = d_x \times m_x + t_x d_x$ , therefore for any  $x \in \mathcal{R}$ , any  $t \in \mathbb{R}$ ,  $f(x)_t$  can be expressed as a feature attached to the point  $s_b(x) + t d_x$  along the ray  $x$ , i.e.,  $f(x)_t = f'(s_b(x) + t d_x, d_x)$  as shown in figure 14.
Therefore, we have  $f^{\prime}(\pmb {x},\pmb {d}) = f((\pmb {d},\pmb {x}\times \pmb {d}))_{\langle \pmb {x} - \pmb {d}\times (\pmb {x}\times \pmb {d}),\pmb {d}\rangle}$ , one can easily check:
$$
\left(\mathcal {L} _ {g} f ^ {\prime}\right) (\boldsymbol {x}, \boldsymbol {d}) = \rho_ {1} \left(\mathrm {h} _ {a} \left(R _ {g ^ {- 1}}, \boldsymbol {d}\right)\right) ^ {- 1} f ^ {\prime} \left(R _ {g ^ {- 1}} \boldsymbol {x} + \boldsymbol {t} _ {g ^ {- 1}}, R _ {g ^ {- 1}} \boldsymbol {d}\right) = \rho_ {1} \left(R _ {Z} \left(R _ {g ^ {- 1}}, \boldsymbol {d}\right)\right) ^ {- 1} f ^ {\prime} \left(g ^ {- 1} \boldsymbol {x}, R _ {g ^ {- 1}} \boldsymbol {d}\right) \tag {15}
$$
We should note the difference of the point  $\pmb{x}$  along the ray and the independent point  $\pmb{x}$ , as shown in the above equation, the point  $\pmb{x}$  along the ray  $x = (d, x \times d)$  is denoted as  $(\pmb{x}, d)$  instead of  $\pmb{x}$ . Actually, it can be viewed as a homogeneous space of  $SE(3)$  larger than  $\mathbb{R}^3$ , whose elements are in  $\mathbb{R}^3 \times \mathbb{S}^2$ , as shown in figure 15.
To summarize, the features attached to the ray, whose type corresponds to the regular representation of translation, can be considered as the features attached to the points along the ray. The action of  $SE(3)$  on features attached to these points can be expressed as shown in Eq. 15.
The solution  $\kappa$  also can be expressed as
$$
\kappa (x) _ {t} = \kappa_ {1} (x) \kappa_ {2} (x) _ {t} \tag {16}
$$
Figure 14: The feature attached to the ray, which corresponds to the regular representation of translation, can also be treated as the features attached to the points along the ray.
for any  $t \in \mathbb{R}$ , and their constraint is also the same as Eq. 9 and Eq. 10. As a result, the solution for  $\kappa_{1}$  should be the same. We only need to solve  $\kappa_{2}$ :
$$
\kappa_ {2} \left(\left(\gamma , t ^ {\prime}\right) x\right) _ {t} = e ^ {i \omega_ {i n} ^ {2} \langle [ 0, 0, 1 ] ^ {T}, d _ {x} \rangle t ^ {\prime}} \kappa_ {2} (x) _ {t - t ^ {\prime}} \tag {17}
$$
for any  $(\gamma ,t^{\prime})\in SO(2)\times \mathbb{R}$
When  $\pmb{d}_x\in \mathbb{S}^2 -\left\{[0,0,1]^T, - [0,0,1]^T\right\}$
$$
\kappa_ {2} (x) _ {t} = f \left(d (\eta , x), \angle \left([ 0, 0, 1 ] ^ {T}, \boldsymbol {d} _ {x}\right)\right) e ^ {i \omega_ {i n} ^ {2} \langle [ 0, 0, 1 ] ^ {T}, \boldsymbol {d} _ {x} \rangle g (x)} \delta (t - g (x)), \tag {18}
$$
where  $f$  and  $g$  are the same function as defined in 14, and  $\delta(t) = 1$  only when  $t = 0$ .
when  $\pmb{d}_x\in \left\{\left[0,0,1\right]^T, - \left[0,0,1\right]^T\right\} ,\kappa_2(x)_t = 0$  for any  $t\in \mathbb{R}$
Example 10.  $SE(3)$  equivariant convolution from  $\mathcal{R}$  to  $\mathbb{R}^3$ : Following [18], the convolution from rays to points becomes:
$$
f _ {2} ^ {l _ {\text {o u t}}} (x) = \int_ {\mathcal {R}} \kappa \left(s _ {2} (x) ^ {- 1} y\right) \rho_ {i n} \left(\mathrm {h} _ {1} \left(s _ {2} (x) ^ {- 1} s _ {1} (y)\right)\right) f _ {1} ^ {l _ {\text {i n}}} (y) d y, \tag {19}
$$
where  $\mathrm{h}_1$  is the twist function corresponding to section  $s_1: \mathcal{R} \to SE(3)$  defined aforementioned,  $\rho_{in}$  is the group representation of  $SO(2) \times \mathbb{R}$ , corresponding to the feature type  $l_{in}$ ,  $s_2: \mathbb{R}^3 \to SE(3)$  is the section map defined in paper as  $s_2(\pmb{x}) = (I, \pmb{x})$ .
In this paper, we give the analysis and solutions for the kernel where the input is the scalar field over the ray space, i.e.,  $\rho_{in} = 1$ , the trivial group representation, which is also the case of our application in reconstruction.
The convolution is equivariant if and only if
$$
\kappa (h _ {2} x) = \rho_ {\text {o u t}} (h _ {2}) \kappa (x),
$$
vector features(type-1) vector features(type-1)
Figure 15: As shown in the figure, the point along the ray is distinct from the independent point. Moreover, we can observe that the type-1 feature of the point along the ray differs from that of the independent point. Specifically, the type-1 feature for the point along the ray can be interpreted as a vector on the plane orthogonal to the ray direction. In contrast, the type-1 feature for the independent point can be interpreted as a three-dimensional vector.
for any  $h_2 \in SO(3)$ , where  $\rho_{out}$  is the group representation of  $SO(3)$  corresponding to the feature type  $l_{out}$ .
We can derive  $\kappa(h_2x) = \rho_{out}(h_2)\kappa(x)$  analytically. For irreducible representation  $\rho_{out}$  and any  $x = (d_x, m_x) \in \mathcal{R}$ , if  $\|m_x\| = 0$ ,  $\kappa(x) = cY^{l_{out}}(d_x)$ , where  $c$  is an arbitrary constant and  $Y^{l_{out}}$  is the spherical harmonics and  $l_{out}$  is the order (type) of output tensor corresponding to the representation  $\rho_{out}$ ; With  $\|m_x\| \neq 0$ ,  $\kappa(x)$  becomes  $\rho_{out}(\hat{x})f(\|m\|_x)$ , where  $\hat{x}$  denotes the element  $(d_x, \frac{m_x}{\|m_x\|}, d_x \times \frac{m_x}{\|m_x\|})$  in  $SO(3)$  and  $f: \mathbb{R} \to \mathbb{R}^{(2l_{out} + 1) \times 1}$ .
Similar to the convolution from rays to rays, we also can have the local support of the kernel. We set  $\kappa(x) \neq 0$  when  $\|m_x\| \leq d_0$ , otherwise  $\kappa(x) = 0$ . One can easily check that it doesn't break the equivariant constraint for the kernel.
Specifically, when we set  $d_0 = 0$ , the neighborhood of the target points in the convolution only includes the rays from all views going through the point. Hence, we can simplify the convolution to  $f_2^{l_{out}}(x) = \int_{d(y,x) = 0} Y^{l_{out}}(\pmb{d}_{s_2(x)^{-1}y}) f_1^{in}(y) dy$ . This equation shows that for every point  $x$ , we can treat the ray  $y$  going through  $x$  with feature  $f_1^{in}$  as a point  $y'$ , where  $y' - x = \pmb{d}_{s_2(x)^{-1}y}$ , as shown in figure 17.
B Equivariant 3D Reconstruction
B.1 Approximation of the Equivariant Convolution from Rays to Rays
In practical  $3D$  reconstruction, we have multiple views instead of the whole light field. Although the convolution above is defined on the continuous ray space, the equivariance still strictly holds when the ray sampling (pixels from camera views) is the same up to coordinate change. In this case, we will show how we adjust the equivariant convolution from rays to rays and approximate it by an intra-view  $SE(2)$ -convolution.
Figure 16: Visualization of  $g(x)$ . The left is the case that the ray  $x$  and the ray  $\eta$  are intersected, and the right is the case that the ray  $x$  and the ray  $\eta$  are not intersected. For the left, the point  $Q$  is the intersection of  $x$  and  $\eta$ , and  $Q = (0,0,g(x))$ ; for the right, the point  $Q$  is the intersection of the line  $y$  and the ray  $\eta$ , where  $y$  is perpendicular to both  $\eta$  and  $x$ , and intersects with both  $\eta$  and  $x$ . From the figure, in both cases, we can see that for any  $t\in \mathbb{R}$ ,  $g((0,t)x) = t + g(x)$ . In general, we actually have for any  $(\gamma ,t)\in SO(2)\times \mathbb{R}$ ,  $g((\gamma ,t)x) = t + g(x)$ .
Figure 17: Interpreting rays  $y_{i}$  as points  $y_{i}'$
B.1.1 From Light Field to Intra-view Convolution
Following Fig. 18, neighboring rays are composed of two parts: a set of rays from the same view and another set of rays from different views. For one ray  $x$  in view  $A$ , the neighboring rays from view  $B$  are in the neighborhood of the epipolar line of  $x$  in view  $B$ . When the two views are close, the neighborhood in the view  $B$  would be very large.
The kernel solution in Ex. 9 suggests that  $\kappa(x)$  is related to  $\angle(\pmb{d}_x, [0,0,1]^T)$  and  $d((x,\eta)$ , where  $\eta = ([0,0,1]^T, [0,0,0]^T)$  as mentioned before. It would be memory- and time-consuming to memorize the two metrics beforehand or to compute the angles and distances on the fly. Practically, the light field is only sampled from a few sparse viewpoints, which causes the relative angles of the rays in different views to be large and allows them to be excluded from the kernel neighborhood; therefore, in our implementation, the ray neighborhood is composed of only rays in the same view.
Figure 18: For simplification, we show a situation of two views. For a ray  $x$  from view A, one part of the neighboring rays is from view A (the blue rays in the figure),  $\mathcal{N}_A(x)$ . For any ray  $y \in \mathcal{N}_A(x)$ , we have  $d(y, x) = 0$ , and we require  $\angle(\pmb{d}_y, \pmb{d}_x) \leq \beta_0$ . The other part is from the other view B (the red rays in the figure). As illustrated in figure 5, the neighboring rays always cross a cylinder around  $x$ ; therefore, the neighboring rays from view B are the projection of the cylinder with radius  $r = d_0$  in view B, that is,  $\mathcal{N}_B$  is composed of the neighboring pixels of the epipolar line (the black dotted dash) corresponding to  $x$  in view B. For any ray  $y$  in the projection of the cylinder, we have  $d(y, x) \neq d_0$ . Since we require that  $\angle(\pmb{d}_y, \pmb{d}_x) \leq \beta_0$  for any ray  $y \in \mathcal{N}_B(x)$ ,  $\mathcal{N}_B(x)$  is part of the projection of the cylinder, denoted as the shaded yellow part in view B.
B.1.2 From Intra-view Light Field to Spherical Convolution
After showing that a small kernel support in the case of sparse views affects only intra-view rays, we can prove that an intra-view light-field convolution is equivalent to a spherical convolution when we constrain the feature field types over  $\mathcal{R}$ .
We exploit the desired property that a feature defined on a ray is constant along the ray. This means that the translation part of the stabilizer group (translation along the ray) leaves the feature as is. In math terms, the irreducible representation for the translation  $\mathbb{R}$  is the identity, which means that the field function is a scalar field for the translation group, with the formula  $(\mathcal{L}_t f)(x) = f(t^{-1} x)$ . We prove that, in this case, the intra-view convolution over rays is equivalent to the spherical convolution; please see Sec. C.
B.1.3 From SO(3)- to SE(2)-convolution
While there is an established framework for spherical convolution using a Fourier transform [17, 22, 24] it is not applicable in our case because the boundaries of the constrained field of view cause an explosion in the high frequencies of the spherical harmonics. We will make a compromise here and approximate the SO(3) convolution with an SE(2) convolution on the image plane by making the assumption that the field of view is small. One can see the rationale behind this approximation by keeping only the first order terms in the optical flow equation: the rotational term is only due to  $\Omega_z$  while the translational term is  $(-T_x - \Omega_y, - T_y + \Omega_x)$  with  $(\Omega_{x},\Omega_{y},\Omega_{z})$  as the angular velocity. We provide a justification using the formalism of the previous paragraphs in appendix Sec. E.
B.2 Ray Fusion: Equivariant Convolution and Transformer
To reconstruct a 3D object, we use an implicit function known as the signed distance function (SDF) defined on  $\mathbb{R}^3$ . As a result, we require an equivariant model that can transform features from rays to
points to obtain the SDF. This can be achieved using the equivariant convolution in Sec. 3.4.2 and transformer in Sec. 3.2 in the paper 3.5, which allows us to transform features from the ray space to points in 3D space while maintaining equivariance.
B.2.1 Equivariant Convolution from Rays to Points
In this paper, we obtain the scalar feature field over rays after the SE(2)-equivariant CNNs. As illustrated in figure 3, we utilize the equivariant convolution (discussed in Sec. 3.4.2) to compute features for a query point by convolving over neighboring rays. Our experiments have shown that convolving only over rays that go through the point achieves the best results, and the equivariant kernel used for this convolution is provided in Ex.10. Moreover, in the implementation, we can concatenate the input feature  $f_1^{in}$  with the depth embedding of the query point  $x$ . While this theoretically breaks the ideal equivariance for continuous light fields, it does not affect the practical equivariance, as it is rare for two cameras to share the same ray.
B.2.2 Equivariant Transformer from Rays to Points
For the third step, we introduce an equivariant transformer to alleviate the loss of expressivity due to the constrained kernel  $\kappa$  in Eq. 19. Again, the attention key and values are generated from the feature attached to rays, while the query is generated from the feature attached to points.
In the implementation, we apply a transformer over the rays going through the query point. We can continue to use the interpretation that treats any ray  $y$  passing through the point  $x$  as a point  $y'$  such that  $y' - x = d_{s_2(x)^{-1}y}$ , as shown in figure 17. Since  $y$  becomes point  $y'$ , the ray feature  $f_1^{in}$  becomes the feature over  $\mathbb{R}^3$  attached to "points"  $y'$ . We can update the neighboring ray feature by directly concatenating the equivariant feature of the point to every ray feature before through a  $SO(3)$  equivariant MLP. The transformer in Eq. 4 would be converted to the transformer in [27] over  $\mathbb{R}^3$ . See appendix Sec. H for details. The composition of the ray updating block and transformer block are shown in figure 22.
C Proof of Equivalence of Intra-view Light Field Convolution and Spherical Convolution
The property that a feature defined on a ray is constant along the ray means that the translation part of the stabilizer group (translation along the ray) leaves the feature as is. In math terms, the irreducible representation for the translation  $\mathbb{R}$  is the identity, which means that the field function is a scalar field for the translation group, with the formula  $(\mathcal{L}_t f)(x) = f(t^{-1} x)$ . The equivariant condition on the kernel can then be simplified as
$$
\kappa ((h, t) x) = \rho_ {o u t} (h) \kappa (x) \rho_ {i n} \left(\mathrm {h} _ {a} ^ {- 1} (h, \boldsymbol {d} _ {x})\right),
$$
where  $h \in SO(2)$  and  $t \in \mathbb{R}$ ,  $\rho_{in}$  and  $\rho_{out}$  are irreducible representations for  $SO(2)$ , and  $\mathrm{h}_a$  is the twist function as shown in Ex. 6 that  $\mathrm{h}(g, x) = (\mathrm{h}_a(R_g, \pmb{d}_x), \mathrm{h}_b(g, x))$ , i.e., the twist of the fiber introduced by action of  $SO(3)$  corresponding to the section map  $s_a$  of  $SO(3)$  in Ex. 5 and Ex. 6. Now we describe the relationship between the intra-view light-field convolution and the spherical convolution:
Proposition C.1. When the translation group acts on feature  $f: \mathcal{R} \to V$  as  $(\mathcal{L}_t f)(x) = f(t^{-1} x)$  for any  $x \in \mathcal{R}$ , the equivariant intra-view light-field convolution:
$$
f ^ {l _ {o u t}} (x) = \int_ {y \in \mathcal {N} (x)} \kappa (s (x) ^ {- 1} y) \rho_ {i n} (h (s (x) ^ {- 1} s (y))) f ^ {l _ {i n}} (y) d y
$$
becomes a spherical convolution:
$$
f ^ {l _ {o u t}} (x) = \int_ {\boldsymbol {d} _ {y} \in \mathbb {S} ^ {2}} \kappa^ {\prime} (s _ {a} (\boldsymbol {d} _ {x}) ^ {- 1} \boldsymbol {d} _ {y}) \rho_ {i n} (h _ {a} (s _ {a} (\boldsymbol {d} _ {x}) ^ {- 1} s _ {a} (\boldsymbol {d} _ {y})))
$$
$$
f ^ {\prime l _ {i n}} \left(\boldsymbol {d} _ {y}\right) d \boldsymbol {d} _ {y}, \tag {20}
$$
where  $f^{l_{in}}(\pmb{d}_y) = f^{l_{in}}(\pmb{d}_y, \pmb{c}_x \times \pmb{d}_y)$ ,  $\pmb{c}_x$  denotes the camera center that  $x$  goes through,  $s_a$  is the section map of  $SO(3)$  as defined in appendix Ex. 5, and  $\kappa'(s_a(\pmb{d}_x)^{-1}\pmb{d}_y) = \kappa(s_a(\pmb{d}_x)^{-1}\pmb{d}_y, (s(x)^{-1}\pmb{x}_c) \times (s_a(\pmb{d}_x)^{-1}\pmb{d}_y))$ .
Proof. The  $SE(3)$  equivariant convolution over rays transforms into intra-view convolution when the neighboring lights are in the same view. Moreover, the simplified kernel constraint derived in the paper is that for any  $(h,t)\in SO(2)\times \mathbb{R}$  and  $x = (d_x,m_x)\in \mathcal{R}$ :
$$
\kappa ((h, t) x) = \rho_ {o u t} (h) \kappa (x) \rho_ {i n} \left(\mathrm {h} _ {a} ^ {- 1} (h, d _ {x})\right),
$$
where  $\mathbf{h}_a:SO(3)\times \mathbb{S}^2\to SO(2)$  is the twist function:  $\mathbf{h}_a(g,\pmb {d}) = s_a(g\pmb {d})^{-1}gs_a(\pmb {d})$  for any  $g\in SO(3)$  and  $\pmb {d}\in \mathbb{S}^2$
With the simplified kernel constraint, we can prove that intra-view light field convolution is equivalent to spherical convolution:
$$
\begin{array}{l} f ^ {l _ {o u t}} (x) \\ = \int_ {d (y, \boldsymbol {c} _ {x}) = 0} \kappa (s (x) ^ {- 1} y) \rho_ {i n} (\mathrm {h} (s (x) ^ {- 1} s (y))) f ^ {l _ {i n}} (y) d y (21) \\ = \int_ {d (y, \boldsymbol {c} _ {x}) = 0} \kappa (s (x) ^ {- 1} y) \rho_ {i n} \left(\mathrm {h} _ {a} \left(s _ {a} \left(\boldsymbol {d} _ {x}\right) ^ {- 1} s _ {a} \left(\boldsymbol {d} _ {y}\right)\right)\right) f ^ {l _ {i n}} (y) d y (22) \\ = \int_ {\boldsymbol {d} _ {y} \in \mathbb {S} ^ {2}} \kappa (s _ {a} (\boldsymbol {d} _ {x}) ^ {- 1} \boldsymbol {d} _ {y}, s (x) ^ {- 1} \boldsymbol {x} _ {c} \times (s _ {a} (\boldsymbol {d} _ {x}) ^ {- 1} \boldsymbol {d} _ {y})) \\ \rho_ {i n} \left(\mathrm {h} _ {a} \left(s _ {a} \left(\boldsymbol {d} _ {x}\right) ^ {- 1} s _ {a} \left(\boldsymbol {d} _ {y}\right)\right)\right) f ^ {l _ {i n}} \left(\boldsymbol {d} _ {y}, \boldsymbol {c} _ {x} \times \boldsymbol {d} _ {y}\right) d \boldsymbol {d} _ {y} (23) \\ = \int_ {\boldsymbol {d} _ {y} \in \mathbb {S} ^ {2}} \kappa^ {\prime} (s _ {a} (\boldsymbol {d} (x)) ^ {- 1} \boldsymbol {d} _ {y}) \rho_ {i n} \left(\mathrm {h} _ {a} (s _ {a} (\boldsymbol {d} _ {x}) ^ {- 1} s _ {a} (\boldsymbol {d} _ {y}))\right) \\ f ^ {\prime l _ {i n}} \left(\boldsymbol {d} _ {y}\right) d \boldsymbol {d} _ {y}. (24) \\ \end{array}
$$
In line 21,  $c_x$  is the camera center that  $x$  goes through.
The line 21 is equal to the line 22 because we assume that the irreducible representation for the translation  $\mathbb{R}$  is the identity as mentioned in the paper.
From line 22 to line 23, We can replace  $s(x)^{-1}y$  with
$$
\left(s _ {a} \left(\boldsymbol {d} _ {x}\right) ^ {- 1} \boldsymbol {d} _ {y}, \left(s (x) ^ {- 1} \boldsymbol {x} _ {c}\right) \times \left(s _ {a} \left(\boldsymbol {d} _ {x}\right) ^ {- 1} \boldsymbol {d} _ {y}\right)\right)
$$
due to the facts that  $s_a(\pmb{d}_x)^{-1}\pmb{d}_y = \pmb{d}_{s(x)^{-1}y}$  and point  $s(x)^{-1}\pmb{x}_c$  is on the ray  $s(x)^{-1}y$ . Since  $y$  goes through  $\pmb{c}_x$ , we can replace  $y$  with  $(\pmb{d}_y, \pmb{c}_x \times \pmb{d}_y)$ .
From line 23 to 24, we have  $f^{l_{in}}(\pmb{d}_y) = f^{l_{in}}(\pmb{d}_y, \pmb{c}_x \times \pmb{d}_y)$  because  $\pmb{c}_x$  is fixed for any view. Additionally, from line 23 to 24 we replace
$$
\kappa \left(s _ {a} \left(\boldsymbol {d} _ {x}\right) ^ {- 1} \boldsymbol {d} _ {y}, \left(s (x) ^ {- 1} \boldsymbol {x} _ {c}\right) \times \left(s _ {a} \left(\boldsymbol {d} _ {x}\right) ^ {- 1} \boldsymbol {d} _ {y}\right)\right)
$$
with  $\kappa^{\prime}(s_a(\pmb {d}_x)^{-1}\pmb {d}_y)$  . It is because according to
we have  $\kappa((e, t)x) = \kappa(x)$  for any  $t \in \mathbb{R}$ , where  $e$  is the identity element in  $SO(2)$ ; thus when  $t = ((-s(x)^{-1}x_c))^T[0, 0, 1]^T$ , we have
$$
\begin{array}{l} \kappa \left(s _ {a} (x) ^ {- 1} \boldsymbol {d} _ {y}, s (x) ^ {- 1} \boldsymbol {x} _ {c} \times \left(s _ {a} (x) ^ {- 1} \boldsymbol {d} _ {y}\right)\right) \\ = \kappa \left(s _ {a} (x) ^ {- 1} \boldsymbol {d} _ {y}, \left(s (x) ^ {- 1} \boldsymbol {x} _ {c} + t [ 0, 0, 1 ] ^ {T}\right) \times \left(s _ {a} (x) ^ {- 1} \boldsymbol {d} _ {y}\right)\right) (25) \\ = \kappa \left(\left(s _ {a} (x) ^ {- 1} \boldsymbol {d} _ {y}, [ 0, 0, 0 ] ^ {T}\right) \right. (26) \\ = \kappa^ {\prime} \left(\left(s _ {a} (x) ^ {- 1} \boldsymbol {d} _ {y}\right). \right. \\ \end{array}
$$
Line 25 is equal to 26 because  $s(x)^{-1}\pmb{x}_c$  is always on the  $z$  axis, and thus  $s(x)^{-1}\pmb{x}_c + t[0,0,1]^T = [0,0,0]^T$ .
D Spherical Convolution Expressed in Gauge Equivariant Convolution Format
Group convolution is a particular case of gauge equivariant convolution [64], where gauge equivariant means the equivariance with respect to the transformation of the section map (transformation of the
Figure 19: Illustration of  $h_{x \to y}$ .  $s(x)[1,0,0]^T$  and  $s(x)[0,1,0]^T$  (yellow) attached to  $x$  are tangent vectors on  $x$ . We parallel transport  $s(x)[1,0,0]^T$  and  $s(x)[0,1,0]^T$  along the geodesic (black dashed line) between  $x$  and  $y$ . The transported tangent vectors need to undergo a transformation  $h_{x \to y}$  in  $SO(2)$  to align with the vectors  $s(y)[1,0,0]^T$  and  $s(y)[0,1,0]^T$  (green) attached to  $y$ .
tangent frame). In the following paragraph we give the elaborated definition of gauge equivariance for the sphere.
Suppose  $f: \mathbb{S}^2 \to V$  is the field function corresponding to the section choice  $s_a: \mathbb{S}^2 \to SO(3)$ , we use  $\mathcal{L}_{s_a \to s_a'}$  acting on  $f$  to denote the change of section map from  $s_a$  to  $s_a'$ :  $(\mathcal{L}_{s_a \to s_a'} f)(x) = \rho(s_a(x)^{-1} s_a'(x))^{-1} f(x)$ , where  $\rho$  is the irreducible representation of  $SO(2)$  corresponding to the field type of  $f$ . The convolution  $\Phi$  is gauge equivariant when  $\Phi(\mathcal{L}_{s_a \to s_a'} f) = \mathcal{L}_{s_a \to s_a'} (\Phi(f))$ .
In this section, we show that the spherical convolution can be expressed in terms of the gauge equivariant convolution [16], which provides the convenience for us to verify the approximation of spherical convolution through the  $SE(2)$  convolution:
$$
f ^ {l _ {o u t}} (x) = \int_ {y \in \mathcal {N} (x)} \kappa^ {\prime} (s (x) ^ {- 1} y) \rho_ {i n} (h _ {y \rightarrow x}) ^ {- 1} f ^ {l _ {i n}} (y) d y,
$$
where  $\kappa^{\prime}(hx) = \rho_{out}(h)\kappa^{\prime}(x)\rho_{in}^{-1}(h)$  for any  $h\in SO(2)$ .
Since the focus of this section's discussion is spherical convolution, here we use  $s(x)$  to denote  $s_a(x)$  for any  $x \in \mathbb{S}^2$ .
For any  $x, y \in \mathbb{S}^2$ ,  $s(x)[1,0,0]^T$ ,  $s(x)[0,1,0]^T$  attached to  $x$  are tangent vectors on  $x$ , we parallel transport  $s(x)[1,0,0]^T$  and  $s(x)[0,1,0]^T$  along the geodesic between  $x$  and  $y$  and get two tangent vectors on  $y$ , denoted as  $s(x \to y)_1$  and  $s(x \to y)_2$  as shown in the figure 19, where the parallel transport along a smooth curve is a way to translate a vector "parallelly" based on the affine connection, that is, for a smooth curve  $\gamma : [0,1] \to \mathbb{S}^2$ , the parallel transport  $X: \mathrm{Im}(\gamma) \to \mathcal{T}\mathbb{S}^2$  along the curve  $\gamma$  satisfies that  $\nabla_{\dot{\gamma}(t)}X = 0$ , where  $\mathrm{Im}(\gamma) = \{\gamma(t) | t \in [0,1]\}$  and  $\nabla$  is the affine connection.
$s(x \to y)_1$  and  $s(x \to y)_2$  need to undergo a transformation in  $SO(2)$  to align with  $s(y)[1,0,0]^T$  and  $s(y)[0,1,0]^T$  on  $\mathbf{y}$  as shown in the figure 19. We denote the transformation as  $h_{x \to y}$ .
With the above notation, the spherical convolution can be expressed as: