SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In addition, the most important details of the different models we trained are discussed in Appendix B.
E Computational resources
Training times substantially depend on the dataset used for fitting. For instance, fitting the PET model with  $d_{\mathrm{PET}} = 128$  on the  $\mathrm{CH}_4$  dataset using 1000 energy-only training samples (the very first point of the learning curve in Fig. 3d) takes about 40 minutes on a V100 GPU. In contrast, fitting the PET model with  $d_{\mathrm{PET}} = 256$  on energies and forces using 300,000 samples is estimated to take about 26 GPU-days on a V100 (in practice the model was fitted partially on a V100 and partially on an RTX-4090). A similar computational effort is associated with training the  $d_{\mathrm{PET}} = 256$  model on the COLL dataset. It is worth mentioning, however, that the model already improves upon the previous state-of-the-art model in forces after approximately one-third of the total fitting time on the validation dataset. We used  $d_{\mathrm{PET}} = 256$  models only for 3 cases: 1)  $\mathrm{CH}_4 \mathrm{E} + \mathrm{F}$ , 100k samples, 2)  $\mathrm{CH}_4 \mathrm{E} + \mathrm{F}$ , 300k samples, and 3) COLL dataset. All the other calculations were substantially cheaper. For example, using a V100 GPU it takes about 30h to achieve the best validation error on the HME21 dataset for a single  $d_{\mathrm{PET}} = 128$  model.
F ECSE
The overall idea behind the ECSE symmetrization protocol is relatively simple, but an efficient implementation requires a rather complicated construction, that we discuss here in detail.
Figure 5: Smooth cutoff functions  $f_{c}$ ,  $q_{c}^{1}$  and  $q_{c}^{2}$ .
F.1 Preliminary definitions
We begin by defining several mathematical functions that we use in what follows. In order to obtain smooth predictions, PET uses several cutoff functions. While there is a degree of flexibility in defining them, we found the following functional forms, that are also illustrated in Fig. 5, to be effective:
$$
f _ {c} \left(r \mid R _ {\mathrm {c}}, \Delta_ {R _ {c}}\right) = \left\{ \begin{array}{l l} 1, & \text {i f} r \leq R _ {\mathrm {c}} - \Delta_ {R _ {c}} \\ \frac {1}{2} \left(\tanh  \left(\frac {1}{x + 1} + \frac {1}{x - 1}\right) + 1\right); x = 2 \frac {r - R _ {c} + 0 . 5 \Delta_ {R _ {c}}}{\Delta_ {R _ {c}}}, & \text {i f} R _ {\mathrm {c}} - \Delta_ {R _ {c}} <   r <   R _ {\mathrm {c}}, \\ 0, & \text {i f} r \geq R _ {\mathrm {c}} \end{array} \right. \tag {9}
$$
$$
q _ {c} ^ {1} (z | \omega , \Delta_ {\omega}) = \left\{ \begin{array}{l l} 0, & \text {i f} z \leq \omega \\ \frac {1}{2} (- \tanh  \left(\frac {1}{x + 1} + \frac {1}{x - 1}\right) + 1); x = 2 \frac {z - \omega - 0 . 5 \Delta_ {\omega}}{\Delta_ {\omega}} & \text {i f} \omega <   z <   \omega + \Delta_ {\omega}, \\ 1, & \text {i f} z \geq \omega + \Delta_ {\omega} \end{array} \right. \tag {10}
$$
$$
q _ {c} ^ {2} (z | \omega , \Delta_ {\omega}) = \left\{ \begin{array}{l l} 0, & \text {i f} z \leq \omega \\ \frac {z}{2} (- \tanh  \left(\frac {1}{x + 1} + \frac {1}{x - 1}\right) + 1); x = 2 \frac {z - \omega - 0 . 5 \Delta_ {\omega}}{\Delta_ {\omega}} & \text {i f} \omega <   z <   \omega + \Delta_ {\omega}, \\ z, & \text {i f} z \geq \omega + \Delta_ {\omega} \end{array} \right. \tag {11}
$$
We also need smooth functions to prune the ensemble of coordinate systems. To do so, we use several variations on a well-known smooth approximation for the max function. Given a finite set of numbers  $\{x_i\}_i$  and a smoothness parameter  $\beta > 0$  one can define:
$$
\operatorname {S m o o t h M a x} \left(\left\{x _ {i} \right\} _ {i} | \beta\right) = \frac {\sum_ {i} \exp \left(\beta x _ {i}\right) x _ {i}}{\sum_ {i} \exp \left(\beta x _ {i}\right)}. \tag {12}
$$
This function satisfies the following properties:
$$
\lim  _ {\beta \rightarrow + \infty} \operatorname {S m o o t h M a x} (\{x _ {i} \} _ {i} | \beta) \leq \max  (\{x _ {i} \} _ {i}) \tag {13}
$$
For a set of just two numbers  $\{x_1, x_2\}$  one can show that SmoothMax is bounded from below:
$$
\begin{array}{c}\operatorname {S m o o t h M a x} (\left\{x _ {1}, x _ {2} \right\}) + T (\beta) \geq \max  (\left\{x _ {1}, x _ {2} \right\})\\\lim  _ {\beta \rightarrow + \infty} T (\beta) = 0\end{array}\tag {14}
$$
for  $T(\beta) = W(\exp(-1)) / \beta$ , where  $W$  is the Lambert W function.
For the case when the numbers  $x_{i}$  are associated with weights  $p_i \geq 0$  it is possible to extend the definition of SmoothMax as:
$$
\operatorname {S m o o t h M a x W e i g h t e d} \left(\left\{\left(x _ {i}, p _ {i}\right) \right\} _ {i} | \beta\right) = \frac {\sum_ {i} \exp \left(\beta x _ {i}\right) p _ {i} x _ {i}}{\sum_ {i} \exp \left(\beta x _ {i}\right) p _ {i}}. \tag {15}
$$
This function satisfies the properties in Eq. (13), where maximum is taken only of the subset of  $\{x_{i}\}$  where  $p_i > 0$ . A useful consequence of the definition of SmoothMaxWeighted is that its value is not changed when including a point with zero weight,  $(x,0)$ :
$$
\operatorname {S m o o t h M a x W e i g h t e d} \left(\left\{\left(x _ {i}, p _ {i}\right) \right\} _ {i} \cup \{(x, 0) \} | \beta\right) = \operatorname {S m o o t h M a x W e i g h t e d} \left(\left\{\left(x _ {i}, p _ {i}\right) \right\} _ {i} | \beta\right) \tag {16}
$$
Corresponding functions that provide a smooth approximation to the min operation, SmoothMin and SmoothMinWeighted, can be defined the same way as SmoothMax and SmoothMaxWeighted using  $(- \beta)$  instead of  $\beta$ .
Finally, given an ordered pair of noncollinear unit vectors  $\hat{v}_1$  and  $\hat{v}_2$  we define the associated coordinate system by computing  $\mathbf{u}_1 = \hat{v}_1 \times \hat{v}_2$ ,  $\hat{u}_1 = \mathbf{u}_1 / \| \mathbf{u}_1\|$  and  $\hat{u}_2 = \hat{v}_1 \times \hat{u}_1$ . The reference frame is given by the vectors  $\hat{v}_1$ ,  $\hat{u}_1$  and  $\hat{u}_2$ . This procedure generates only right-handed coordinate systems, and thus, the transformation between the absolute coordinate system to the atom-centered one can always be expressed as a proper rotation, never requiring inversion. When we later refer to the coordinate system given by a central atom  $i$  and two neighbors  $j$  and  $j'$ , we mean that the vectors  $\hat{v}_1$  and  $\hat{v}_2$  are the ones pointing from the central atom to neighbors  $j$  and  $j'$  respectively.
F.2 Assumptions
We assume that (1) there is a finite number of neighbors within any finite cutoff sphere and (2) there is a lower boundary  $d_{\mathrm{min}}$  to the distance between pairs of points. Neither of these assumptions is restrictive in chemical applications, because molecules and materials have a finite atom density, and because atoms cannot overlap when working on a relevant energy scale.
F.3 Fully-collinear problem
The simplest version of ECSE protocol was defined in the main text in the following way:
$$
\mathbf {y} _ {\mathrm {S}} \left(A _ {i}\right) = \sum_ {j j ^ {\prime} \in A _ {i}} w _ {j j ^ {\prime}} \hat {R} _ {j j ^ {\prime}} \left[ \mathbf {y} _ {0} \left(\hat {R} _ {j j ^ {\prime}} ^ {- 1} [ A _ {i} ]\right) \right] / \sum_ {j j ^ {\prime} \in A _ {i}} w _ {j j ^ {\prime}}, \tag {17}
$$
$$
w _ {j j ^ {\prime}} = w \left(\mathbf {r} _ {j}, \mathbf {r} _ {j ^ {\prime}}\right) = f _ {\mathrm {c}} \left(r _ {j} \mid R _ {\mathrm {c}}, \Delta_ {R _ {\mathrm {c}}}\right) f _ {\mathrm {c}} \left(r _ {j ^ {\prime}} \mid R _ {\mathrm {c}}, \Delta_ {R _ {\mathrm {c}}}\right) q _ {\mathrm {c}} \left(\left| \hat {r} _ {j} \times \hat {r} _ {j ^ {\prime}} \right| ^ {2} \mid \omega , \Delta_ {\omega}\right). \tag {18}
$$
The  $q_{c}$  function can be chosen as either  $q_{c}^{1}$  or  $q_{c}^{2}$ , while  $\omega$  and  $\Delta_{\omega}$  are user-specified constant parameters.
This simple formulation doesn't handle correctly the corner case of when all pairs of neighbors form nearly collinear triplets with the central atom. In this scenario, for all pairs of  $j$  and  $j'$ ,  $|\hat{r}_j \times \hat{r}_{j'}|^2$  is smaller than  $\omega$  which implies that all the angular cutoff values  $q_{\mathrm{c}}(|\hat{r}_j \times \hat{r}_{j'}|^2|\omega, \Delta_{\omega})$  are zeros, and so all the weights  $w_{jj'}$ . As a result, the ECSE weighted average is undefined, falling to  $\frac{0}{0}$  ambiguity. We propose two solutions to treat this case.
First solution to the fully-collinear problem. The first solution we propose is to incorporate predictions of some internally equivariant model in the functional form of ECSE:
$$
\mathbf {y} _ {\mathrm {S}} \left(A _ {i}\right) = \left(w _ {\text {a u x}} \mathbf {y} _ {\text {a u x}} \left(A _ {i}\right) + \sum_ {j j ^ {\prime} \in A _ {i}} w _ {j j ^ {\prime}} \hat {R} _ {j j ^ {\prime}} \left[ \mathbf {y} _ {0} \left(\hat {R} _ {j j ^ {\prime}} ^ {- 1} [ A _ {i} ]\right) \right]\right) / \left(w _ {\text {a u x}} + \sum_ {j j ^ {\prime} \in A _ {i}} w _ {j j ^ {\prime}}\right), \tag {19}
$$
In this case,  $\mathbf{y}_{\mathrm{S}}(A_i)$  can fall to  $\mathbf{y}_{\mathrm{aux}}(A_i)$  if all the weights  $w_{jj'}$  are zeros.
Under the assumption that the (typically simpler) auxiliary model will be less accurate than the main backbone architecture, it is desirable to ensure that it is only used to make predictions for corner cases. For this purpose we define  $w_{\mathrm{aux}}$  as follows:
$$
w _ {\text {a u x}} = f _ {\mathrm {c}} \left(\operatorname {S m o o t h M a x W e i g h t e d} \left(\left\{w _ {j j ^ {\prime}}, w _ {j j ^ {\prime}} \right\} _ {j j ^ {\prime}} \mid \beta_ {\omega}\right) \mid t _ {\text {a u x}}, \Delta_ {\text {a u x}}\right). \tag {20}
$$
With this expression,  $w_{\mathrm{aux}}$  takes a non-zero weight only when the smooth maximum of the coordinate-system weights  $w_{jj'}$  is below  $t_{\mathrm{aux}}$ : in other words, as long as at least one pair of neighbors is non-collinear, the auxiliary model will be ignored. Since SmoothMaxWeighted does not depend on quantities with zero weight (see Eq. (16)), one can compute Eq. (20) efficiently, without explicit iterations over the coordinate systems with zero weight.  $\beta_{\omega}$ ,  $t_{\mathrm{aux}}$  and  $\Delta_{\mathrm{aux}}$  are user-specified constant parameters.
Second solution to the fully-collinear problem. Even though the use of an auxiliary model is an effective and relatively simple solution, it would be however desirable to use consistently the non-equivariant backbone architecture. In this section, we discuss a sketch of how one could tackle this corner-case without the usage of an auxiliary model.
To eliminate the  $0 / 0$  singularity, we need to use an adaptive definition for the angular cutoff parameters  $\omega$  and  $\Delta_{\omega}$  that enter the definition (18) of the ECSE weights:
$$
\omega = \Delta_ {\omega} = \frac {1}{2} \operatorname {S m o o t h M a x} \left(\left\{\left(\left| \hat {r} _ {j} \times \hat {r} _ {j ^ {\prime}} \right| ^ {2}\right) \right\} _ {j j ^ {\prime}} \mid \beta\right). \tag {21}
$$
With this definition, the weights are never zero except for the exact fully-collinear singularity, which makes the weighted average of the ECSE protocol well-defined everywhere. For environments approaching the exact fully-collinear singularity, the coordinate systems used by ECSE are not stable. However, given the definition of the coordinate systems discussed in Appendix F.1, the  $x$  axes of all the coordinate systems are aligned along the collinear direction. Given that the atomic environment itself is nearly collinear, the input to the non-equivariant models is always the same, irrespective of the  $y$  and  $z$  axes, that will be arbitrarily oriented in the orthogonal plane. Thus, all evaluations of the backbone architecture  $\mathbf{y}_0$  lead to approximately the same predictions. Note that this is only true if all the neighbors that are used for the backbone architecture are also considered to build coordinate systems, which requires that the cutoff radius employed by ECSE is larger than that of the backbone architecture. When atoms approach smoothly a fully-collinear configuration, the coordinates that are input to the backbone architecture converge smoothly to being fully aligned along  $x$  for all coordinate systems, making the ECSE average smooth.
One small detail is that, within this protocol, the orientation of the  $x$  axis can be arbitrary. This problem is easily solved by always including an additional coordinate system in which  $\hat{v}_1$  is oriented along  $-\mathbf{r}_{ij}$  rather than along  $\mathbf{r}_{ij}$ . Both coordinate systems should be used with the same weight  $w_{jj'}$ . One should note, however, that this approach has a number of limitations. In particular, it doesn't support covariant inputs, such as vectorial spins associated with each atom, and covariant outputs, such as dipole moments.
F.4 Adaptive inner cutoff radius
The minimal inner cutoff radius  $R_{\mathrm{in}}$  should be chosen to ensure that it includes at least one pair of neighbors that define a "good" coordinate system. To determine an inner cutoff that contains at least a pair of neighbors, we proceed as follows
1. For each pair of neighbors, we use SmoothMax  $(r_j,r_j^{\prime}|\beta)$  to select the farthest distance in each pair
2. Applying a SmoothMin function to the collection of neighbor pairs
$$
\operatorname {S m o o t h M i n} \left(\left\{\operatorname {S m o o t h M a x} \left(\left\{r _ {j}, r _ {j ^ {\prime}} \right\} \mid \beta\right) + T (\beta) \right\} _ {j j ^ {\prime}} \mid \beta\right) \tag {22}
$$
selects a distance that contains at least a pair of neighbors. That this function is always greater or equal to the second-closest pair distance follows from the inequality (14) and the SmoothMin analogue of Eq. (13).
3. We ensure to pick at least one non-collinear pair (if there is one within  $R_{\mathrm{out}}$ ) by defining the adaptive inner cutoff as
$$
R _ {\text {i n}} = \operatorname {S m o o t h M i n W e i g h t e d} \left(\left\{\operatorname {S m o o t h M a x} \left(\left\{r _ {j}, r _ {j ^ {\prime}} \right\} \mid \beta\right) + T (\beta), p _ {j j ^ {\prime}} \right\} _ {j j ^ {\prime}} \cup \left\{\left(R _ {\text {o u t}}, 1. 0\right) \mid \beta\right) + \Delta_ {R _ {c}}, \right. \tag {23}
$$
where  $p_{jj'}$  are defined as:
$$
p _ {j j ^ {\prime}} = f _ {c} \left(r _ {j} \mid R _ {\text {o u t}}, \Delta_ {R _ {c}}\right) f _ {c} \left(r _ {j ^ {\prime}} \mid R _ {\text {o u t}}, \Delta_ {R _ {c}}\right) q _ {c} ^ {1} \left(\left| \hat {r} _ {j} \times \hat {r} _ {j ^ {\prime}} \right| ^ {2} \mid \omega + \Delta_ {\omega}, \Delta_ {\omega}\right). \tag {24}
$$
$j$  and  $j^{\prime}$  run over all the neighbors inside the outer cutoff sphere.
This definition maintains smoothness with respect to the (dis)appearance of atoms at the outer cutoff sphere. It ensures that if there's at least one good pair of neighbors inside the outer cutoff sphere, there's at least one within the inner cutoff as well. If no such good pair exists within the outer cutoff sphere, the definition falls back to  $R_{\mathrm{out}}$ . Note that the expression includes also some "tolerance" parameters  $\Delta_{R_c}$  and  $\Delta_{\omega}$ . These are introduced to ensure that at least one of the weights computed to define the equivariant ensemble in Eq. (24) is of the order of 1.
F.5 Weights pruning
Given that the cost of applying ECSE depends on the number of times the inner model needs to be evaluated, it is important to minimize the number of coordinate systems that have to be considered. To do so, one can e.g. disregard all coordinate systems with weights below a threshold, e.g. half of the maximum weight. A smooth version of this idea can be implemented as follows:
Constant parameters:   
 $\beta_{\mathrm{w}}:\beta_{\mathrm{w}} > 0$ $T_{f},\Delta_{T_{f}}:T_{f} > 0,\Delta_{T_{f}} > 0,T_{f} <   1$ $T_{e},\Delta_{T_{e}}:T_{e} > 0,\Delta_{T_{e}} > 0,\Delta_{T_{e}} <   T_{e}$  (T stands for Threshold).   
Smooth weights pruning:   
1:  $w_{\mathrm{MAX}}\gets \mathrm{SmoothMaxWeighted}(\{(w_{jj^{\prime}},w_{jj^{\prime}})\}_{jj^{\prime}}|\beta_{\mathrm{w}})$    
2:  $f_{jj^{\prime}}\gets q_c^1 (w_{jj^{\prime}}|w_{\mathrm{MAX}}T_f,w_{\mathrm{MAX}}\Delta_{T_f})$    
3:  $e\gets f_c(w_{\mathrm{MAX}}|T_e,\Delta_T_e)$    
4:  $w_{jj^{\prime}}\gets ew_{jj^{\prime}} + (1 - e)w_{jj^{\prime}}f_{jj^{\prime}}$
First, one computes the maximum weight present, using a SmoothMaxWeighted function. Then, pruning factors  $f_{jj'}$  are computed using the cutoff  $q_c^1$  function, aiming to zero out the smallest weights with  $w_{jj'} \gets w_{jj'} f_{jj'}$ . An additional modification is needed to ensure smooth behavior around fully-collinear configurations, where all weights become zero. If this occurs,  $q_c^1(w_{jj'} | w_{MAX} T_f, w_{MAX} \Delta T_f)$  converges to a step function. Therefore, it makes sense to activate weight pruning only if  $w_{MAX}$  is larger than a certain threshold  $T_e$ , which is implemented using a smooth activation factor  $e$ . We found this pruning to be more efficient if the  $q_c$  function in Eq. (4) is selected to be  $q_c^2$  rather than  $q_c^1$ . For multi-component systems, an alternative approach (which can be used on top of the discussed strategy) to increase efficiency involves using only a subset of all the atoms of specific atomic species to define coordinate systems (e.g., only O atoms in liquid water).
F.6 Tradeoff between smoothness and computational efficiency
For any set of selected parameters, the ECSE protocol yields a smooth approximation of the target property in a rigorous mathematical sense, meaning the resulting approximation of the target property is continuously differentiable. The important question, however, pertains to the smoothness of this approximation in a physical sense. That is, how quickly it oscillates or, equivalently, how large the associated derivatives are. In this section, we discuss the tradeoff between "physical" smoothness and the computational efficiency of the ECSE protocol. This tradeoff is controlled by the user-specified parameters discussed in the previous paragraphs.
$\beta$  is an example of such parameters, with a clear impact on the regularity of equivariant predictions. Among other things, it is used to determine the inner cutoff radius,  $R_{\mathrm{in}}$ , which is defined along the lines with a smooth approximation of the second minimum of all the distances from the central atom to all its neighbors. Assigning a relatively low value to  $\beta$  generates a very smooth approximation of this second minimum, but it significantly overestimates the exact value, and therefore  $R_{\mathrm{in}}$  potentially includes more neighbor pairs than needed. In contrast, for a large value of  $\beta$ , the second minimum approximation reflects the exact value of the second minimum more precisely, at the cost of decreasing the smoothness of the approximation when the distances between atoms close to the center vary. Even though the ensemble average remains continuously differentiable for any  $\beta > 0$ , it might develop unphysical spikes in the derivatives, reducing its accuracy. To see more precisely the impact of  $\beta$  consider the limit of a large value (considering also that other adjustment parameters such as  $\Delta_{R_c}$  are set to a value that does not entail an additional smoothening effect). In this limit, the inner cutoff selected by the ECSE protocol encompasses only one pair of neighbors for a general position, so most of the time the ECSE protocol uses only one coordinate system defined by the two closest neighbors. Only when the nearest pair of atoms changes, within a narrow transition region, two coordinate systems are used simultaneously. The value of  $\beta$  determines the characteristic size of this transition region. Thus, for excessively large values of  $\beta$ , there is a sharp (but still continuously differentiable) transition from one coordinate system to another. On the other hand, if  $\beta$  is set to a very low value,  $R_{\mathrm{in}}$  will always be significantly larger than the exact distance to the second neighbor. As a result, the ECSE protocol will constantly utilize multiple coordinate systems, leading to a smoother symmetrized model but with a higher computational cost.
In summary, the parameters used by the ECSE dictate the balance between smoothness and computational efficiency. We name the selection of such parameters that leads to a very smooth model as "loose". The opposite selection that minimizes computational cost is labeled as "tight". The proper selection of these parameters directly impacts the accuracy of the symmetrized model in forces (and in higher derivatives), which we discuss in the next section.
F.7 Accuracy of a-posteriori-ECSE-symmetrized models
The most computationally efficient method for fitting a model is to initially train a non-equivariant backbone architecture with random rotational augmentations and then apply the ECSE protocol a posteriori. There are general considerations one can make on the expected accuracy of the symmetrized model relative to the initial, non-equivariant one.
The functional form of ECSE is a weighted average of predictions generated by a backbone architecture for several coordinate systems. Therefore, it can be seen as implementing the rotational augmentation strategy during inference. This implies that the accuracy of ECSE in energies (and in any other direct targets) is expected to be bounded between the accuracy of standard single inference of the backbone architecture and that evaluated with fully-converged rotational augmentations.
Forces, however, present a more complicated situation. The forces produced by ECSE are not just linear combinations of the predictions given by the backbone architecture. This complexity stems from the fact that the coordinate systems and weights used by ECSE depend on atomic positions themselves. Since forces are defined by the negative gradient of energy, it is necessary to take the derivatives of Eq. (17) with respect to atomic positions to obtain the predictions of ECSE. Additional terms related to  $\frac{\partial\hat{R}_{jj^{\prime}}}{\partial\mathbf{r}_k}$  and  $\frac{\partial w_{jj^{\prime}}}{\partial\mathbf{r}_k}$  are not associated with the forces produced by a backbone architecture  $\frac{\partial\mathbf{y}_0([A_i])}{\partial\mathbf{r}_k}$ . Consequently, the error in forces of the final, symmetrized model may significantly exceed that of the backbone architecture. This issue is more pronounced for a tight selection of ECSE parameters and less pronounced for a loose one. In principle, this can be mitigated by fine-tuning the overall symmetrized model. In this work, we have selected loose parameters of the ECSE protocol sufficient to ensure that the difference in accuracy between the backbone architectures and symmetrized models is minuscule, even without fine-tuning, which results however in a more pronounced computational overhead.
F.8 Additional rotational augmentations
In some cases, such as for the  $\mathrm{CH}_4$  configurations, the total number of all possible coordinate systems is very limited. However, using a larger number of rotational augmentations may still be beneficial during inference. For such cases, it's possible to incorporate additional augmentations into the ECSE protocol as follows:
$$
\mathbf {y} _ {\mathrm {S}} \left(A _ {i}\right) = \frac {\sum_ {j j ^ {\prime} \in A _ {i}} \sum_ {\hat {R} _ {\mathrm {a u g}} \in S _ {\mathrm {a u g}}} w _ {j j ^ {\prime}} \hat {R} _ {j j ^ {\prime}} \hat {R} _ {\mathrm {a u g}} \left[ \mathbf {y} _ {0} \left(\hat {R} _ {\mathrm {a u g}} ^ {- 1} \hat {R} _ {j j ^ {\prime}} ^ {- 1} [ A _ {i} ]\right) \right]}{N _ {\mathrm {a u g}} \sum_ {j j ^ {\prime} \in A _ {i}} w _ {j j ^ {\prime}}}, \tag {25}
$$
where  $S_{\mathrm{aug}}$  is a predefined constant set of random rotations  $\hat{R}_{\mathrm{aug}}$ .
F.9 Application of the ECSE protocol to long-range models