SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \underset {\hat {\mathbf {v}} ^ {0}} {\operatorname {a r g m i n}} \sum_ {p _ {i}} \left(\mathbf {x} _ {i} ^ {T} - \hat {\mathbf {x}} _ {i} ^ {T}\right) (26) \\ = \sum_ {p _ {i}} \left(\mathbf {x} _ {i} ^ {T} - \mathbf {x} _ {i} ^ {0} - \hat {\mathbf {v}} _ {i} ^ {0} T\right) (27) \\ = \sum_ {p _ {i}} T \left(\frac {\mathbf {x} _ {i} ^ {T} - \mathbf {x} _ {i} ^ {0}}{T} - \hat {\mathbf {v}} _ {i} ^ {0}\right) (28) \\ = T \sum_ {p _ {i}} \left(\mathbf {v} _ {i} ^ {t ^ {*}} - \left(\phi_ {v} \left(\mathbf {h} _ {i}\right) \mathbf {v} _ {i} ^ {0} + \frac {\sum_ {j \neq i} \left(\mathbf {x} _ {i} ^ {0} - \mathbf {x} _ {j} ^ {0}\right) \mathbf {m} _ {i j} ^ {0}}{N - 1}\right)\right) (29) \\ = T \sum_ {p _ {i}} \left(\mathbf {v} _ {i} ^ {t ^ {*}} - \left(w ^ {0} \mathbf {v} _ {i} ^ {0} + \mathbf {b} ^ {0}\right)\right), (30) \\ \end{array}
$$
where  $w^0 \in \mathbb{R}^1$  and  $\mathbf{b}^0 \in \mathbb{R}^3$  denote the learnable variables irrelevant to  $\mathbf{v}_i^0$  and  $t$ , concluding the proof.
Proof. As the higher order cases  $(k\geq 1)$  have already been proved, we only need to show that  $\epsilon_{\mathrm{NC}(0)}\geq \epsilon_{\mathrm{NC}(1)}$ . The first order of Newton-Cotes formula NC (1) is also known as Trapezoidal rule, i.e.:
$$
\int_ {0} ^ {T} \mathbf {v} (t) d t \approx \frac {T}{2} \left(\mathbf {v} ^ {0} + \mathbf {v} ^ {T}\right). \tag {31}
$$
As aforementioned, the actual integration  $\mathbf{x}^T -\mathbf{x}^0$  for different training examples is different, and we assume that it fluctuates around the base estimation  $\frac{T}{2}\big(\mathbf{v}^0 +\mathbf{v}^T\big)$  and follows a normal distribution  $\mathcal{N}_{NC(1)}$ , where the variance  $\sigma_{NC(1)}^2$  is positively correlated with the difficulty of optimizing the overall objective. The variance of  $\mathcal{N}_{NC(1)}$  is:
$$
\sigma_ {N C (1)} ^ {2} = \frac {\sum_ {p} \left(\int_ {0} ^ {T} (\mathbf {v} (t) - \sum_ {k = 0} ^ {1} C ^ {k} t ^ {(k)}) d t\right) ^ {2}}{N T ^ {2}}, \tag {32}
$$
where the integration term is a general form of polynomial interpolation error. According to Equation 14, it can be derived to:
$$
\int_ {0} ^ {T} \left(\frac {\left(t - t ^ {0}\right) \left(t - t ^ {1}\right) \mathbf {v} ^ {\prime \prime} (\xi)}{2 !}\right) d t, \tag {33}
$$
where  $\mathbf{v}''$  denote the second derivative of  $\mathbf{v}$ . Let  $s = \frac{t - t^0}{T}$ , then  $t = t^0 + sh$  and  $dt = d(\mathbf{v}^0 + sh) = Tds$ . The above equation can be re-written as:
$$
T \int_ {0} ^ {1} \frac {s (s - 1) T ^ {2} \mathbf {v} ^ {\prime \prime} (\xi)}{2} d s = - \frac {1}{1 2} T ^ {3} \mathbf {v} ^ {\prime \prime} (\xi) = \mathcal {O} \left(T ^ {3}\right). \tag {34}
$$
Therefore, the final  $\sigma_{NC(1)}^2$  is:
$$
\begin{array}{l} \sigma_ {N C (1)} ^ {2} = \frac {\sum_ {p} \left(- \frac {1}{1 2} T ^ {3} \mathbf {v} ^ {\prime \prime} (\xi)\right) ^ {2}}{N T ^ {2}} (35) \\ = \mathcal {O} \left(T ^ {4}\right) \leq \mathcal {O} \left(T ^ {2}\right) = \sigma_ {N C (0)} ^ {2}, (36) \\ \end{array}
$$
Figure 6: The prediction errors of intermediate velocities on MD17 dataset, w.r.t. training epoch. The blue and green lines denote the prediction errors of NC and  $\mathrm{NC}^{+}$ , respectively.
(a) NC(2)
Figure 7: The prediction errors of intermediate velocities on N-body dataset, w.r.t. training epoch. The blue and green lines denote the prediction errors of NC and  $\mathrm{NC}^{+}$ , respectively.
(a) NC(2) intermediate
(b) NC(3) intermediate
(d) NC(2) final
Figure 8: Visualization of the intermediate velocities w.r.t.  $k$  of MD17 (Aspirin). The red, blue, and green lines denote the target, prediction of NC, and prediction of  $\mathrm{NC}^{+}$ , respectively.
(c) NC(5) intermediate
Proof. The GNN models possessing equivariance property are equivariant to the translation, rotation, and permutation of input. NC directly feeds the input into these backbone models and naturally possesses this property.
Formally, let  $T_g: \mathbf{x} \to \mathbf{x}$  be a group of transformation operations. If the backbone model  $\mathcal{M}$  is equivariant then we will have:
$$
\mathcal {M} \left(T _ {g} (\mathbf {x})\right) = S _ {g} \left(\mathcal {M} (\mathbf {x})\right), \tag {37}
$$
where  $S_{g}$  is an equivalent transformation to  $T_{g}$  on the output space. NC can be regarded as a weighted combination of the outputs of  $\mathcal{M}$ :
$$
\sum_ {i} w _ {i} \mathcal {M} \left(T _ {g} (\mathbf {x} _ {i})\right) = \sum_ {i} w _ {i} S _ {g} \left(\mathcal {M} (\mathbf {x} _ {i})\right), \tag {38}
$$
where the Newton-Cotes weights  $w_{i}$  is constant and irrelevant to the input, the output, and the model  $\mathcal{M}$  itself. Therefore, the above equation will always hold.
The average intermediate velocity prediction errors on MD17 and Motion datasets are shown in Figure 6 and Figure 7, respectively. NC still learned the intermediate velocities we did not feed
Table 6: Hyper-parameter settings in the main experiments.
Datasets	k	velocity regularization	velocity regularization decay	parameter regularization	parameter regularization decay	loss criterion	input feature normalization	intermediate velocity normalization
N-body	2	0.001	0.999	1.0	0.99	MSE	False	True
MD17	2	0.1	0.999	1.0	0.95	MSE	True	True
Motion	2	0.01	0.999	1.0	0.95	MSE	True	True
# epoch	batch-size	# training examples	activation	# layers	learning rate	optimizer	clip gradient norm
N-body	1,500	200	500	ReLU	4	0.0005	Adam	1.0
MD17	1,000	100	500	ReLU	4	0.001	Adam	0.1
Motion	1,500	100	200	ReLU	4	0.0005	Adam	1.0
the intermediate into it. Particularly, the error on N-body dataset was small and stable, which may demonstrate the effectiveness of estimating intermediate velocities even without supervised data.
We also provide some visualized examples on MD17 dataset in Figure 8, from which we can observe the similar results compared with Figure 4 in the main paper.
D Hyper-parameter Setting
We list the main hyper-parameter setting of NC with different EGNN models on different datasets in Table 6. We used the Adam optimizer [53] and adopted layer normalization [54] and ReLU activation [55]) for all settings. For a fair comparison, the parameter settings for the backbone EGNN models were identical to these in the existing implementation [19].