SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lingbing Guo $^{1,2,3}$ , Weiqing Wang $^{4}$ , Zhuo Chen $^{1,2,3}$ , Ningyu Zhang $^{1,2}$
Zequn Sun $^{5}$ , Yixuan Lai $^{1,2,3}$ , Qiang Zhang $^{1,3\dagger}$  and Huajun Chen $^{1,2,3\dagger}$
$^{2}$ Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph
$^{3}$ ZJU-Hangzhou Global Scientific and Technological Innovation Center
$^{4}$ Department of Data Science & AI, Monash University
$^{5}$ State Key Laboratory for Novel Software Technology, Nanjing University
Reasoning system dynamics is one of the most important analytical approaches for many scientific studies. With the initial state of a system as input, the recent graph neural networks (GNNs)-based methods are capable of predicting the future state distant in time with high accuracy. Although these methods have diverse designs in modeling the coordinates and interacting forces of the system, we show that they actually share a common paradigm that learns the integration of the velocity over the interval between the initial and terminal coordinates. However, their integrand is constant w.r.t. time. Inspired by this observation, we propose a new approach to predict the integration based on several velocity estimations with Newton-Cotes formulas and prove its effectiveness theoretically. Extensive experiments on several benchmarks empirically demonstrate consistent and significant improvement compared with the state-of-the-art methods.
Reasoning the time evolution of dynamic systems has been a long-term challenge for hundreds of years [1, 2]. Despite that the advances in computer manufacturing nowadays make it possible to simulate long trajectories of complicated systems constrained by multiple force laws, the computational cost still imposes a heavy burden on the scientific communities [3-8].
Recent graph neural networks (GNNs) [9-13]-based methods provide an alternative solution to predict the future states directly with only the initial state as input [14-20]. Take molecular dynamics (MD) [4, 21-24] as an example, the atoms in a molecule can be regarded as nodes with different labels, and thus can be encoded by a GNN. As the input and the target are atomic coordinates, the learning problem becomes a complicated regression task of predicting the coordinate at each dimension of each atom. Furthermore, some directional information (e.g., velocities or forces) are also important features and cannot be directly leveraged especially considering the rotation and translation of molecules in the system. Therefore, the existing works put great effort into the reservation of physical symmetries, e.g., transformation equivariance and geometric constraints [18, 19]. The experimental results on a variety of benchmarks also demonstrate the advantages of their methods.
Without loss of generality, the main target of the existing works is to predict the future coordinate  $\mathbf{x}^T$  distant to the given initial coordinate  $\mathbf{x}^0$ , with  $\mathbf{x}^0$  and the initial velocity  $\mathbf{v}^0$  as input. Then, the
$^{\dagger}$  Correspondence to: {qiang.zhang.cs, huajunsir} @ zju.edu.cn,
predicted coordinate  $\hat{\mathbf{x}}^T$  can be written as:
$$
\hat {\mathbf {x}} ^ {T} = \mathbf {x} ^ {0} + \hat {\mathbf {v}} ^ {0} T. \tag {1}
$$
For a complicated system comprising multiple particles, predicting the velocity term  $\hat{\mathbf{v}}^0$  rather than the coordinate  $\hat{\mathbf{x}}^T$  improves the robustness and performance. Specifically, by subtracting the initial coordinate  $\mathbf{x}^0$ , the learning target can be regarded as the normalized future coordinate, i.e.,  $\hat{\mathbf{v}}^0 = (\mathbf{x}^T - \mathbf{x}^0) / (T - 0)$ , which is why all state-of-the-art methods adopt this strategy [18, 19]. Nevertheless, the current strategy still has a significant drawback from the view of numerical integration.
As illustrated in Figure 1, supposed that the actual velocity of a particle is described by the curve  $v(t)$ . To predict the future state  $\mathbf{x}^T$  at  $t = T$ , the existing methods adopt a constant estimation approach, i.e.,  $\hat{\mathbf{x}}^T = \mathbf{x}^0 + \hat{\mathbf{v}}^0 T = \mathbf{x}^0 + \int_0^T \hat{\mathbf{v}}^0 dt$ . Evidently, the prediction error could be significant as it purely relies on the fitness of neural models. If we alternatively choose a simple two-step estimation (i.e., Trapezoidal rule [25]), the prediction error may be reduced to only the blue area. In other words, the model only needs to compensate for the blue area.
In this paper, we propose Newton-Cotes graph neural networks (abbr. NC) to estimate multiple velocities at different time points and compute the integration with Newton-Cotes formulas. Newton-Cotes formulas are a series of formulas for numerical integration in which the integrand (in our case,  $v(t)$ ) are evaluated at equally spaced points. One most important characteristic of Newton-Cotes formulas is that the integration is computed by aggregating the values of  $v(t)$  at these points with a group of weights  $\{w^0, \dots, w^k\}$ , where  $k$  refers to the order of Newton-Cotes formulas and  $\{w^0, \dots, w^k\}$  is irrelevant
Figure 1: Illustration of different estimations. The rectangle, trapezoid, and striped areas denote the basic (used in the existing works), two-step, and true estimations, respectively. Blue denotes the error of two-step estimation that a model needs to compensate for, whereas blue + yellow denotes the error of the existing methods.
to the integrand  $v(t)$  and can be pre-computed by Lagrange interpolating polynomial [26].
We show that the existing works can be naturally derived to the basic version NC ( $k = 0$ ) and theoretically prove that the prediction error of this estimation as well as the learning difficulty will continually reduce as the increase of estimation step  $k$ .
To better train a NC (k), we may also need the intermediate velocities as additional supervised data. Although these data can be readily obtained (as the sampling frequency is much higher than our requirement), we argue that the performance suffers slightly even if we do not use them. The model is capable of learning to generate promising velocities to better match the true integration. By contrast, if we do provide these data to train a NC (k), denoted by  $\mathrm{NC}^{+}$  (k), we will obtain a stronger model that not only achieves high prediction accuracy in conventional tasks, but also produces highly reliable results of long-term consecutive predictions.
We conduct experiments on several datasets ranging from N-body systems to molecular dynamics and human motions [27-29], with state-of-the-art methods as baselines [17-19]. The results show that NC improves all the baseline methods with a significant margin on all types of datasets, even without any additional training data. Particularly, the improvement for the method RF [17] is greater than  $30\%$  on almost all datasets.
In this section, we first introduce the related works using GNNs to learn system dynamics and then discuss more complicated methods that possess the equivariance properties.
Graph Neural Networks for Reasoning Dynamics Interaction network [30] is perhaps the first GNN model that learns to capture system dynamics. It separates the states and correlations into two different parts and employs GNNs to learn their interactions. This progress is similar to some simulation tools where the coordinate and velocity/force are computed and updated in an alternative fashion. Many followers extend this idea, e.g., using hierarchical graph convolution [9, 10, 31, 32] or auto-encoder [33-36] to encode the objects, or leveraging ordinary differential equations for energy
conservation [37]. The above methods usually cannot process the interactions among particles in the original space (e.g., Euclidean space). They instead propose an auxiliary interaction graph to compute the interactions, which is out of our main focus.
**Equivariant Graph Neural Networks** Recent methods considering physical symmetry and Euclidean equivariance have achieved state-of-the-art performance in modeling system dynamics [14-19, 38]. Specifically, [14-16] force the translation equivariance, but the rotation equivariance is overlooked. TFN [39] further leverages spherical filters to achieve rotation equivariance. SE(3) Transformer [29] proposes to use the Transformer [13] architecture to model 3D cloud points directly in Euclidean space. EGNN [18] simplifies the equivariant graph neural network and make it applicable in modeling system dynamics. GMN [19] proposes to consider the physical constraints (e.g., sticks and hinges sub-structures) widely existed in systems, which achieves the state-of-the-art performance while maintaining the same level of computational cost. SEGNN [38] extends EGNN with steerable vectors to model the covariant information among nodes and edges. Note that, not all EGNNs are designed to reason system dynamics, but the best-performing ones (e.g., EGNN [18] and GMN [19]) in this sub-area usually regard the velocity  $\mathbf{v}$  as an important feature. These methods can be set as the backbone model in NC, and the models themselves belong to the basic NC (0).
Neural Ordinary Differential Equations The neural ordinary differential equation (neural ODE) methods [40, 41] parameterize ODEs with neural layers and solve them by numerical solvers such as 4th order Runge-Kutta. These solvers are originally designed for numerical integration. Our work draws inspiration from numerical integration algorithms, where the integrand is known only at certain points. The goal is to efficiently approximate the integral to desired precision. Neural ODE methods repeatedly perform the numerical algorithms on small internals to obtain the numerical solution. Therefore, it may be feasible to iteratively perform our method to solve problems in neural ODEs.
We start from preliminaries and then take the standard EGNN [18] as an example to illustrate how the current deep learning methods work. We show that the learning paradigm of the existing methods can be derived to the simplest form of numerical integration. Finally, we propose NC and theoretically prove its effectiveness in predicting future states.
We suppose that a system comprises  $N$  particles  $p_1, p_2, \ldots, p_N$ , with the velocities  $\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_N$  and the coordinates  $\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_N$  as states. The particles can also carry various scalar features (e.g., mass and charge for atoms) which will be encoded as embeddings. We follow the corresponding existing works [17-19] to process them and do not discuss the details in this paper.
Reasoning Dynamics Reasoning system dynamics is a classical task with a very long history [1]. One basic tool for reasoning or simulating a system is numerical integration. Given the dynamics (e.g., Langevin dynamics, well-used in MD) and initial information, the interaction force and acceleration for each particle in the system can be estimated. However, the force is closely related to the real-time coordinate, which means that one must re-calculate the system states for every very small time step (i.e.,  $dt$ ) to obtain a more accurate prediction for a long time interval. For example, in MD simulation, the time step  $dt$  is usually set to 50 or 100 fs ( $1\mathrm{fs} = 10^{-15}$  second), whereas the total simulation time can be 1 ms ( $10^{-3}$  second). Furthermore, pursuing highly accurate results usually demands more complicated dynamics, imposing heavy burdens even for super-computers. Therefore, leveraging neural models to directly predict the future state of the system has recently gained great attention.
3.2 EGNN
Although the existing EGNN methods have great advantage in computational cost over the conventional simulation tools, the potential prediction error that a neural model needs to compensate for is also huge. Take the standard EGNN [18] as an example, the equivariant graph convolution can be written as follows:
$$
\mathbf {m} _ {i j} ^ {0} = \phi_ {e} \left(\mathbf {h} _ {i}, \mathbf {h} _ {j}, \left\| \mathbf {x} _ {i} ^ {0} - \mathbf {x} _ {j} ^ {0} \right\| ^ {2}, e _ {i j}\right), \tag {2}
$$
where the aggregation function  $\phi_{e}$  takes three types of information as input: the feature embeddings  $\mathbf{h}_i$  and  $\mathbf{h}_j$  for the input particle  $p_i$  and an arbitrary particle  $p_j$ , respectively; the Euclidean distance  $\| \mathbf{x}_i^0 - \mathbf{x}_j^0 \|^2$  at time  $t = 0$ ; and the edge attribute  $e_{ij}$ . Then, the predicted velocity and future coordinate can be calculated by the following equation:
$$
\hat {\mathbf {v}} _ {i} ^ {0} = \phi_ {v} (\mathbf {h} _ {i}) \mathbf {v} _ {i} ^ {0} + \frac {1}{N - 1} \sum_ {j \neq i} \left(\mathbf {x} _ {i} ^ {0} - \mathbf {x} _ {j} ^ {0}\right) \mathbf {m} _ {i j} ^ {0}, \tag {3}
$$
$$
\hat {\mathbf {x}} _ {i} ^ {T} = \mathbf {x} _ {i} ^ {0} + \hat {\mathbf {v}} _ {i} ^ {0} T, \tag {4}
$$
where  $\phi_v: \mathbb{R}^D \to \mathbb{R}^1$  is a learnable function. From the above equations, we can find that the predicted velocity  $\hat{\mathbf{v}}_i^0$  is also correlated with three types of information: 1) the initial velocity  $\mathbf{v}_i^0$  as a basis; 2) the pair-wise Euclidean distance  $||\mathbf{x}_i^0 - \mathbf{x}_j^0||^2$  (i.e.,  $\mathbf{m}_{ij}^0$ ) to determine the amount of force (analogous to Coulomb's law); and the pair-wise relative coordinate difference  $(\mathbf{x}_i^0 - \mathbf{x}_j^0)$  to determine the direction. For multi-layer EGNN and other EGNN models,  $\hat{\mathbf{v}}_i^0$  is still determined by these three types of information, which we refer interested readers to Appendix A for details.
Proposition 3.1 (Linear mapping). The existing methods learn a linear mapping from the initial velocity  $\mathbf{v}^0$  to the average velocity  $\mathbf{v}^{t^*}$  over the interval  $[0,T]$ .
Proof. Please see Appendix B.1
As the model makes prediction only based on the state and velocity at  $t = 0$ , we assume that  $\mathbf{v}^{t^*}$  fluctuates around  $\mathbf{v}^0$  and follows a normal distribution denoted by  $\mathcal{N}_{NC(0)} = (\mathbf{v}^0, \sigma_{NC(0)}^2)$ . Then, the variance term  $\sigma_{NC(0)}^2$  reflects how difficult to train a neural model on data sampled from this distribution. In other words, the larger the variance is, the worse the expected results are.
Proposition 3.2 (Variance of  $\mathcal{N}_{NC(0)}$ ). Assume that the average velocity  $\mathbf{v}^{t^*}$  over  $[0, T]$  follows a normal distribution with  $\mu = \mathbf{v}^0$ , then the variance of the distribution is:
$$
\sigma_ {N C (0)} ^ {2} = \mathcal {O} \left(T ^ {2}\right) \tag {5}
$$
Proof. According to the definition,  $\sigma_{NC(0)}^2$  can be written as follows:
$$
\begin{array}{l} \sigma_ {N C (0)} ^ {2} = \frac {\sum_ {p} \left(\mathbf {v} ^ {t ^ {*}} - \mathbf {v} ^ {0}\right) ^ {2}}{M} = \frac {\sum_ {p} \left(\left(\mathbf {v} ^ {t ^ {*}} T - \mathbf {v} ^ {0} T\right) / T\right) ^ {2}}{M} (6) \\ = \frac {\sum_ {p} \left(\left(\mathbf {x} ^ {T} - \mathbf {x} ^ {0}\right) - \mathbf {v} ^ {0} T\right) ^ {2}}{M T ^ {2}} = \frac {\sum_ {p} \left(\int_ {0} ^ {T} (\mathbf {v} (t) - \mathbf {v} ^ {0} T) d t\right) ^ {2}}{M T ^ {2}}, (7) \\ \end{array}
$$
where  $M \gg N$  is the number of all samples. The term  $(\mathbf{v}(t) - \mathbf{v}^0 T)$  is a typical (degree 0) polynomial interpolation error and can be defined as:
$$
\begin{array}{l} \epsilon_ {\mathrm {I P} (0)} (t) = \mathbf {v} (t) - \mathbf {v} ^ {0} T = \mathbf {v} (t) - P _ {k} (t) | k = 0 (8) \\ = \frac {\mathbf {v} ^ {(k + 1)} (\xi)}{(k + 1) !} (t - t _ {0}) (t - t _ {1}) \dots (t - t _ {k}) | k = 0 = \mathbf {v} (\xi) t, (9) \\ \end{array}
$$
where  $0 \leq \xi \leq T$ , and the corresponding integration error is:
$$
\epsilon_ {\mathrm {N C} (0)} = \int_ {0} ^ {T} \epsilon_ {\mathrm {I P} (0)} (t) d t = \mathbf {v} (\xi) \int_ {0} ^ {T} t d t = \frac {1}{2} \mathbf {v} (\xi) T ^ {2} = \mathcal {O} \left(T ^ {2}\right). \tag {10}
$$
Therefore, we obtain the final variance:
$$
\sigma_ {N C (0)} ^ {2} = \frac {M \mathcal {O} \left(T ^ {4}\right)}{M T ^ {2}} = \mathcal {O} \left(T ^ {2}\right), \tag {11}
$$
3.3 NC (k)
In comparison with the 0 degree polynomial approximation  $\mathrm{NC}(0)$ , the high order  $\mathrm{NC}(k)$  is more accurate and yields only a little extra computational cost.
Suppose that we now have  $K + 1$  (usually  $K \leq 8$  due to the catastrophic Runge's phenomenon [42]) points  $\mathbf{x}_i^0, \mathbf{x}_i^1, \dots, \mathbf{x}_i^K$  equally spaced on time interval  $[0, T]$ , then the prediction for the future coordinate  $\hat{\mathbf{x}}_i^T$  in Equation (4) can be re-written as:
$$
\hat {\mathbf {x}} _ {i} ^ {T} = \mathbf {x} _ {i} ^ {0} + \frac {T}{K} \sum_ {k = 0} ^ {K} w ^ {k} \hat {\mathbf {v}} _ {i} ^ {k}, \tag {12}
$$
where  $\{w^k\}$  is the coefficients of Newton-Cotes formulas and  $\hat{\mathbf{v}}_i^k$  denotes the predicted velocity at time  $t^k$ . To obtain  $\hat{\mathbf{v}}_i^k$ ,  $k \geq 1$ , we simply regard EGNN as a recurrent model [43, 44] and re-input the last output velocity and coordinate. Some popular sequence models like LSTM [45] or Transformer [13] may be also leveraged, which we leave to future work. Then, the equation of updating the predicted velocity  $\hat{\mathbf{v}}_i^k$  at  $t^k$  can be written as:
$$
\hat {\mathbf {v}} _ {i} ^ {k} = \phi_ {v} \left(\mathbf {h} _ {i}\right) \mathbf {v} _ {i} ^ {k - 1} + \frac {\sum_ {j \neq i} \left(\mathbf {x} _ {i} ^ {k - 1} - \mathbf {x} _ {j} ^ {k - 1}\right) \mathbf {m} _ {i j}}{N - 1}, \tag {13}
$$
where  $\mathbf{v}_i^{k-1}, \mathbf{x}_i^{k-1}$  denote the velocity and coordinate at  $t^{k-1}$ , respectively. Note that, Equation (13) is different from the form used in a multi-layer EGNN. The latter always uses the same input velocity and coordinate as constant features in different layers. By contrast, we first predict the next velocity and coordinate and then use them as input to get the new trainable velocity and coordinate. This process is more like training a language model [13, 43, 46, 45, 47].
From the interpolation theorem [26, 48], there exists a unique polynomial  $\sum_{k=0}^{K} C^k t^{(k)}$  of degree at most  $K$  that interpolates the  $K+1$  points, where  $C^k$  is the coefficients of the polynomial. To avoid ambiguity, we here use  $t^{(k)}$  to denote  $t$  raised to the power of  $k$ , whereas  $t^k$  to denote the  $k$ -th time point.  $\sum_{k=0}^{K} C^k t^{(k)}$  thus can be constructed by the following equation:
$$
\sum_ {k} ^ {K} C ^ {k} t ^ {(k)} = [ \mathbf {v} (t ^ {0}) ] + [ \mathbf {v} (t ^ {0}), \mathbf {v} (t ^ {1}) ] (t - t ^ {0}) + \dots + [ \mathbf {v} (t ^ {0}), \dots , \mathbf {v} (t ^ {K}) ] (t - t ^ {0}) (t - t ^ {1}) \dots (t - t ^ {K}), \tag {14}
$$
where  $[\cdot ]$  denotes the divided difference. In our case, the  $K + 1$  points are equally spaced over the interval  $[0,T]$ . According to Newton-Cotes formulas, we will have:
$$
\int_ {0} ^ {T} \mathbf {v} (t) d t \approx \int_ {0} ^ {T} \sum_ {k = 0} ^ {K} C _ {k} t ^ {(k)} d t = \frac {T}{K} \sum_ {k = 0} ^ {K} w ^ {k} \mathbf {v} ^ {k}. \tag {15}
$$
Particularly,  $\{w^k\}$  are Newton-Cotes coefficients that are irrelevant to  $\mathbf{v}(t)$ . With additional observable points, the estimation for the integration can be much more accurate. The objective thus can be written as follows:
$$
\underset {\{\hat {\mathbf {v}} ^ {k} \}} {\operatorname {a r g m i n}} \sum_ {p} \mathbf {x} ^ {T} - \hat {\mathbf {x}} ^ {T} = \sum_ {p} \mathbf {x} ^ {T} - \mathbf {x} ^ {0} - \frac {T}{K} \sum_ {k = 0} ^ {K} w ^ {k} \hat {\mathbf {v}} ^ {k}. \tag {16}
$$
Proposition 3.3 (Variance of  $\sigma_{NC(k)}^2$ ).  $\forall k \in \{0,1,\ldots\}$ ,  $\epsilon_{NC(k)} \geq \epsilon_{NC(k+1)}$ , and consequently:
$$
\sigma_ {N C (0)} ^ {2} \geq \sigma_ {N C (1)} ^ {2} \geq \dots \geq \sigma_ {N C (k)} ^ {2}. \tag {17}
$$
Proof. Please see Appendix B.2 for details. Briefly, we only need to prove that  $\epsilon_{\mathrm{NC}(0)} \geq \epsilon_{\mathrm{NC}(1)}$  where NC (1) is associated with Trapezoidal rule.
Proposition 3.4 (Equivariance of NC). NC possesses the equivariance property if the backbone model  $\mathcal{M}$  (e.g., EGNN) in NC possesses this property.
Proof. Please see Appendix B.3.
Algorithm 1 Newton-Cotes Graph Neural Network
1: Input: the dataset  $\mathcal{D}$ , the backbone model  $\mathcal{M}$ , number of steps  $k$  for NC;  
2: repeat  
3: for each batched data in the training set  $(\{X^0, \dots, X^k\}, \{V^0, \dots, V^k\})$  do  
4:  $\hat{X}^0 \gets X^0$ ,  $\hat{V}^0 \gets V^0$ ;  
5: for  $i := 1$  to  $k$  do  
6:  $\hat{X}^i, \hat{V}^i \gets \mathcal{M}(\hat{X}^{i-1}, \hat{V}^{i-1})$ ;  
7: end for  
8: Compute the final prediction  $\hat{X}^k$  by Equation 12;  
9: Compute the main prediction loss  $\mathcal{L}_{main}$  according to Equation (16);  
10: Compute the velocity regularization loss  $\mathcal{L}_r$  according to Equation (18);  
11: Minimize  $\mathcal{L}_{main}, \mathcal{L}_r$ ;  
12: end for  
13: until the loss on the validation set converges.
3.4 NC (k) and  $\mathbf{NC}^{+}$  (k)
In the previous formulation, in addition to the initial and terminal points, we also need the other  $K - 1$  points for supervision which yields a regularization loss:
$$
\mathcal {L} _ {r} = \sum_ {p} \sum_ {k = 0} ^ {K} \| \mathbf {v} ^ {k} - \hat {\mathbf {v}} ^ {k} \|. \tag {18}
$$
We denote NC (k) with the above loss by  $\mathrm{NC}^+$  (k). Minimizing  $\mathcal{L}_r$  is equivalent to minimizing the difference between the true velocity and predicted velocity at each time point  $t^k$  for each particle.  $\|\cdot\|$  can be arbitrary distance measure, e.g., L2 distance. Although the  $K-1$  points of data can be easily accessed in practice, we argue that a loose version (without  $\mathcal{L}_r$ ) NC (k) can also learn promising estimation of the integration  $\int_0^T \mathbf{v}(t) dt$ . Specifically, we still use  $K+1$  points  $\{\hat{\mathbf{v}}^k\}$  to calculate the integration over  $[0, T]$ , except that the intermediate  $K-1$  points are unbounded. The model itself needs to determine the proper values of  $\{\hat{\mathbf{v}}^k\}$  to optimize the same objective defined in Equation 16. In Section 4.5, we design an experiment to investigate the difference between the true velocities  $\{\mathbf{v}^k\}$  and the predicted velocities  $\{\hat{\mathbf{v}}^k\}$ .
3.5 Computational Complexity and Limitations
In comparison with EGNN, NC (k) does not involve additional neural layers or parameters. Intuitively, we recurrently use the last output of EGNN to produce new predicted velocities at next time point, the corresponding computational cost should be  $K$  times that of the original EGNN. However, our experiment in Section 4.4 shows that the training time did not actually linearly increase w.r.t.  $K$ . One reason may be the gradient in back-propagation is still computed only once rather than  $K$  times.
We present an implementation of  $\mathrm{NC}^+$  in Algorithm 1. We first initialize all variables in the model and then recurrently input the last output to the backbone model to collect a series of predicted velocities. We then calculate the final predicted integration and plus the initial coordinate as the final output (i.e., Equation (12)). Finally we compute the losses and update the model by back-propagation.
We conducted experiments on three different tasks towards reasoning system dynamics. The source code and datasets are available at https://github.com/zjukg/NCGNN.
We selected three state-of-the-art methods as our backbone models: RF [17], EGNN [18], and GMN [19]. To ensure a fair comparison, we followed their best parameter settings (e.g., hidden size and the number of layers) to construct NC. In other words, the main settings of NC and the baselines