SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The key idea of Theorem 6 is the following: First, by Lemma 27 in our appendix, the combination of ALL  $\mathcal{FOC}_2$  logical classifiers with quantifier depth no more than  $L$  can already distinguish accepting and rejecting instances of  $c$ . Then by Proposition 26 (This is a key point of this bound; please refer to our appendix), We know the number of intrinsically different bounded-depth  $\mathcal{FOC}_2$  classifiers is finite, so we only need to get an upper bound on this number. Finally, we can get the desired bound by iteratively using the fact that a boolean combination of a set of formulas can be always written as DNF (disjunctive normal form). The tower of power of two comes from  $L$  rounds of DNF enumerations. Although the bound seems scary, it is a rather loose bound. We give a detailed proof of Theorem 6 in the appendix along with the proof of Theorem 5.
5  $\mathbf{R}^2$ -GNN capture  $\mathcal{FOC}_2$  over transformed multi-relational graphs
As we pointed out in the previous section, one of the reasons why  $\mathbf{R}^2$ -GNN cannot capture  $\mathcal{FOC}_2$  classifiers over arbitrary universal graph classes is that in multi-relational graphs, they cannot distinguish whether information about having a neighbour connected via a particular relation comes from the same neighbour node or different neighbour nodes. Towards solving this problem, we propose a graph transformation  $F$  (see Definition 7), which enables  $\mathbf{R}^2$ -GNN to capture all  $\mathcal{FOC}_2$  classifiers on multi-relational graphs. Similar transformation operations have also been used and proved to be an effective way to encode multi-relational graphs in previous studies, e.g., MGNNs Tena Cucala et al. [2021], Indigo Liu et al. [2021] and Time-then-Graph Gao and Ribeiro [2022].
Definition 7. Given a multigraph  $G = (V, \mathcal{E}, P_1, P_2)$ , the transformation  $F$  will map  $G$  to another graph  $F(G) = (V', \mathcal{E}', P_1', P_2')$  with changes described as follows:
- for any two nodes  $a, b \in V$ , if there exists at least one relation  $p \in P_2$  between  $a$  and  $b$ , we add two new nodes  $ab$  and  $ba$  to  $V'$ .
- we add a new unary predicate  $\{\mathrm{primal}\}$  and two new binary predicates  $\{\mathrm{aux1,aux2}\}$ . Hence,  $F(P_{1}) := P_{1}^{\prime} = P_{1} \cup \{\mathrm{primal}\}$ , and  $F(P_{2}) := P_{2}^{\prime} = P_{2} \cup \{\mathrm{aux1,aux2}\}$ . For each node  $v^{\prime} \in V^{\prime}$ ,  $\mathrm{primal}(v^{\prime}) = 1$  iff  $v^{\prime}$  is also in  $V$ ; otherwise,  $\mathrm{primal}(v^{\prime}) = 0$ ;
- for each triplet of the form  $(a, p_2, b)$  in  $\mathcal{E}$ , we add to  $\mathcal{E}'$  four new triples:  $(ab, aux1, a)$ ,  $(ba, aux1, b)$  and  $(ab, aux2, ba)$  as well as  $(ab, p_2, ba)$ .
An example is in Figure 2. We can see that after applying the graph transformation, we need to execute two more hops to propagate information from node  $a$  to node  $b$ . However, now we are able to distinguish whether the information about different relations comes from the same node or different nodes. This transformation can be implemented and stored in linear time/space complexity  $O(|V| + |\mathcal{E}|)$ , which is very efficient.
Definition 8. Given a classifier  $\mathcal{C}$  and a transformation function  $F$ , we define  $\mathcal{C} \circ F$  to be a new classifier, an extension of  $\mathcal{C}$  with an additional transformation operation on the input graph.
With graph transformation  $F$ , we get a more powerful class of classifiers than  $\mathbf{R}^2$ -GNN. We analyze the logical expressiveness of  $\mathbf{R}^2$ -GNN  $\circ F$  in multi-relational graphs, which means first transform a graph  $G$  to  $F(G)$  and then run an  $\mathbf{R}^2$ -GNN on  $F(G)$ . We will see in the following that this transformation  $F$  boosts the logical expressiveness of  $\mathbf{R}^2$ -GNN prominently.
Theorem 9.  $R^2$ - $GNN \subseteq R^2$ - $GNN \circ F$  on any universal graph class.
Theorem 10.  $\mathcal{FOC}_2\subseteq R^2\text{-} GNN\circ F$  on any universal graph class.
Theorem 9 demonstrates that  $\mathbf{R}^2$ -GNN with graph transformation  $F$  have more expressiveness than  $\mathbf{R}^2$ -GNN; and Theorem 10 shows the connection between  $\mathcal{FOC}_2$  and  $\mathbf{R}^2$ -GNN equipped with graph transformation  $F$ . We depict their relations in Figure 3. Theorem 9
is a natural result since no information is lost in the process of transformation, while Theorem 10 is an extension on Theorem 4, whose formal proofs can be found in the Appendix. As for the backward direction, we have the result shown in Theorem 11.
Theorem 11.  $R^2$ -GNN  $\circ F \subseteq \mathcal{FOC}_2$  on any bounded graph class.
Figure 3: Relations of  $\mathbf{R}^2$ -GNN,  $\mathcal{FC}_2$  and  $\mathbf{R}^2$ -GNN  $\circ F$ .
The proof of the theorem is relatively straightforward based on previous results: by Theorem 5, it follows that  $\mathbf{R}^2$ -GNN  $\circ F \subseteq \mathcal{FOC}_2 \circ F$  on any bounded graph class. Then, it suffices to prove  $\mathcal{FOC}_2 \circ F \subseteq \mathcal{FOC}_2$ , which we do by using induction over the quantifier depth.
By combining Theorem 10 and Theorem 11, we obtain Corollary 11.1, stating that  $\mathcal{FOC}_2$  and  $\mathbb{R}^2$ -GNN  $\circ F$  have the same expressiveness with respect to bounded graph classes. Corollary 11.1 does not hold for arbitrary universal graph classes, but our finding is nevertheless exciting because, in many real-world applications there are upper bounds over input graph size.
Corollary 11.1.  $R^2$ -GNN  $\circ F = \mathcal{FOC}_2$  on any bounded graph class.
To show the strict separation as in Figure 3, we can combine Proposition 3 and theorems 4 and 9 and Theorem 10 to directly get the following:
Corollary 11.2.  $R^2$ - $GNN \subsetneq R^2$ - $GNN \circ F$  on some universal graph class, and  $\mathcal{FOC}_2 \subsetneq R^2$ - $GNN \circ F$  on some simple graph class.
One may think after transformation  $F$ , the logic  $\mathcal{FOC}_2 \circ F$  with new predicates becomes stronger as well. However by a similar proof as for Theorem 10 and Lemma 28, we can actually show  $\mathcal{FOC}_2 \circ F \subseteq \mathcal{FOC}_2$  always holds, so  $F$  won't bring added power for  $\mathcal{FOC}_2$ . However, it indeed makes  $\mathbb{R}^2$ -GNN strictly more expressive.
6 Temporal Graphs
As stated in Gao and Ribeiro [2022], a temporal knowledge graph, composed of multiple snapshots, can consistently undergo transformation into an equivalent static representation as a multi-relational graph. Consequently, this signifies that our theoretical results initially devised for multi-relational graphs can be extended to apply to temporal graphs, albeit through a certain manner of transfer.
Following previous work Jin et al. [2019], Pareja et al. [2020], Park et al. [2022], Gao and Ribeiro [2022], we define a temporal knowledge graph as a set of graph "snapshots" distributed over a sequence of finite and discrete time points  $\{1, 2, \dots, T\}$ . Formally, a temporal knowledge graph is a set  $G = \{G_1, \dots, G_T\}$  for some  $T \in \mathbb{N}$ , where each  $G_t$  is a static multi-relational graph. All these  $G_t$  share the same node set and predicate set.
In a temporal knowledge graph, a relation or unary fact between two nodes might hold or disappear across the given timestamps. For example, a node  $a$  may be connected to a node  $b$  via a relation  $p$  in the first snapshot, but not in the second; in this case, we have  $(a,p,b)$  in  $G_{1}$  not in  $G_{2}$ . To keep track of which relations hold at which snapshots, we propose temporal predicates, an operation which we define in Definition 12.
Figure 4: Hierarchic expressiveness.
Definition 12. Given a temporal graph  $G = \{G_1, \dots, G_T\}$ , where each  $G_t$  is of the form  $(V_t, \mathcal{E}_t, P_1, P_2)$ , temporal predicates are obtained from  $G$  by replacing, for each  $t \in \{1, \dots, T\}$  and each  $p \in P_2$ , each triple  $(v_a, p, v_b) \in \mathcal{E}_t$  with  $(v_a, p^t, v_b)$ , where  $p^t$  is a fresh predicate, unique for  $p$  and  $t$ . Similarly, each unary fact  $(v_a, q) \in \mathcal{E}_t$ ,  $q \in P_1$  should be replaced by  $(v_a, q^t)$ .
Note that temporalising introduces  $T \times |P|$  new predicates in total. By temporalizing predicates, we assign a superscript to each predicate and use it to distinguish relations over different timestamps.
Definition 13. Given a temporal knowledge graph  $G = \{G_1, \dots, G_T\}$ , the collapse function  $H$  maps  $G$  to the static graph  $H(G)$  obtained by taking the union of graphs over all timestamps in the temporalization of  $G$ .
As we have proved in Section 5, for multi-relational graphs,  $\mathbf{R}^2$ -GNN with graph transformation is more powerful than the pure  $\mathbf{R}^2$ -GNN. Here, we transfer these theoretical findings in multi-relational graphs to the setting of temporal knowledge graphs. To be more specific, after temporalizing predicates, we apply a graph transformation to each graph snapshot.
Definition 14. We define  $F^T$  to be the temporal graph transformation that takes any temporal knowledge graph as input, applies graph transformation to each snapshot and outputs. Specially, non-primal nodes, aux1 and aux2 edges added in any snapshot should be added into all snapshots.
$\mathbf{R}^2$ -TGNN Gao and Ribeiro [2022] casts node representation in temporal graphs into two frameworks: time-and-graph and time-then-graph. Due to space constraints, we refer interested readers to Gao and Ribeiro [2022] for more details about the two frameworks. Here, we define a more general GNN-based framework abbreviated as  $\mathbf{R}^2$ -TGNN, where each  $\mathbf{R}^2$ -TGNN is a sequence  $\{\mathcal{A}_t\}_{t=1}^T$ , where each  $\mathcal{A}_t$  is an  $\mathbf{R}^2$ -GNN model. Given a temporal knowledge graph  $G = \{G_1, \ldots, G_T\}$ , where  $G_t = (V_t, \mathcal{E}_t, P_1, P_2)$  for each  $t \in \{1, \ldots, T\}$ . The updating rule is as follows:
$$
\mathbf {x} _ {v} ^ {t} = \mathcal {A} _ {t} \left(G _ {t}, v, \mathbf {y} ^ {t}\right) \quad \text {w h e r e} \quad \mathbf {y} _ {v} ^ {t} = \left[ I _ {G _ {t}} (v): \mathbf {x} _ {v} ^ {t - 1} \right], \forall v \in V \left(G _ {t}\right) \tag {4}
$$
where  $I_{G_t}(v)$  is the one-hot initial feature vector of node  $v$  at timestamp  $t$ , and  $\mathcal{A}_t(G_t, v, \mathbf{y}^t)$  calculates the new feature vector of  $v$  by running the  $\mathbb{R}^2$ -GNN model  $\mathcal{A}_t$  on  $G_t$ , but using  $\mathbf{y}^t$  as the initial feature vectors. As shown in Theorem 15,  $\mathbb{R}^2$ -TGNN composed with  $F^T$  have the same expressiveness as time-then-graph<sup>4</sup>, while being more powerful than time-and-graph.
Theorem 15. time-and-graph  $\subsetneq R^2$ -TGNN  $\circ F^T =$  time-then-graph.
We also establish the validity of Theorem 16, which asserts that  $\mathbb{R}^2$ -TGNN with graph transformation maintains the same expressive power, whether it is applied directly to the temporal graph or to the equivalent collapsed static multi-relational graph
Theorem 16.  $R^2$ -TGNN  $\circ F^T = R^2$ -GNN  $\circ F \circ H$
We also prove a strict inclusion that  $\mathbf{R}^2$ -TGNN  $\subsetneq$ $\mathbf{R}^2$ -TGNN  $\circ H$ . Finally we get the following hierarchy of these frameworks as in Figure 6. The proof of Theorem 17 is in the appendix.
Theorem 17. The following hold:
-  $R^2$ -GNN  $\subsetneq$ $R^2$ -GNN  $\circ H \subsetneq R^2$ -TGNN  $\circ F \circ H = R^2$ -TGNN  $\circ F^T = \text{time - then - graph.}$
time-and-graph  $\subsetneq R^2$  -TGNN  $\circ F^T$
-  $R^2$ -TGNN  $\nsubseteq$  time-and-graph.
<sup>4</sup>Since temporalized predicates and timestamps make the definitions of bounded/simple/universal graph class vague, we no longer distinguish them in temporal settings. In theorem statements of this section,  $= ,\subseteq$  always hold for any temporal graph class, and  $\subsetneq ,\nsubseteq$  hold for some temporal graph class
FCO2 classifier Aggregation	φ1 sum	max	mean	φ2 sum	max	mean	φ3 sum	max	mean	φ4 sum	max	mean
Temporal Graphs Setting
R-TGNN	100	60.7	65.4	61.0	51.3	52.4	93.7	82.3	84.4	83.5	60.0	61.3
R2-TGNN	100	63.5	66.8	93.1	57.7	60.2	94.5	83.3	85.9	85.0	62.3	66.2
R2-TGNN oFT	100	67.2	68.1	99.0	57.6	62.2	100	88.8	89.2	98.1	73.4	77.5
Aggregated Static Graphs Setting
R-GNN oH	100	61.2	69.9	62.3	51.3	55.5	94.7	80.5	83.2	80.2	60.1	60.4
R2-GNN oH	100	62.7	66.8	92.4	56.3	58.5	95.5	84.2	85.2	81.0	58.3	64.5
R2-GNN oF oH	100	70.2	70.8	98.8	60.6	60.2	100	85.6	86.5 7	95.5	70.3	79.7
Table 1: Test set node classification accuracies (%) on synthetic temporal multi-relational graphs datasets and their aggregated static multi-relational graphs datasets. The best results are highlighted for two different settings.
7 Experiment
We empirically verify our theoretical findings for multi-relational graphs by evaluating and comparing the testing performance of  $\mathbf{R}^2$ -GNN with graph transformation and less powerful GNNs (R-GNN and  $\mathbf{R}^2$ -GNN). We did two groups of experiments on synthetic datasets and real-world datasets, respectively. Details for datasets generation and statistical information as well as hyper-parameters can be found in the Appendix.
7.1 Synthetic Datasets
We first define three simple  $\mathcal{FOC}_2$  classifiers
$$
\begin{array}{l} \varphi_ {1} := \exists^ {\geq 2} y \left(p _ {1} ^ {1} (x, y) \wedge R e d ^ {1} (y)\right) \wedge \exists^ {\geq 1} y \left(p _ {1} ^ {2} (x, y) \wedge B l u e ^ {2} (y)\right) \\ \varphi_ {\mathcal {2}} := \exists^ {[ 1 0, 2 0 ]} y (\neg p _ {1} ^ {2} (x, y) \wedge \varphi_ {1} (y)) \quad \varphi_ {\mathcal {3}} := \exists^ {\geq 2} y \left(p _ {1} ^ {1} (x, y) \wedge p _ {1} ^ {2} (x, y)\right) \\ \end{array}
$$
Besides, we define another complicate  $\mathcal{FOC}_2$  classifier denoted as  $\varphi_4$  shown as follows:
$$
\begin{array}{l} \varphi_ {4} := \bigvee_ {3 \leq t \leq 1 0} (\exists^ {\geq 2} y (B l a c k ^ {t} (y) \wedge R e d ^ {t - 1} (y) \wedge B l u e ^ {t - 2} (y) \wedge p _ {1} ^ {t} (x, y) \wedge p _ {2} ^ {t - 1} (x, y) \wedge p _ {3} ^ {t - 2} (x, y) \wedge \varphi^ {t} (y)) \\ \text {w h e r e} \quad \varphi^ {t} (y) := \exists^ {\geq 2} x \left(p _ {1} ^ {t} (x, y) \wedge R e d ^ {t} (x)\right) \wedge \exists^ {\geq 1} x \left(p _ {2} ^ {t - 1} (x, y) \wedge B l u e ^ {t - 2} (x)\right) \\ \end{array}
$$
For each of them, we generate an independent dataset containing 7k multi-relational graphs of size up to 50-1000 nodes for training and 500 multi-relational graphs of size similar to the train set. We tried different configurations for the aggregation functions and evaluated the node classification performances of three temporal GNN methods (R-TGNNs,  $\mathbf{R}^2$ -TGNNs and  $\mathbf{R}^2$ -TGNNs  $\circ F^T$ ) on these datasets.
We verify our hypothesis empirically according to models' actual performances of fitting these three classifiers. Theoretically,  $\varphi_{1}$  should be captured by all three models because the classification result of a node is decided by the information of its neighbor nodes, which can be accomplished by the general neighborhood based aggregation mechanism.  $\varphi_{2}$  should not be captured by R-TGNN because the use of  $\neg p_1^2 (x,y)$  as a guard means that the classification result of a node depends on the global information including those isolated nodes, which needs a global readout. For  $\varphi_{3}$  and  $\varphi_{4}$ , they should only be captured by  $\mathbb{R}^2$ -TGNNs  $\circ F^T$ . An intuitive explanation for this argument is that if we temporalise predicates and then collapse the temporal graph into its equivalent static multi-relational graph using  $H$ , we will encounter the same issue as in the Figure 1. Thus we can't distinguish expected nodes without graph transformation.
Results for temporal GNN methods and static GNN methods on four synthetic datasets can be found in Table 1. We can see that  $\mathbf{R}^2$ -GNN with graph transformation achieves the best performance. Our theoretical findings show that it is a more expressive model, and the experiments indeed suggest that the model can exploit this theoretical expressiveness advantage to produce better results. Besides, we can also see that  $\mathbf{R}^2$ -TGNN  $\circ F$ $\circ H$  and  $\mathbf{R}^2$ -TGNN  $\circ F^T$  achieve almost the same performance, which is in line with Theorem 16.
Table 2: Results on temporal graphs.
Models	Category	Source	Brain-10
sum	max	mean
GCRN-M2	time-and-graph	Seo et al. [2018]	77.0	61.2	73.1
DCRNN	time-and-graph	Li et al. [2018]	84.0	70.1	66.5
TGAT	time-then-graph	Xu et al. [2020]	80.0	72.3	79.0
TGN	time-then-graph	Rossi et al. [2020a]	91.2	88.5	89.2
GRU-GCN	time-then-graph	Gao and Ribeiro [2022]	91.6	88.2	87.1
R-TGNN	-	-	85.0	82.3	82.8
R²-TGNN	-	-	94.8	82.3	91.0
R²-TGNN ∘FT	-	-	94.0	83.5	92.5
7.2 Real-world Datasets
For real-world static multi-relational graphs benchmarks, we used AIFB and MUTAG from Ristoski and Paulheim [2016]. Since open source datasets for the node classification on temporal knowledge graphs are rare, we only tried one dataset Brain-10 Gao and Ribeiro [2022] for temporal settings.
Table 3: Results on two static multi-relational graphs.
Models	sum	AIFB max	mean	sum	MUTAG max	mean
R-GNN	91.7	73.8	82.5	76.5	63.3	73.2
R2-GNN	91.7	73.8	82.5	85.3	62.1	79.5
R2-GNN oF	97.2	75.0	89.2	88.2	65.5	82.1
R-GCN	95.8	77.9	86.3	73.2	65.7	72.1
For static multi-relational graphs, we compare the performances of
our methods with RGCN Schlichtkrull et al. [2018]. Note that RGCN assigns each node an index and the initial embedding of each node is initialised based on the node index, so the initialisation functional is not permutation-equivariant Chen et al. [2019a] and RGCN cannot be used to perform an isomorphism test. However, from Table 3, we can see that  $\mathbb{R}^2$ -GNN with graph transformation still achieves the highest accuracy while being able to be used for the graph isomorphism test. Besides,  $\mathbb{R}^2$ -GNN  $\circ F$  also performs better compared with both R-GNN and  $\mathbb{R}^2$ -GNN. This again suggests that the extra expressive power gained by adding a graph transformation step to  $\mathbb{R}^2$ -GNN can be exploited by the model to obtain better results.
For temporal graphs, Gao and Ribeiro [2022] have classified existing temporal models into two categories, time-and-graph and time-then-graph, and shown that time-then-graph models have better performance. We choose five models mentioned in Gao and Ribeiro [2022] as our baseline and include the best accuracy of the dataset Brain-10 reported in Gao and Ribeiro [2022]. As we expected,  $\mathbf{R}^2$ -TGNN and  $\mathbf{R}^2$ -TGNN  $\circ F^T$  achieve better performance than that of the baseline models and R-TGNN according to Table 2. However, we observed that although in theory,  $\mathbf{R}^2$ -TGNN  $\circ F^T$  has stronger expressive power than  $\mathbf{R}^2$ -TGNN, we did not see an improvement when using  $\mathbf{R}^2$ -TGNN  $\circ F^T$  (0.8% accuracy drop). To some extent, it may show that some commonly used benchmarks are inadequate for testing advanced GNN variants. Similar phenomena have also been observed in previous works Chen et al. [2019b], Barceló et al. [2020].
We analyze expressivity of  $\mathbf{R}^2$ -GNNs with and without graph transformation in multi-relational graphs under different situations. Furthermore, we extend our theoretical findings to the temporal graph setting. Our experimental results confirm our theoretical insights, particularly demonstrating the state-of-the-art performance achieved by our graph transformation technique.
5The other three temporal dataset mentioned in Gao and Ribeiro [2022] are not released.
The authors extend their gratitude to Bernardo Cuenca Grau and David Tena Cucala for their valuable insights, stimulating discussions, and support.
Namyong Park, Andrey Kan, Xin Luna Dong, Tong Zhao, and Christos Faloutsos. Estimating node importance in knowledge graphs using graph neural networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 596-606, 2019.
David Jaime Tena Cucala, Bernardo Cuenca Grau, Egor V Kostylev, and Boris Motik. Explainable gnn-based models over knowledge graphs. In International Conference on Learning Representations, 2021.
Dingmin Wang, Yeyuan Chen, and Bernardo Cuenca Grau. Efficient embeddings of logical variables for query answering over incomplete knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 4652-4659, 2023.
Zhongkai Hao, Chengqiang Lu, Zhenya Huang, Hao Wang, Zheyuan Hu, Qi Liu, Enhong Chen, and Cheekong Lee. Asgn: An active semi-supervised graph neural network for molecular property prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 731-752, 2020.
Johannes Gasteiger, Florian Becker, and Stephan Gunnemann. Gemnet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, pages 6790-6802, 2021.
Zhichun Guo, Chuxu Zhang, Wenhao Yu, John Herr, Olaf Wiest, Meng Jiang, and Nitesh V Chawla. Few-shot graph learning for molecular property prediction. In Proceedings of the Web Conference 2021, pages 2559-2567, 2021.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Lio, and Petar Velivckovic. Principal neighbourhood aggregation for graph nets. In NeurIPS, 2020.
Floris Geerts and Juan L Reutter. Expressiveness and approximation properties of graph neural networks. arXiv preprint arXiv:2204.04661, 2022.
Pablo Barceló, Egor V. Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=r1lZ7AEKvB.
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593-607. Springer, 2018.
Namyong Park, Fuchen Liu, Purvanshi Mehta, Dana Cristofor, Christos Faloutsos, and Yuxiao Dong. Evokg: Jointly modeling event time and network structure for reasoning over temporal knowledge graphs. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 794-803, 2022.
Youngjoo Seo, Michael Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence modeling with graph convolutional recurrent networks, 2016. URL https://arxiv.org/abs/1612.07659.
Jinyin Chen, Xueke Wang, and Xuanheng Xu. Gc-lstm: Graph convolution embedded LSTM for dynamic link prediction, 2018. URL https://arxiv.org/abs/1812.04206.
Franco Manessi, Alessandro Rozza, and Mario Manzo. Dynamic graph convolutional networks. Pattern Recognition, 97:107000, jan 2020. doi: 10.1016/j.patcog.2019.107000. URL https://doi.org/10.1016%2Fj.patcog.2019.107000.
Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dynamic graph representation learning via self-attention networks, 2018. URL https://arxiv.org/abs/1812.09430.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs, 2020b. URL https://arxiv.org/abs/2006.10637.
Hava T. Siegelmann and Eduardo D. Sontag. On the computational power of neural nets. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT '92, page 440-449, New York, NY, USA, 1992. Association for Computing Machinery. ISBN 089791497X. doi: 10.1145/130385.130432. URL https://doi.org/10.1145/130385.130432.