SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Limitation 2: The temporal embedding could improve. In this paper, the embedding of the timestamp is randomly initialized and finally learned by the model via logical advantages. Such embedding ignores the order of different timestamps. The order property is learned by the After and Before operators, which may be not enough. We recall that in the field of NLP, positional embeddings also have order features, which may be used for the construction of timestamp embeddings.
Limitation 3: The query generation is time-consuming. There are 40 predefined query structures in our query generation module. Each query structure has  $10\mathrm{k}+$  queries for training. With the scale of TKGs increasing, the number of queries will also increase, even  $40\mathrm{x}$  faster. Therefore, we need to find a more efficient way to generate queries for large scale TKGs.
**Limitation 4:** The MRR and Hits@k are weak. The MRR and Hits@k evaluation metric may not inflect the performance of multi-hop reasoning. We observe that some queries have lots of answers. When the number of answers is larger than k, the MRR and Hits@k will be low, even if all answers are correctly ranked at top. Because the right answers that ranked after k are labeled as false. The Hits@k decreases with the increase number of right answers, which is not reasonable. This disadvantage prevents us from comparing the performance across different datasets. In this paper, GDELT is much denser, and the count of right answers is larger than the other two datasets. Therefore, the MRR of GDELT is lower than the other two datasets. It can also be verified that on the average MRR, group avg $_t$  is lower than group avg $_e$ . The reason is that the number of right answers of group avg $_t$  is larger than that of group avg $_e$ , which can be seen from the statistic of average answer count of queries in Table 10. In the future, there should be a more reasonable evaluation metric for multi-hop reasoning.
A More Details about Neural Operators
A.1 Fuzzy Logic
Fuzzy logic is a generalization of Boolean logic, which is based on the idea that the truth of a statement can be expressed as a number between 0 and 1. It is a mathematical framework for reasoning with imprecise information. In fuzzy logic, there are also logical operators, such as conjunction, disjunction, negation, implication, equivalence, and exclusive or. Most popular fuzzy logic operators are defined as follows ( $a, b \in [0,1]$ ):
(1) conjunction:  $a \wedge b = \min(a, b)$ . Disjunction:  $a \vee b = \max(a, b)$ .
(2) negation:  $\neg a = 1 - a$ . exclusive or:  $a \oplus b = \min(a, 1 - b) + \min(1 - a, b)$ .
(3) implication:  $a \to b = \max (1 - a, b)$ . equivalence:  $a \leftrightarrow b = \min (a, b) + \max (1 - a, 1 - b)$ .
However, the fuzzy logic operators are defined over real space, not suitable for reasoning over vector space. We expect that the fuzzy logic operators can be executed by matrix computation. Because the embedding space of knowledge graph is a vector space. In order to cope with tensor computation in neural networks, we introduce the fuzzy logic operators over vector space in the following, which is called vector logic.
A.2 Vector Logic
Vector logic is an elementary logical model based on matrix algebra. In vector logic, true values are mapped to the vector, and logical operators are executed by matrix computation.
Truth Value Vector Space  $V_{2}$ . A two-valued vector logic uses two  $d$ -dimensional ( $d \geq 2$ ) column vectors  $\vec{s}$  and  $\vec{n}$  to represent true and false in the classic binary logic. The two vectors  $\vec{s}$  and  $\vec{n}$  are real-valued, normally orthogonal to each other, and normalized vectors, i.e.,  $\| \vec{s} \| = 1$ ,  $\| \vec{n} \| = 1$ . Truth value vector space are generated by  $V_{2} = \{\vec{s}, \vec{n}\}$ , and operations on vectors in truth value space is based on scalar product.
Operators. The basic logical operators are associated with their own matrices by vectors in truth-value vector space. Two common types of operators are monadic and dyadic.
(1) Monadic Operators are functions:  $V_{2} \rightarrow V_{2}$ . Two examples are Identity  $I = \vec{s}\vec{s}^{T} + \vec{n}\vec{n}^{T}$  and Negation  $N = \vec{n}\vec{s}^{T} + \vec{s}\vec{n}^{T}$  such that  $I\vec{s} = \vec{s}, I\vec{n} = \vec{n}, N\vec{n} = \vec{s}, N\vec{s} = \vec{n}$ . Note that the truth tables of  $I$  and  $N$  fulfill the real logical rules of identity and negation in classic boolean logic.
(2) Dyadic operators are functions:  $V_{2} \otimes V_{2} \to V_{2}$ , where  $\otimes$  denotes Kronecker product. Dyadic operators include conjunction  $C$ , disjunction  $D$ , implication IMPL, equivalence  $E$ , exclusive or XOR, etc. For example, the conjunction between two logical propositions  $(p \wedge q)$  is performed by  $C(\vec{u} \otimes \vec{v})$ , where  $C = \vec{s} (\vec{s} \otimes \vec{s})^T + \vec{n} (\vec{s} \otimes \vec{n})^T + \vec{n} (\vec{n} \otimes \vec{s})^T + \vec{n} (\vec{n} \otimes \vec{n})^T$ . It can be verified that  $C(\vec{s} \otimes \vec{s}) = \vec{s}, C(\vec{s} \otimes \vec{n}) = C(\vec{n} \otimes \vec{s}) = C(\vec{n} \otimes \vec{n}) = \vec{n}$ . Dyadic operators which correspond to logical operations in classic binary logic are defined by its formulation to perform logical operations on truth value vectors. Their associated matrices have  $d^{2}$  rows and  $d$  columns.
Many-valued Two-dimensional Logic. Many-valued logic is introduced to include uncertainties in the logic vector. Weighting  $\vec{s}$  and  $\vec{n}$  by probabilities, uncertainties are introduced:  $\vec{f} = \epsilon \vec{s} + \delta \vec{n}$ , where  $\epsilon, \delta \in [0,1]$ ,  $\epsilon + \delta = 1$ . Besides, operations on vectors can be simplified to computation on the scalar of these vectors. For example, given two vectors  $\vec{u} = \alpha \vec{s} + \beta \vec{n}, \vec{v} = \alpha' \vec{s} + \beta' \vec{n}$ , we have:
$$
\operatorname {N O T} (\vec {u}) = N \vec {u} = (1 - \alpha) \vec {s} + \alpha \vec {n}
$$
$$
\operatorname {N O T} (\alpha) = \bar {s} ^ {T} N \vec {u} = 1 - \alpha
$$
$$
\mathrm {O R} (\vec {u}, \vec {v}) = D (\vec {u} \otimes \vec {v})
$$
$$
\mathrm {O R} (\alpha , \alpha^ {\prime}) = \bar {s} ^ {T} D (\vec {u} \otimes \vec {v}) = \alpha + \alpha^ {\prime} - \alpha \alpha^ {\prime}
$$
$$
\mathrm {A N D} (\vec {u}, \vec {v}) = C (\vec {u} \otimes \vec {v})
$$
$$
\operatorname {A N D} \left(\alpha , \alpha^ {\prime}\right) = \bar {s} ^ {T} C (\vec {u} \otimes \vec {v}) = \alpha \alpha^ {\prime} \tag {7}
$$
$$
\operatorname {I M P L} (\vec {u}, \vec {v}) = L (\vec {u} \otimes \vec {v})
$$
$$
\operatorname {I M P L} \left(\alpha , \alpha^ {\prime}\right) = \bar {s} ^ {T} L (\vec {u} \otimes \vec {v}) = 1 - \alpha \left(1 - \alpha^ {\prime}\right)
$$
$$
\operatorname {X O R} (\vec {u}, \vec {v}) = X (\vec {u} \otimes \vec {v})
$$
$$
\mathrm {X O R} (\alpha , \alpha^ {\prime}) = \vec {s} ^ {T} X (\vec {u} \otimes \vec {v}) = \alpha + \alpha^ {\prime} - 2 \alpha \alpha^ {\prime}
$$
A.3 Neural Dyadic Operators
In this section, we show that all dyadic operators are generated from fuzzy logic in our framework. Below we take union operator for example. We define entity union operator  $\mathcal{U}_e$  and time union operator  $\mathcal{U}_t$  according to Alignment Rule 4.2 as follows:
where AND, OR are the conjunction, disjunction operators in fuzzy logic respectively,  $\alpha_{i}$  and  $\beta_{i}$  are attention weights, as designed in intersection operators. In the time part of entity union operator  $\mathcal{U}_e$  and the entity part of time union operator  $\mathcal{U}_t$ , we follows Alignment Rule 4.2 to perform intersection. Besides, please be aware that each operator owns its MLPs and parameters. These operators do not share parameters with each other.
For  $n$  queries to intersection/union, the AND and OR can be written as:
$$
\underset {i = 1} {\overset {n} {\mathbf {A N D}}} \left\{\boldsymbol {q} _ {i, l} \right\} = \prod_ {i = 1} ^ {n} q _ {i, l}
$$
$$
\mathbf {O R} _ {i = 1} ^ {n} \left\{\boldsymbol {q} _ {i, l} \right\} = \sum_ {i = 1} ^ {n} q _ {i, l} - \sum_ {1 \leqslant i <   j \leqslant n} q _ {i, l} q _ {j, l} + \sum_ {1 \leqslant i <   j <   k \leqslant n} q _ {i, l} q _ {j, l} q _ {k, l} + \dots + (- 1) ^ {n - 1} \prod_ {i = 1} ^ {n} q _ {i, l}
$$
The equations come from the definition of AND and OR in vector logic. The cost of OR operator's implementation is not as expensive as it seems to be. In fact, it's  $O(n * d)$  complexity, where  $d$  is the embedding dimension. We provide the proof in next section.
Other dyadic operators can be generated in the same way. Since listing all operators is boring, we provide the last example. Then, the entity implication operator  $\mathcal{L}_e$  and time implication operator  $\mathcal{L}_t$  are defined as follows:
A.4 Theoretical Analysis
To understand why the feature-logic framework works, we have the following proposition, which shows that the designed intersection and union operators obey commutative law of real logical operations. The proofs are presented in next section in Appendix A.5:
Proposition 1. Commutativity: Given Temporal Feature-Logic embedding  $\mathcal{V}_{q_a},\mathcal{V}_{q_b}$  we have  $\mathcal{I}_e(\mathcal{V}_{q_a},\mathcal{V}_{q_b}) = \mathcal{I}_e(\mathcal{V}_{q_b},\mathcal{V}_{q_a})$  and  $\mathcal{U}_e(\mathcal{V}_{q_a},\mathcal{V}_{q_b}) = \mathcal{U}_e(\mathcal{V}_{q_b},\mathcal{V}_{q_a}),\mathcal{I}_t(\mathcal{V}_{q_a},\mathcal{V}_{q_b}) = \mathcal{I}_t(\mathcal{V}_{q_b},\mathcal{V}_{q_a})$  and  $\mathcal{U}_t(\mathcal{V}_{q_a},\mathcal{V}_{q_b}) = \mathcal{U}_t(\mathcal{V}_{q_b},\mathcal{V}_{q_a})$
A.5 Proofs
In this part, we prove that TFLEX has the property of commutativity. Besides, we also show that the computation complexity of OR operation is linear to the number of queries.
Proof. (Proposition 1) Commutativity:
For the intersection operations, as the calculations of  $\mathcal{I}_e$  and  $\mathcal{I}_t$  are identical, here, we only prove that  $\mathcal{I}_e$  complies with commutative law. The entity feature part and the time feature part of the result are computed as a weighted summation of each query's corresponding parts. Since addition is commutative and the attention weights do not concern the order of calculations, both feature parts' calculations are commutative.
Then, we discuss the logic parts. The logic parts only include the AND in fuzzy logic which, essentially, is just the multiplication of each element by the definition provided above. Because multiplication is surely commutative, the calculation of either entity logic part or time logic part is commutative. Thus the intersection operation  $\mathcal{I}_e(\mathcal{I}_t)$  is commutative.
As for  $\mathcal{U}_e$  and  $\mathcal{U}_t$ , their feature parts have the same form of weighted summation as the intersection operations do. Thus, the feature parts of both  $\mathcal{U}_e$  and  $\mathcal{U}_t$  comply to commutative law. Also, the time logic part of  $\mathcal{U}_e$  and the entity logic part of  $\mathcal{U}_t$  solely concern AND operator which has been proved commutative before. The OR operator, by definition, gives the aggregation of different accumulative parts each of which is commutative itself. Also, the multiplications in each of the summations are commutative. Hence, OR operation is invariant to the order of calculations, which finally gives the calculations of entity logic part of  $\mathcal{U}_e$  and the time logic part of  $\mathcal{U}_t$  are commutative. Then we can naturally affirm that  $\mathcal{U}_e$  and  $\mathcal{U}_t$  are commutative as well.
Proof. Additionally, we prove that the cost of OR operator's implementation is not as expensive as it seems to be. The process of computation can be described as follows:
Algorithm 1: OR
Input: queries  $\{q_{1}\dots q_{n}\}$
Output: the union of queries
1: Let  $u = 0$ .
2: For  $q$  In  $\{q_1 \dots q_n\}$
3:  $u = u + q - u*q$
4: EndFor
5: return  $u$
As we implement OR step by step on a series of  $n$  queries, the loop goes n-1 times in total. Assuming the embedding dimension of a query is  $d$ , we can have the cost of OR is  $O(n^{*}d)$ .
B More Details about Experiments
In this section, we show more details about our experiments. Firstly, we introduce the origin datasets in Section B.1. Then, we describe the details on how we define the query structure and how to sample queries in Section B.2. With the generated queries, we construct three datasets and report their statistics in Section B.3. We also show the details of our model implementation in Section B.4 and evaluation protocol in Section B.5. Besides, we show more experimental data for sensitive analysis in Section B.6. Finally, we present the detail metrics of main results in Section B.7.
B.1 Details of Origin Datasets
To construct the reasoning dataset, we use three datasets of temporal KGs that have become standard benchmarks for TKGC: ICEWS14 [37], ICEWS05-15 [37], and GDELT-500 [38]. The two subsets of ICEWS are generated by Boschee et al. [37]: ICEWS14 corresponding to the facts in 2014 and ICEWS05-15 corresponding to the facts between 2005 and 2015. GDELT-500 generated by Leetaru and Schrodt [38] corresponds to the facts from April 1, 2015, to March 31, 2016. The statistics of the dataset are shown in Table 8.
Table 4: Definition of query structures in static query embeddings [2-4]. Operators are defined on entity sets only, including Pe, And, Or, and Not. 'e', 'r', 'n', 'u' denote entity, relation, negation and union, respectively. The bracket ' $)$  denotes a relational projection or intersection operation.
Query Name	Query Structure Definition
1p	('e', ('r',))
2p	('e', ('r', 'r'))
3p	('e', ('r', 'r', 'r'))
2i	(( 'e', ('r',)), ('e', ('r',)))
3i	(( 'e', ('r',)), ('e', ('r',)), ('e', ('r',)))
ip	((( 'e', ('r',)), ('e', ('r',))), ('r',))
pi	(( 'e', ('r', 'r'), ('e', ('r',)))
2in	(( 'e', ('r',)), ('e', ('r', 'n')))
3in	(( 'e', ('r',)), ('e', ('r',)), ('e', ('r', 'n')))
inp	((( 'e', ('r',)), ('e', ('r', 'n')), ('r',))
pin	(( 'e', ('r', 'r'), ('e', ('r', 'n')))
pni	(( 'e', ('r', 'r', 'n')), ('e', ('r',)))
2u-DNF	(( 'e', ('r',)), ('e', ('r',)), ('u',))
up-DNF	((( 'e', ('r',)), ('e', ('r',)), ('u',)), ('r',))
2u-DM	((( 'e', ('r', 'n')), ('e', ('r', 'n')), ('n',))
up-DM	((( 'e', ('r', 'n')), ('e', ('r', 'n')), ('n', 'r'))
Table 5: Basic functions.  $Qe$  is entity set and  $Qt$  is timestamp set.  $\mathcal{E}$  is the set of all entities,  $\mathcal{T}$  is the set of all timestamps and  $\mathcal{F}$  is the set of triples.
Name	Symbol	Input	Output
And	And	\( Qe_1 \), \( Qe_2 \)	\( Qe_1 \cap Qe_2 \)
Or	Or	\( Qe_1 \), \( Qe_2 \)	\( Qe_1 \cup Qe_2 \)
Not	Not	\( Qe \)	\( \mathcal{E} - Qe \)
EntityProjection	Pe	\( Qe \), r, Qt	\( \{o|s \in Qe, t \in Qt, (s,r,o,t) \in \mathcal{F}\} \)
TimeProjection	Pt	\( Qe_1 \), r, \( Qe_2 \)	\( \{t|s \in Qe_1, o \in Qe_2, (s,r,o,t) \in \mathcal{F}\} \)
TimeAnd	TimeAnd	\( Qt_1 \), \( Qt_2 \)	\( Qt_1 \cap Qt_2 \)
TimeOr	TimeOr	\( Qt_1 \), \( Qt_2 \)	\( Qt_1 \cup Qt_2 \)
TimeNot	TimeNot	Qt	\( \mathcal{T} - Qt \)
before	before	Qt	\( \{t|t < \min(Qt)\} \)
after	after	Qt	\( \{t|t > \max(Qt)\} \)
B.2 Details of Query Generation
The Design of Query Structure Schema. Previous query embedding methods share the same query structure schema (Table 4) to perform complex logical reasoning over static knowledge graph. When it comes to dynamic knowledge graph (TKG), the static schema is not sufficient to capture the temporal reasoning. In this paper, we propose a novel query structure schema for reasoning. The schema is composed of basic functions and query structures. The basic functions are defined in Table 5. The query structures are shown in Table 6 with visualization in Figure 5 and Figure 6. The query structures are based on the basic functions: And, Or, Not, EntityProjection, TimeProjection, TimeAnd, TimeOr, TimeNot, Before, and After. Of course, users can define more basic functions or query structures to extend the schema to multi-modal KGs, hyper-relation KGs and so on. It is a flexible schema overall.
In order to keep a similar experiment settings with previous static query embeddings, the query structures are further aggregated into groups, shown in Table 7. The groups  $\mathrm{avg}_e$ ,  $\mathrm{avg}_t$ ,  $\mathrm{avg}_{e,\mathcal{C}_e}$ ,  $\mathrm{avg}_{t,\mathcal{C}_t}$  are for training and testing. Besides, extra groups  $\mathrm{avg}_{\{\mathcal{U}_e\}}$ ,  $\mathrm{avg}_{\{\mathcal{U}_t\}}$ ,  $\mathrm{avg}_x$  are for evaluation and testing only.
In implementation, we define the basic functions and query structures as functions in python, which are named Complex Query Function (CQF). There are lots of advantages of using python functions to define query structures. Firstly, the functions are easy to understand and debug. Secondly, the functions are easy to be reused in the dataset-sampling process and model-training process. Thirdly,
Table 6: Definition of query structures in temporal query embeddings. Operators on entity sets include Pe, And, Or, and Not. Operators on timestamp sets include Pt, TimeAnd, TimeOr, TimeNot, after, and before. The prefix "a" and "b" represent "after" and "before" respectively. The prefix "s" or "o" mean that the sub-query is a subject query or an object query respectively.
Type	Query Name	Query Structure Definition
entity multi-hop	Pe	Pe(s, r, t)
Pe2	Pe(Pe(s1, r1, t1), r2, t2)
Pe3	Pe(Pe(Pe(s1, r1, t1), r2, t2), r3, t3)
e2i	And(Pe(s1, r1, t1), Pe(s2, r2, t2))
e3i	And(Pe(s1, r1, t1), Pe(s2, r2, t2), Pe(e3, r3, t3))
entity not	e2i_N	And(Pe(s1, r1, t1), Not(Pe(s2, r2, t2)))
e3i_N	And(Pe(s1, r1, t1), Pe(s2, r2, t2), Not(Pe(s3, r3, t3)))
Pe_e2i_N	Pe(And(Pe(s1, r1, t1), Not(Pe(s2, r2, t2))), r3, t3)
e2i_PeN	And(Pe(Pe(s1, r1, t1), r2, t2), Not(Pe(s2, r3, t3)))
e2i_NPe	And(Not(Pe(Pe(s1, r1, t1), r2, t2), Pe(s2, r3, t3))
entity union	e2u	Or(Pe(s1, r1, t1), Pe(s2, r2, t2))
Pe_e2u	Pe(Or(Pe(s1, r1, t1), Pe(s2, r2, t2)), r3, t3)
time multi-hop	Pt	Pt(s, r, o)
aPt	after(Pt(s, r, o))
bPt	before(Pt(s, r, o))
Pe_Pt	Pe(s1, r1, Pt(s2, r2, o1))
Pt_sPe_Pt	Pt(Pe(s1, r1, Pt(s2, r2, o1)), r3, o2)
Pt_oPe_Pt	Pt(s1, r1, Pe(s2, r2, Pt(s3, r3, o1)))
t2i	TimeAnd(Pt(s1, r1, o1), Pt(s2, r2, o2))
t3i	TimeAnd(Pt(s1, r1, o1), Pt(s2, r2, o2), Pt(s3, r3, o3))
time not	t2i_N	TimeAnd(Pt(s1, r1, o1), TimeNot(Pt(s2, r2, o2)))
t3i_N	TimeAnd(Pt(s1, r1, o1), Pt(s2, r2, o2), TimeNot(Pt(s3, r3, o3)))
Pe_t2i_N	Pe(s1, r1, TimeAnd(Pt(Pe(s2, r2, t1), r3, o1), TimeNot(Pt(s3, r4, o2))))
t2i_NPt	TimeAnd(TimeNot(Pt(Pe(s1, r1, t1), r2, o1)), Pt(s2, r3, o2))
t2i_PtN	TimeAnd(Pt(Pe(s1, r1, t1), r2, o1), TimeNot(Pt(s2, r3, o2)))
time union	t2u	TimeOr(Pt(s1, r1, o1), Pt(s2, r2, o2))
Pe_t2u	Pe(s1, r1, TimeOr(Pt(s2, r2, o1), Pt(s3, r3, o2)))
hybrid multi-hop	between	TimeAnd(after(Pt(s1, r1, o1)), before(Pt(s2, r2, o2)))
e2i_Pe	And(Pe(Pe(s1, r1, t1), r2, t2), Pe(s2, r3, t3))
Pe_e2i	Pe(e2i(s1, r1, t1, s2, r2, t2), r3, t3)
t2i_Pe	TimeAnd(Pt(Pe(s1, r1, t1), r2, o1), Pt(s2, r3, o2))
Pe_t2i	Pe(s1, r1, t2i(s2, r2, o1, s3, r3, o2))
Pe_aPt	Pe(s1, r1, after(Pt(s2, r2, o1)))
Pe_bPt	Pe(s1, r1, before(Pt(s2, r2, o1)))
Pe_at2i	Pe(s1, r1, after(t2i(s2, r2, o1, s3, r3, o2)))
Pe bt2i	Pe(s1, r1, before(t2i(s2, r2, o1, s3, r3, o2)))
Pt_sPe	Pt(Pe(s1, r1, t1), r2, o1)
Pt_oPe	Pt(s1, r1, Pe(s2, r2, t1))
Pt_se2i	Pt(e2i(s1, r1, t1, s2, r2, t2), r3, o1)
Pt_oe2i	Pt(s1, r1, e2i(s2, r2, t1, s3, r3, t2))
Figure 5: Visualization of temporal query structures answering entity sets. These structures are basic functions that can be used to construct more complex temporal query structures.
Figure 6: Visualization of temporal query structures answering timestamp sets. These structures are basic functions that can be used to construct more complex temporal query structures.
Table 7: The query structures are aggregated into groups. The groups are inspired by the experiment settings in static query embeddings [1-4]. The groups  $\mathrm{avg}_e$ ,  $\mathrm{avg}_t$ ,  $\mathrm{avg}_{e,\mathcal{C}_e}$ ,  $\mathrm{avg}_{t,\mathcal{C}_t}$  are for training and testing. Besides, extra groups  $\mathrm{avg}_{\{\mathcal{U}_e\}}$ ,  $\mathrm{avg}_{\{\mathcal{U}_t\}}$ ,  $\mathrm{avg}_x$  are for evaluation and testing only.
avge	1p	2p	3p	2i	3i
Pe	Pe2	Pe3	e2i	e3i
avgt	Pt, aPt, bPt	Pe_Pt	Pt_sPe_Pt, Pt_oPe_Pt	t2i	t3i
avge,eCe	2in	3in	inp	pin	pni
e2i_N	e3i_N	Pe_e2i_N	e2i_PeN	e2i_NPe
avgt,Ct	t2i_N	t3i_N	Pe_t2i_N	t2i_PtN	t2i_NPt
avg{Ue}	2u	up
e2u	Pe_e2u
avg{Ut}	t2u	Pe_t2u
avgx	pi	ip
e2i_Pe	Pe_e2i	Pe_aPt	Pe_bPt
t2i_Pe	Pe_t2i	Pe_at2i	Pe bt2i
between	Pt_sPe	Pt_oPe
Pt_se2i	Pt_oe2i
the functions are easy to be extended to support more complex query structures and reimplement the existing query embedding methods.
As is introduced in Section 4, we use the computation graph to represent the reasoning process. To get the computation graph of a CQF, we use the python interpreter to parse the function to Abstract Syntax Tree (AST), which is a more friendly readable subset of computation graph. In this way, executing the computation graph is equivalent to executing the CQF in python interpreter. Since we have the god privileged access to the AST, we can modify the interpreter to support various dynamic reasoning processes. Below we show how to unify (1) the reasoning of ground-truth, (2) the sampling of anchors, and (3) the reasoning of embedding-based methods by modifying the python interpreter.
Ground-Truth Reasoning Interpreter.
In the reasoning of ground-truth, we need to get all real answers of a query under a given TKG. To achieve this, the first we need to do is to implement the basic functions. We wrap the python built-in set to QuerySet class to store entities and timestamps. The basic functions are implemented as lambda function in python, with input and output as QuerySet. Then, the basic functions are registered as symbols to the python interpreter. When executing the CQF, the python interpreter will call the corresponding basic functions to get the final answer. The answers are finally generated from subgraph matching, which depends on the ground truth in TKG. We show the pseudocode of the ground-truth reasoning interpreter in Figure 7.
Anchors Sampling Interpreter.
The anchor, which may be entity, relation or timestamp, is the input argument of the CQF. The aim of the anchors sampling interpreter is to get the possible anchors and answers of a query under a given TKG. For the sampled anchors, we expect the answer set not empty. Since the answers could be obtained from the ground-truth reasoning interpreter, we only focus on the sampling of anchors in the anchors sampling interpreter.
Given the standard split of edges into training  $(\mathcal{F}_{\mathrm{train}})$ , validation  $(\mathcal{F}_{\mathrm{valid}})$  and test  $(\mathcal{F}_{\mathrm{test}})$  sets, we append inverse relations and double the number of edges in the graph. Then we create three graph:  $\mathcal{G}_{\mathrm{train}} = \{\mathcal{V},\mathcal{R},\mathcal{T},\mathcal{F}_{\mathrm{train}}\}$ ,  $\mathcal{G}_{\mathrm{valid}} = \{\mathcal{V},\mathcal{R},\mathcal{T},\mathcal{F}_{\mathrm{train}} + \mathcal{F}_{\mathrm{valid}}\}$ ,  $\mathcal{G}_{\mathrm{test}} = \{\mathcal{V},\mathcal{R},\mathcal{T},\mathcal{F}_{\mathrm{train}} + \mathcal{F}_{\mathrm{valid}} + \mathcal{F}_{\mathrm{test}}\}$ . Given a query  $q$ , let  $\llbracket q\rrbracket_{\mathrm{train}}$  and  $\llbracket q\rrbracket_{\mathrm{test}}$  denote a set of answers (entities or timestamps) obtained by subgraph matching of  $q$  on  $\mathcal{G}_{\mathrm{train}}$ ,  $\mathcal{G}_{\mathrm{valid}}$  and  $\mathcal{G}_{\mathrm{test}}$ . For each query  $q$ , the reasoning process starts from anchor nodes and computes the final answer set via subgraph matching.
For the basic function Pe and Pt, we simply use all the triples and extract the entities and timestamps as the anchors. These anchors are supposed to have no empty answers under Pe and Pt. However, when it comes to some hard queries, such as e2i and e3i, the random sampled anchors may have empty answers. Because the TKG is sparse and incomplete, the intersection of two query sets has a high probability to be empty. The empty answers are meaningless for model-training and damage to