SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Implications  $d, T_{a}, \omega, C_{p}, C_{n}$  jointly result in the probability distributions of  $c_{1}, c_{2}$ , where  $c_{1}, c_{2} > 0$  is generally guaranteed by experimental settings. Two conclusions can then be drawn from the two theorems. (1) Convergence. We can see from theorem 1 that  $g(\tau) \to 0$  as  $\eta \to 0$ . This implies that the  $g(\tau)$  of perfect TRs ( $gc(\tau) = 1$ ,  $\eta = 1$ ) will converge to its minimum of 0. (2) Competition. Since  $g(\tau)$  is invariant to  $t$ , comparing these two theorems also tells us about the competing effect of well-trained sample pairs for the value of  $g(\tau)$ , generally affected by the ratio  $\frac{m_1}{m_2}$ . These two conclusions, along with the fact that  $g(\tau)$  is proportional to the support  $sp(\tau)$  (as illustrated in A.2.1), jointly make  $g(\tau)$  reveal the TR validity  $gc(\tau)$  and thus guarantees NE's effectiveness in TR detection.
It is worth noting that  $d$  is generally set as  $d > T_{a}$  to control the values of  $c_{1}, c_{2}$ , where  $T_{a}$  is the number of absolute time points of the whole event set. Otherwise, NE's TR detection performance will be interfered. For example, if  $d \ll T_{a}$ , then  $d \ll m$  in most cases. It will thus be very likely that  $c_{1} = 0$  so that theorem 1 can not be applied in NE.
B Experimental Supplements
All models are trained for 100 epochs on each dataset using the Adagrad optimizer (with a learning rate of 0.01) and the StepLR learning rate scheduler (with a step size of 10 and gamma of 0.9). The experiments are conducted on a single GPU ( GeForce RTX 3090). The hyper-parameters of NE are fine-tuned using a grid search to achieve relatively optimal results.
B.2 Main Results with Error Bars
To ensure the reliability of the results, the experiments are repeated three times, and the error bars are derived accordingly. Table 5 and 6 show that the main results are quite stable with small error bars. Note that the precision and recall rates in Table 6 correspond exactly to the F1 scores in Table 5. The reason why the recall rate of NE is lower than that of TASTER is that we report the highest F1 score of each model in comparative studies by tuning their respective global threshold, denoted as  $g_{th}$ . As the F1 score is calculated as the harmonic mean of precision and recall rates, TASTER achieves its highest F1 score by reporting many false positives, resulting in a relatively high recall rate but an extremely low precision rate.
B.3 Additional Ablations
Unless otherwise specified, the experiments below adopt the original parameter settings as described in the main text.
B.3.1 Normalization of Event Type Vectors
Table 7 demonstrates that normalized event type vectors slightly outperform unnormalized ones in the TR detection task.
Table 5: TR detection by F1 scores and TR query by confidence ratios
Embedding	TR Detection (F1)	TR Query (r)
D14	D18	GDELT	D14	D18	GDELT
TNTComplEx	0.26±0.01	0.18±0.00	0.08±0.01	0.08±0.00	0.08±0.00	0.01±0.00
DE-Simple	0.22±0.00	0.20±0.00	-	0.09±0.00	0.09±0.00	-
TASTER	0.18±0.00	0.15±0.00	0.08±0.00	0.09±0.00	0.09±0.00	0.00±0.00
TeRo	0.43±0.02	0.64±0.01	0.16±0.00	0.08±0.00	0.08±0.00	0.01±0.00
BoxTE	0.40±0.01	0.40±0.02	0.18±0.01	0.08±0.00	0.08±0.00	0.01±0.00
ATISE	0.40±0.01	0.44±0.01	0.18±0.01	0.08±0.00	0.08±0.00	0.01±0.00
NE with g'(τ)	0.78±0.00	0.79±0.00	0.48±0.00	0.85±0.00	0.83±0.00	0.83±0.00
NE with g(τ)	0.82±0.00	0.83±0.00	0.51±0.00	0.87±0.00	0.86±0.00	0.85±0.00
Table 6: TR Detection by precision and recall rates
Embedding	Precision	Recall
D14	D18	GDELT	D14	D18	GDELT
TNTComplEx	0.22±0.01	0.11±0.00	0.04±0.00	0.33±0.02	0.50±0.01	1.00±0.01
DE-Simple	0.16±0.00	0.14±0.00	-	0.35±0.03	0.33±0.00	-
TASTER	0.10±0.00	0.08±0.00	0.04±0.00	0.99±0.01	0.99±0.01	0.97±0.03
TeRo	0.51±0.04	0.91±0.01	0.20±0.04	0.37±0.02	0.49±0.00	0.14±0.02
BoxTE	0.40±0.02	0.39±0.05	0.15±0.01	0.41±0.02	0.41±0.02	0.22±0.03
ATISE	0.35±0.01	0.49±0.03	0.15±0.01	0.47±0.01	0.40±0.00	0.21±0.01
NE with g'(τ)	0.99±0.00	0.98±0.00	0.90±0.00	0.64±0.00	0.66±0.00	0.32±0.00
NE with g(τ)	0.99±0.00	0.99±0.00	0.83±0.00	0.70±0.00	0.72±0.00	0.37±0.00
B.3.2 Dimension of Embedding Vectors
Table 8 and 9 demonstrate that  $d$  scarcely affects the performance of NE as long as it is more than some certain value. It is worth noting that NE's detection performance in GDELT may be further improved with larger values of  $ds$ , as illustrated in A.2.2, because  $T_{a} = 2976$  in GDELT, which is much larger than the tested dimensions.
B.3.3 Threshold for Valid TRs
In the main text, the threshold for distinguishing valid and invalid TRs is chosen as  $\theta = 0.8$ . Here we report NE's results on D14 with varying  $\theta$ s in Table 10. It is shown that NE still has an overwhelming advantage over all baselines.
B.4 Mined TRs
Additional cases of mined TRs are shown in Table 11 as below.
Table 7: F1 of NE on ICEWS14 with different event type vectors
Normalized	Unnormalized
F1	0.82	0.80
Table 8: NE on GDELT with different dimensions
d	100	200	300	400	500
TR Query (r)	0.83	0.84	0.85	0.85	0.84
TR Detection (F1)	0.22	0.45	0.50	0.51	0.51
Table 9: NE on ICEWS14 with different dimensions
d	50	100	200	400	600	800
TR Query (r)	0.81	0.86	0.87	0.87	0.87	0.87
TR Detection (F1)	0.29	0.55	0.76	0.82	0.83	0.83
Table 10: Statistical results with different thresholds  $\theta$  on ICEWS14
Embedding	Detection(F1)	Query(r)
θ=0.7	θ=0.8	θ=0.9	θ=0.7	θ=0.8	θ=0.9
TNTComplEx	0.29	0.26	0.25	0.07	0.08	0.06
DE-Simple	0.28	0.22	0.21	0.09	0.09	0.21
TASTER	0.28	0.18	0.17	0.09	0.09	0.08
TeRo	0.35	0.43	0.40	0.08	0.08	0.06
BoxTE	0.38	0.40	0.41	0.08	0.18	0.06
ATISE	0.28	0.35	0.33	0.08	0.08	0.06
NE with g'(τ)	0.63	0.78	0.80	0.81	0.85	0.86
NE with g(τ)	0.71	0.82	0.84	0.87	0.87	0.87
Table 11: Cases of Mined TRs from ICEWS18
Body Event	Head Event	gc	sp
\(s_b\)	\(p_b\)	\(o_b\)	\(s_h\)	\(p_h\)	\(o_h\)	τ
North Korea	Threaten with military force	South Korea	North Korea	Make em- pathetic comment	South Korea	87	0.86	3
Bahrain	Reduce or break diplomatic relations	Foreign Affairs (United States)	Bahrain	Consult	Foreign Affairs (United States)	0	0.86	3
Japan	Host a visit	Yoshitaka Shindo	Yoshitaka Shindo	Make a visit	Japan	0	1	7
Government (Syria)	Sign formal agreement	Armed Rebel (Syria)	Armed Rebel (Syria)	Sign formal agreement	Government (Syria)	0	1	5
Protester (Thailand)	Make statement	Thailand	Protester (Thailand)	Defy norms, law	Military (Thailand)	129	0.86	3
China	Make pes-simistic comment	Japan	China	Host a visit	Yasuo Fukuda	148	0.83	5
Court Judge (India)	Express intent to cooperate	Citizen (India)	Citizen (India)	Accuse	Villager (India)	132	0.83	5
Kim Jong-Un	Appeal for diplomatic cooperation (such as policy support)	South Korea	South Korea	Engage in diplomatic cooperation	Iran	27	0.86	3
South Korea	Make pes-simistic comment	North Korea	Canada	Sign formal agreement	South Korea	68	0.86	3
China	Appeal for diplomatic cooperation (such as policy support)	Malaysia	South Korea	Express intent to settle dispute	China	111	0.86	3
Iraq	Host a visit	Massoud Barzani	Mohammad Javad Zarif	Consult	Massoud Barzani	57	0.86	3
Japan	Appeal for diplomatic cooperation (such as policy support)	Thailand	Citizen (Thailand)	Fight with small arms and light weapons	Thailand	118	0.86	3