Noether Embedding: Efficient Learning of Temporal Regularities
Authors: Chi Gao
Zidong Zhou
Luping Shi
Venue: NeurIPS 2023 poster (2023)
Keywords: Schema Learning
Temporal Regularity
Event Embedding
TL;DR: An event embedding that enables the data-efficient formation and time-efficient retrieval of temporal regularities in events simply through embedding each event sample.

Abstract: Learning to detect and encode temporal regularities (TRs) in events is a prerequisite for human-like intelligence. These regularities should be formed from limited event samples and stored as easily retrievable representations. Existing event embeddings, however, cannot effectively decode TR validity with well-trained vectors, let alone satisfy the efficiency requirements. We develop Noether Embedding (NE) as the first efficient TR learner with event embeddings. Specifically, NE possesses the intrinsic time-translation symmetries of TRs indicated as conserved local energies in the embedding space. This structural bias reduces the calculation of each TR validity to embedding each event sample, enabling NE to achieve data-efficient TR formation insensitive to sample size and time-efficient TR retrieval in constant time complexity. To comprehensively evaluate the TR learning capability of embedding models, we define complementary tasks of TR detection and TR query, formulate their evaluation metrics, and assess embeddings on classic ICEWS14, ICEWS18, and GDELT datasets. Our experiments demonstrate that NE consistently achieves about double the F1 scores for detecting valid TRs compared to classic embeddings, and it provides over ten times higher confidence scores for querying TR intervals. Additionally, we showcase NE's potential applications in social event prediction, personal decision-making, and memory-constrained scenarios.

Noether Embedding:
Efficient Learning of Temporal Regularities
Chi Gao Zidong Zhout Luping Shi"
Center for Brain- Inspired Computing Research,
Optical Memory National Engineering Research Center,
Tsinghua University China Electronics Technology HIK Group
Co. Joint Research Center for Brain-Inspired Computing.
IDG / McGovern Institute for Brain Research at Tsinghua University,
Department of Precision Instrument, Tsinghua University, Beijing 100084, China. {gaoc20, zzd21)Gmaill ismaalss... tsinghua. edu. cn aashi@mail.. tsinghua. edu.
Abstract
Learning detect and encode temporal regularities (TRs) in events is prerequisite for human-like intelligence. These regularities should be formed from limited event samples and stored easily retrievable representations. Existing event embeddings, however, cannot effectively decode TR validity with well-trained vectors, let alone satisfy the efficiency requirements. We develop Noether Embedding (NE) the first efficient TR learner with event embeddings. Specifically, NE possesses the intrinsic translranslation symmetries of TRs indicated conserved local energies in the embedding space. This structural bias reduces the calculation of each TR validity embedding each event sample, enabling NE to achieve data-efficient TR formation insensitive to sample size and time-efficien TR retrieval in constant time complexity. To comprehensively evaluate the TR learning capability of embedding models, we define complementary tasks of TTR detection and TR query. formulate their evaluation metrics, and assess embeddings on classic ICEWS14. ICEWS18, and GDELT datasets. Our experiments demonstrate that NE consistently achieves about double the Fl scores for detecting valid TRs compared to classic embeddings, and it provides over ten times higher confidence scores for querying TR intervals. Additionally, we showease NE's potential applications in social event prediction, personal decision dccision-making. and ceemory-constraing scenarios.
1
Introduction
Recall the last time you went to restaurant but waited for half an hour after ordering dishes. You probably knew something was wrong and may have called the waitperson for help. This behavior guided by the temporal regularity (TR) of 'order dishes -(about 10 minutes)-> have meals' stored in your brain schemas (Ghosh & Gilboal 2014). Such TRs play significant role in enabling humans to exhibit flexible out-of-distribution and systematic generalization abilities (Goyal Bengio 2022), and directly learned from experience through statistical accumulation of common event structures (Pudhiyidath et al.) 2020), as shown in Figure [
Since there exist enormous potential TRs due large number of event types and time intervals, detecting valid TRs from all
'quual contribution. *Corresponding author.
The code is publicly available at: httppp///ttttti..cmmm/eeeeeaeeee
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

potential ones is therefore necessary, serving as a prerequisite capability for humans to form more complex event schemas in the brain support downstream cognitive functions (Schapiro et al. 2017; McClelland et al., 1995). To attain human-level abilities in event scenarios, it is crucial to possess two fundamental properties when learning TRs. Firstly, TRs should formed from even limited number of experiences, which humans achieve since childhood (Pudhiyidath et al. [2020). Secondly, TRs should be stored as representations that can be instantly retrieved given appropriate cues, which is a central feature of human memory (Chaudhuri & Ficte 2016).
Leamed
Restaurant Scenarios
wijp
Potential TR
Order Dishes
time
TR
TR
Play with the Phone
f
Have
time
TR
Free Chat
fs
Pay Bills
Event Items
time
Event Schema
Learning TRs
Figure 1: Illustration of TR learning. TRs indicate temporal associations invariant to time shifts, and are learned from event items through statistical accumulation.
remains an open problem jointly achieve the data-efficient formation and time-efficiem retrieval of 1-1 TRs with event embeddings, although embedding models have achieved outstanding performance in various tasks such event completion and event prediction (Cai et al. event embeddings can only encode patterns such as inversion and composition and decode the fitted event occurrences for better performance in completion tasks (Xu et al. 2020b Wang al.) 2020 Messner et al.] 2022). However, we aim to encode TRs by directly training the embeddings of each event sample and decode TRs by calculating well-trained embeddings without first decoding the fitted event occurrences. Our primary challenge in achieving such counterintuitive function is to design the inductive bias that automatically integrates the event statisties of each potential TR over time.
2022 Zhao I02T). Classic
Symmetry governs regularities in nature (Tanaka Kunin 202T). long before Noether proved the equivalence between symmetries and conservation laws in physical system (Noether 1918). Inspired by Noether's 's theorem, we develop Noether Embedding (NE) with the intrinsic imn-trannlation symmetries of TRs indicated conserved local energies in the embedding space. Calculating the event statistics of each potential TR is therefore converted to reducing the training loss of all event embeddings. This allows the direct revelation of TR validity by decoding the corresponding local energies through calculating well-trained embeddings after training convergence.
Contributions are twofold. Firstly, we develop NE which for the first time jointly achieves the data- efficient formation and time-efficient retrieval of TRs solely by embedding event samples. Secondly, we define complementary tasks of TR detection and TR query, formulate their evaluation metrics, and adopt classic datasets for evaluations, aiming at complete evaluations of embeddings' TR learning capabilities. Both our tasks and method generalize arbitrary forms of structured events.
2
Problem Formalization
2.1 Definitions
Temporal Regularity (TR) An event item 9 can generally be represented by the basic symbolic form of (ev, t), where ev is the event type, and f is the discrete occurrence time. Building on the interpretations from cognitive science literature (Ghosh & Gilboa 2014 Pudhiyidath al. 2020). we formally define TRs as temporal associations that remain invariant to time shifts:
(evb, t) (eun, t t- 7) Vt
Tg
evn * eun, 1 + respectively refer to the body and head event type and their occurrence time. To refers to the complete collection of the absolute time points in the whole event set, and r denotes the relative time. Note that 0 indicates the synchrony of body and head event occurrences.

---

For example, a TR could be: Whenever someone orders dishes in restaurant, he or she will have meals in around ten minutes, where 7 10.
Metrics for TR Validity Since real-world data often contain noise, we introduce an adaptive A = [r(1 7)) 7(1 + 7)) to replace when evaluating statistically learned temporal regularities from events. For an evaluated TR abbreviated as tr : (evs, e T, T))). if an event (ev, t) satisfies that ev evb, denote it b(g; tr); if of a TR as the number of event pairs respectively satisfying the body and head:
evh, we denote it as h(g ir). We define the support
sp(tr) = rlbb(q; tr) A h(d'; A (t' - t) E ())
denotes 'and', denotes 'in', and q: (ev, t). (ev', t') refer arbitrary two different events in the event set. Note that when calculating sp(tr), we can only count one event once and in one pair to avoid overcounting when events occur in consecutive periods.
We respectively define the standard confidence, head coverage, and general confidence of a TR as: sp(tr)
sp(tr) sc((r) n(b(tr))' he((r)) m((((tr))
ge(tr) = 2
sa(tr) he(tr)
mbb(tr)) n(((t))) respectively represent the number of events q (ev, t) satisfying ev = etb, ev in the event set. Here we borrow the metrics sc, he, ge generally used in the rule mining field Galárraga et al. 2015) to ensure fair and reasonable evaluations. We modify them by introducing evaluate TR validity. Intuitively, standard confidence can be viewed as
an adaptive with the probability that the head event will occur within time + once a body event occurs at time t, whose statistical sufficiency is supported by sp. he and ge can be interpreted similarly.
Above some sp, the higher the ge, the more valid TR is. For potential TR tr (erg, eth, T, T,7) with fixed event types evg, evn, and ratio 1), its general confidence can be written as a function of T: gc(T).
2.2 Tasks
For fixed 17 in A. potential TR can be arbitrary evy, r). where i,j E P,T T, (P is the set of event types, Tr the set of relative time points). Therefore, we define the two complementary tasks below comprehensively evaluate the TR learning capabilities of event embeddings.
TR Detection For query (eug, ev,), ). its ground truth confidence gc, max ge(T). Queries whose rET.
gCg 2 0 are considered to imply valid TRs that reveal good regularities, while queries whose gCg < considered to imply invalid TRs, where is fixed threshold.
The task of TR detection to identify valid TRs from all tested ones. The model is expected to determine whether a query (evs, ev%) implies valid TR. The F1 score of all judgments is reported.
TR Query Only queries that imply valid TRs are considered for testing. The ground truth relative time Ty is set as what maximizes ge(T) in computing gCg- The model outputs 7'.
The task of TR query is output the correct 7' Ty for valid TRs. For each tested query (cU,, eul). ratio r' = so(rr) is computed. The averaged ratio of all queries is reported.
3 Noether Embedding
3.1 Inspirations from Noether's Theorem
Noether's theorem In 1915, mathematician Emmy Noether proved one of the most fundamental theorems in thcoretical physics: every diffeerentialee symmetry of the action of physical system with conservative forces has a corresponding conservation law. Specifically, time-translation symmetry corresponds to energy conservation.
TRs indicate time-translation symmetries An event pair (ev,, + (evn, L + +T can be regarded as mapping of the body and the head event type over / with a parameter 7. Therefore, ideal TRs indicate the invariance of such mappings under the transformation of time translation since the mapping holds Vt e T. for TRs.
3

---

Construct embeddings with conserved local energies Denote q(t: ev) as the embedding of each event sample, and ((q(t; ev;). q(t + т; evR)) as the local energy corresponding body-and-head event pair of potential TR. If B is innately conserved, meaning that g(T; Evb; eti,) is invariant to i, it indicates time-translation symmetry. We can then use the value of g to approximate TR validity after training each event embedding q(t; ev). A more strict correspondence between NE variables and those in physical system is shown in Appendix AA.1.
Woether-Inspired structural biases Accordingly. the enabling factors of NE can be summarized as follows: (i) the event embedding should be constructed to make each local energy remain invariant f; (ii) the training loss should be designed make the value of approximate TRR validity; (iii) the local energy g should be used as the decoding function. We thus construct NE as below.
event
Relevant events
reveal bad regularity
(T)
Event
Noether Embeddlnng time
Decoding Results
event type
Itemss
absolute time
relativa time
encode
decode
(v(r)
event type
Relevant events TRp
(r)
type
gb
bg
reveal
good regularity
relative
Figure 2: Illustration of NE. Solid lines and purple graphs in the middle jointly represent the data flow of NE. The red graphs below and the blue ones above demonstrate cases of TR with significant temporal regularities and TR with no. It is shown that each decoding result reveals an integrated temporal association of its relevant event pairs separated in time.
3.2 NE's Framework and Formulas
Framework As illustrated Figure 2 NE uses distributed storage of complex vectors to learn TRs. At the encoding stage, the event items are converted to NE representations through embedding each event sample, where TR validity is automatically calculated and stored in the embedding space. At the decoding stage, given each query (evr, evx), potential TRs are detected and queried by directly calculating their relevant embeddings to derive the corresponding decoding results g(T).
The Encoding Stage
(4) In the event embedding above, denotes the Hadmard (or element-wise product. u(ev), r(t) C v(er)
(t::ev) = u(ev) o r!))
are complex vectors where u(ev) = ][[ev]]| v(ev) E Cd, r(t) = ei w E Rd, and is the dimension of vectors. Each event type corresponds to an independently trainable vector of u(ev), while w a global time vector for r() of all event embeddings. Note that the d ws in w are fixed to certain distribution. The event embedding q(t; ev) can thus be depicted as rotation of event-type vectors u (ev) by time rtt) in the d-dimensional complex space.
The score function and loss function of each event sample are defined as follows:
f(t:ev)= Σ (et))g(ttev)).
(5)
L(E: C(E:Cp,C,, ( c,)
(6)
We denote & as positive event sample and Ç's as its generated negative samples whose number N. For positive sample { (ev,t) its negative samples ć's are the whole set of { t' * t)}

---

where t' E Ta. Cp. On are two different constants for positive and negative samples, respectively. Cp.Cn @ because |u(er)]| = 1, and we generally set Cp 1, C, 0.
Until the training converges, events and TRs form a distributed storage, which includes the global time vector of w and trainable event type vectors u u(ev). At the decoding stage, no training but only the inference of vectors is conducted, as described below.
The Decoding Stage
The decoding function for a query (evs, ev%) is:
g(7) = uu - un r[])
ub; U4 are the event type vectors respectively of etb, e0%. T is traversed through set T, of the relative time points such as Tr: (-Tmaz:-.. 0, .... Tmar to plot the decoding results. min g(T) is computed, TET, which is compared with a global threshold Ith to decide whether potential TR is valid not (for TR detection). For a valid TR, the r' which minimizes g(T), T E TT, is selected as the model output of the relative time (for TR query).
Since r(t +
r(t) r()), r(t) or() = 1, Vt T.. the decoding function exactly indicates the conserved local energy: g(T) = |u un o r(r)ll² = I|u, o r(t) un r(t + +) = (t),(t) - (t (t + r)ll Vt T.. This indicates that g (T: evb, eva) is invariant to and the conserved energy of arbitrary two event samples is of a quadratic form in the embedding space.
3.3
Why NE is Efficient
Briefly speaking, the Fourier-like representations enable NE's large-capacity storage for TR validity and event occurrences, serving as a prerequisite for NE to learn TR effectively. The iosthee-inspired structural biases further leads NE's efficient TR learning capabilities. Here we only illustrate the effect of the structural biases. The reasons why NE learns TR effectively is explained in Appendix [A.2
Data-efficiency by Encoding Translation Symmetries The invariance of gT)) to means that the value of g(T) is determined by competitive effect between sample pairs across time. Considering event sample pairs (evg, t), (ev,, + 7) with varying in the embedding space. if a sample pair are both positive or both negative samples, they will decrease g(T) since their score functions are mapped the same constant. Otherwise, they will increase g(r). Since g(T) is invariant to f, g(7) trained to balance these two forces. Therefore, the value of g(T) after training convergence is generally determined by the ratio of sample pairs with increasing or decreasing forces. g(T) is thus insensitive to the number of sample pairs that are both positive. This results in data-efficient TR formation in NE, even with limited event occurrences from which learn TR.
Time-efficiency by Decoding Conserved Energies By calculating g(T) 1|u un (12 E T,, we enable efficient TR querying for each query (ets, eUR). This process has a constant time complexity since T, is arbitrary user-selected set of relative time points, and the vector dimension can be effectively handled using GPUs. Importantly, the querying time is independent of the number of events in the entire event set and the relevant event occurrences supporting the queried TR.
4 Experiment
It is important to highlight that in our evaluation, we initially compare NE and classic embeddings in terms of learning effectiveness (Section 4.2). without considering the efficiency requirements. As classic embeddings are shown to be ineffective in learning TRs, we then focus on demonstrating the
learning efficiency of NE in Section 4.3
4.1 Experimental Setting
Dataset A temporal knowledge graph (TKG) comprises (s, t) quadruples (Leblay & Chekol 2018), where 8, o, represent the subject, predicate, object, and time. TKG is widely used in variety of fields to represent global political events (Trivedi et al. 2017). financial incidents (Yang
et al. 2019), user-item behaviors (Xiao et al. 2020), etc. Notably, ICESS (Boschee et al.) 2015) and GDELT LT (Leetaru & Schrodt 2013) are two popular data sources for TKG research (Cai et al. 2022)..
5

---

In our experiments, we use ICEWS14 and ICEWS18, the same as in (Han et al. 2020). They contain global political events in 2014 and 2018, and we denote them as D14 and D18, respectively. We also use the GDELT released by (Jin et al. 2019). which contains global social events from 2018/1/1 to 2018/1/31. the experiments, we denote each (s, P, o) triple as a specific event type eu. It is worth mentioning that alternative settings, such as representing each predicate p as an event type are also applicable to our model.
Model Implementation For NE, 400, Cp = C, 0 and the global time vector w set as
0, ..... d - 1, where Tu is the number of absolute time points, and wmar is a tunable hyperparameter set as 600. The training details are in Appendix [.. We compare NE with six classic and vastly different TKG embeddings, DE-SimplE (Goel et al. 2020), TeRo (Xul et al. (020b), ATiSE (Xu et al. 2020a). TNTComplex (Lacroix et al. 2020), Box TE: (Messner et al. 2022). and TASTER (Wang et al. 2023) with their original parameter settings and 400 the same as NE. We set the queried time set T,, {1 Ta..... O,... Ta } at the decoding stage. In this way, only one query needs to be decoded between each (ev, en2) and (evj, en;), i #
W%
(2 Xammam
T.
Adaptations. All baselines can only output their score function (t) to decode event occurrences but cannot directly decode TR validity by (T) NE does. For comparison, we add an interface g'(r) Emetrer, ((0)),(
Eier, (((t) Eier, (t) these models that indirectly compute TR validity from the decoded event occurrences. We also evaluate NE with g' (T) to show the validity of g (T) itself.
Evaluation Details We select (evp, eun)s )s for tests whose event occurrences of ev, and ev,, both 2 2. Otherwise, their supports (sp in Definition 2 will be too small for evaluating TR validity. We set 17 0.1 in As for strict evaluations and take the upper integer A [T [ [Tn]]. Note that in the extreme situation where body and head event occurrences both = 2, stochastic noises still quite unlikely interfere with the evaluation of TR validity since 17 0.1 is striet. Only forward or reverse queries ((ev), EVR ) whose 8, = 8h. O = o or 8 = On. Ob = Sh. for (s, t) quadruples) are tested for better interpretability without sacrificing generality. We set 0 = 0.8 to distinguish between valid and invalid TRs. The fact that TRs whose gCg ~ 0.8 are of a tiny percentage of all tested TRs adds to the rationality of such metrics. Ablations where 0 0.7, 0.9 are in Appendix B.3.3
report the highest Fl in TR detection by tuning the global threshold 9th (defined in Section 3.2) after embedding the whole event set to achieve full evaluations. We also remove TRs whose Tg 0 for TR query because they account for most valid TRs but can hardly reflect the query difficulty. In NE's demonstration studies 4.3 4.4 we first use D14 to derive the global threshold 9t,, with the highest F1 in TR detection and then apply the same Ir. for evaluating NE's performance in D18. This setting better demonstrates NE's practicality.
In comparative studies with basclines 4.2
4.2 Comparisons of Learning Effectiveness
Table 1: Statistical results on ICEWS14, ICEWS18,, and GDELT
TR Detection (F1)
TR Query (r)
Embedding
D14
D18
GDELT
D14
D18
GDELT
TNTComplEx
0.26
0.18
0.08
0.08
0.08
0.01
DE-SimplE
0.22
0.20
0.09
0.09
TASTER
0.18
0.15
0.08
0.09
0.09
0.00
TeRo
0.43
0.64
0.16
0.08
0.08
0.01
BOXTE
0.40
0.40
0.18
0.08
0.08
0.01
ATISE
0.40
0.44
0.18
0.08
0.08
0.01
NE with g'(T)
0.78
0.79
0.48
0.85
0.83
0.83
NE with (T)
0.82
0.83
0.51
0.87
0.86
0.85
Performances Table 10 shows that NE with g(T) has an overwhelming advantage over baselines with g' (T), both in detecting valid TRs and querying the relative time on all evaluated datasets. The

---

excellent performance of NE with gT(T) indicates that (T) itself is valid and thus guarantees fair comparisons. is worth noting that NE intrinsically diffferent from all existing baselines because only NE can directly decode TR validity by q(r). In contrast, baselines can only decode event occurrences (t) from which to indirectly calculate TR validity (such as by g d7()).. Detailed results of precision and recall rates with error bars are reported in Appendix B.2
1.50
2.0
1.5
1.25
Event Score
Minimal
1.00
TR Score
0.75
0.50
0.25
Volid 1R
Ervale
9.2
Thresholk
0.
0.0
0.4
-400
-200
200
100
200
300
Maximal ge
Relative Time (Day)
Absolute Time (Day)
(a) Decoding distribution of NE
(b) g(T)s of NE
(c) Aligned f(t)s of models
Figure 3: Illustrations of NE or baselines
Discussion Figure ((a) shows NE's decoding distribution 9m min g(T) by each query's ground rET,
truth 9Cg = max ge(T). It can be observed that the decoded conserved local energy accurately reveals FET,
the TR validity, which enables NE to successfully distinguish between valid and invalid TRs, as demonstrated the case shown Figure 3(b) Table shows that baselines with g'(T) still perform poorly. This is mainly because their f(t)s do fill well. Specifically, Figure 3(c) illustrates that INTCCoppEx has much noise in its (t) compared to NE. The reason is that baseline models are generally designed to achieve good performance in the completion task and, therefore, over-apply the generalization capabilities of distributed representations, which hinders the fit of event occurrences.
4.3
NE's Superior Learning Capabilities
1.0
1.0
1.0
0.8
0.8
0.8
0.6
0.6
0.0
Number
(Day)
(a) TR detection by n
(b) TR query by n
(c) TR query by Tg
Figure 4: Grouped performances NE
Data Efficiency In Figure 4(a) and 4(b) group TRs by their number of relevant events. It shown that NE accurately detects valid TRs and reports correct rS with only two event pairs as positive samples. This performance is comparable humans, able to form temporally associative memories with minimal experience (Hudson et al. 1992; Bauer & Mandler 1989 Schaairo et al. 2017). Note that the maximum group in the test has 400, while we only show groups with 40 for display considerations.
7 &
Time Efficiency As explained in 3.3 NE's specific decoding function g(T) enables NE to retrieve TRs constant time complexity by vector computations. Calculating g g'(T) of classic embeddings, however, requires additional time complexity relevant to T, (the number of absolute time points).
Storage Efficiency addition to the data-efficient and time-efficient properties, NE is, in fact, also storage-efficient memory for TRs and event occurrences. Here is detailed analysis:
7

---

The storage of NE vectors, denoted as S(NE), can be calculated as follows: S(NE) = S(ev vector) S[time vector) 2 N d 64bit 2 d 64bit. In our experiments, used Lorc..LoggTenoor and N represents the number of event-type vectors. On the other hand, the storage of exact counting, denoted as S(CT), can calculated follows: S(CT) S(TR) + S(event) = N To logg(n/NNbbt + N (n/N) logz(? (T Jbit. Here, represents the number of all event occurrences. We reserved the storage accuracy of TR validity to effectively distinguish different values, resulting in approximately (og2(n/N) (bit for each TR validity (EVb, eUh, T).
For the ICEWS14 and ICEWS18 datasets, where 400, To 365, and n 90730, 468558, S(CT) 50295, 266631, we calculated the compression ratio 3(NE) of NE as 421 and 2336, respectively. This remarkable capability of NE can be attributed to the fact that it separately stores the information TR validities (et,, et;; 7) using event-type vectors and global time vector. By representing the common information of related TRs efficiently in memory. NE achieves a compression ratio that is approximately linear to the number of event types.
Flexibility In Figure 4(c) we group valid TRs by their golden TyS. NE is shown to be flexible for learning TRs with 7s varying broadly, comparable to humans with stable memory codes for various time intervals in the hippocampus (Mankin et al. 2012).
4.4 NE's Wide Potential Use
head
Look
body
Order
Absolute
300
Timestamps
250
300
(a) Case
(a) Event occurrences
hoad
Order
Look
Eat
Pay
Absolute (Day)
Order Look
Eat
(b) Case 2
(b) TR validity
Figure 5: Social event prediction
Figure 6: Personal decision making
Potential Use in Social Event Prediction In D18, NE successfully reports 21010 valid TRs with an F1 score of 0.83. The encoding stage takes around hour, while decoding only takes less than minute. Cases presented below and in Figure with additional cases available in Appendix B.4
(1) Case 1. Citizen (India) will Reject to Narendra Modi (events in day 23, 122, and 168) in around 87 days whenever Narendra Modi Appeal for diplomatic cooperation (such as policy support) to Citizen (India) (events in day 102, 200, and 264).
(2) Case 2. Russia will Meet at a 'third' location with Ukraine (events in day and 138) in around
136 days whenever Ukraine Use conventional military force Russia (events in day 161 and 288).
Since the TRs mined can generalize across time, the results above imply NE's potential use both reliable and interpretable event predictions urgently needed in the big data Zhao 2021).

---

Potential Use in Personal Decision Making Consider an intelligent machine that has visited restaurant four times, with the occurrence time of each event episode used as imput for NE, as shown in Figure (((a) After training all events, the decoded TR validity min g(T) is transformed linearly -eI, and demonstrated in Figure 6(b) Despite the recurrent TRs the slash that be set aside, valid TRs such as 'order dishes -about 10 minutes)-> have meals' are well distinguished from invalid ones such as "order dishes (about 5 minutes) -> look at the floor'.
Combining NE with front-end methods that take unstructured videos as input and output event items in the form of (ev, !), and with back-end methods that use the decoded valid TRs to guide decision- making. NE has the potential to aid intelligent machines in surviving in changing environments by generalizing from little experience, just as human beings (Goyal & Bengio 2022
Xue al., 2018
Scenarios As discussed in NE approximately reduces the required storage space from M to M in our experimental settings, where is the number of event types. Therefore, NE holds significant potential for applications in memory-constrained scenarios like the edge. This is important when M large, which is usual in the big-data era.
Potential Use in Memory-constraine
4.5
Ablation Studies
Here we demonstrate ablation studies of loss constants Cp. Cn and time vector w, while those for dimension and event type vector are shown in Appendix B.3
Table 2: NE on ICEWS14 in different C₂ and C, settings
C
TR Detection (F1)
TR Query (r)
Cp
0.4
0.2
0
-0.2
-0.4
0.4
0.2
-0.2
-0.4
0.78
0.80
0.82
0.81
0.80
0.86
0.87
0.87
0.87
0.87
0.8
0.64
0.67
0.79
0.80
0.80
0.85
0.86
0.86
0.87
0.86
0.6
0.29
0.40
0.68
0.79
0.79
0.44
0.82
0.86
0.86
0.85
0.4
0.18
0.31
0.49
0.76
0.79
0.12
0.35
0.84
0.84
0.81
0.2
0.53
0.19
0.27
0.71
0.78
0.27
0.05
0.28
0.79
0.71
Loss Constants Table ( shows that NE performs optimally when C, ===11 and Cn 0. In fact, as Cp approaches 1, the g(T) of perfect TRs (ge(r) = 1 = 1) is enforced to converge to its minimum 0. This global constant for all potential TRs in the embedding space allows g(T) to reveal TR validity better. In terms of Cn, setting it to O results negative samples occupying the largest embedding space. Since negative samples comprise most of all trained event samples, this setting improves the fit of negative samples and optimizes NE's performance.
Table 3: NE on GDELT with different WmazS
Wmar
10
50
100
200
400
600
800
TR Query (r)
0.15
0.93
0.92
0.85
0.85
0.85
0.85
0.85
0.85
TR Detection (F1)
0.22
0.45
0.55
0.56
0.54
0.53
0.53
0.51
0.53
Maximal Frequency Coefficient Table that NE performs optimally with different values of Wmars respectively, in the TR detection and query task.
Table 4: NE on the three datasets with increasing events, and with different distributions of (wx)
D14 (90730 events)
DI8 (468558 events)
GDELT (2278405 events)
{
linear
exponential
linear
exponential
linear
exponential
TR Query (r)
0.81
0.82
0.75
0.82
0.24
0.85

---

Frequency Distribution Table shows that the larger the dataset, the more exponential distribution To
(Wk (2 (ux))mu))
0, d 1) surpasses linear distribution (wi 2 2 k dxT,
0,1, 1) with the same parameters of d 400, Wmar 600. This suggests that real-world event occurrences depend more on low-frequency terms than high-frequency ones.
5 Related Work
Event Schema Induction In the natural language processing (NLP) field, a significant research focus is on inducing event schemas from text (Huang al.] 2016 Li et al. (021), including from language models (Dror et al. 2022). to support downstream applications such as search, question- answering, and recommendation (Guan al. 2022). These NLP methods aim organize known event regularities already given as priors for the extracting algorithm (such as extracting 'earthquake tsunami' from the sentence 'An earthquake causes tsunami.") and focus the schemas for use. In contrast, our tasks are designed to learn event regularities directly from experience without supervision. Specifically, the only prior models know is whether an event occurs, and models are required to detect valid TRs from all potential ones and report the correct relative time of valid TRs.
Temporal Rule Mining Various temporal rules are mined from event sets to reveal regularities in industry. security, healthcare, etc (Segura- (Segura-Delgado et al. 2020, Chen et al., 2007; Yoo & Shekhar 2008 Namaki (amaii et al., 2017). Although the search methods used discover event regularities directly from events without supervision, both the mined rules and source events are generally stored as symbolic representations in list form. In contrast, by applying event embeddings, NE is distributed and approximate memory for both TRs and event items. NE strikes a balance between storage efficiency and storage accuracy compared to exact counting, as detailedly discussed in 4.3
Within all embedding models of static and temporal knowledge graphs (Chen et al. 2020) Cai et al. 2022 Wang et al., 2020; Messner et al., 2022), three most related to NE. RotatE (Sun et al. 2019) represents each entity and relationship a complex vector to model relation patterns on knowledge graphs, and TeRo (Xu et al. 2020b) represents time rotation of entities to model time relation patterns temporal knowledge graphs. While both RotatE and TeRo introduce complex vectors for better completion performance, NE first explores using complex vectors for TR detection in events. In particular, the specific use of complex vectors in RotatE encodes inverse relations and in TeRo encodes asymmetric and reflexive relations. NE, instead, apply rotating complex unit vectors to encode time-translation symmetries of all potential TRs. IterE (Zhang et al. 2019) construct a decodable embedding model to discover rules for better knowledge graph completion performance. While we take functional inspiration from IterE that embedding models can jointly encode data and decode regularities, we focus on event data and define the new problems of TR detection and TR query. Specifically, while IterE focuses on discrete variables, NE focuses on the continuous variable of time that involves Fourier-lke like transformations.
Embedding Models of Structured Data
To summarize, TR detection and TR query focus on achieving human- like schema learning capabilities rather than pursuing better support for NLP applications like the event schema induction task. Meanwhile, NE leverages the advantages of distributed representations over symbolic ones of search methods in temporal rule mining and is distinct from existing embedding models of structured data.
6 Conclusion
We have developed NE which for the first time enables data-cfficient TR formation and time-efficient TR retrieval simply through embedding event samples. We have formally defined the tasks of TR detection and TR query to comprehensively evaluate the TR learning capabilities of embedding models. We have demonstrated NE's potential use in social event prediction, personal decision- making, and himan-like event intelligence.
scenarios. We hope that we have facilitated the development of
One limitation of NE is that when the vector dimension d is set much lower than the number of absolute time points T,, significant performance degradation of NE will occur as observed in the GDELT experiment. Future research is needed to improve this weakness. The privacy issues potentially brought about by TR detection and the causality of TRs should also be handled properly.
10

---

Acknowledgements
We sincerely thank Rong Zhao, Hao Zheng, Dahu Feng, Lukai Li, Hui Zeng, Wenhao Zhou, Yu Du. Songchen Ma, and Faqiang Liu for their valuable comments and encouragements. This work was supported by the National Nature Science Foundation of China (nos. 02088102, 61836004) and the National Key Research and Development Program of China (no. 2021 20212D0200300)
References
Bauer, P. J. and Mandler, J. M. One thing follows another: Effects of temporal structure on 1-to 2-year-olds' recall of events. Developmental psychology, 25(2):197, 1989.
Boschee, E.,, Lautenschlager, J., O' O'Brien, S., Shellman, S., Starz, J., and Ward, M. Icews coded event data. Harvard Dataverse, 12, 2015.
Bright, M., Meister, M. L.. Cruzado, N. A., Tiganj, Z., Buffalo, E. A., and Howard, M. W. A temporal record of the past with spectrum of time constants in the monkey entorhinal cortex. Proceedings of the National Academy of Sciences, 17(33):20274-20283 2020.
Cai, B., Xiang, Y., Gao, L., Zhang, H., Li, Y., and Li, J. Temporal knowledge graph completion: A survey. arXiv preprini arXiv:2201.088 082366, 2022.
Chaudhuri, R. and Fiete, I. Computational principles of memory. Nature neuroscience, 19(3): 394-403, 2016.
Chen, M., Chen, S.-C., and Shyu, M.-L.. Hierarchical temporal association mining for video event detection in video databases. In 2007 IEEE 23rd International Conference Data Engineering Workshop, pp. 137-145. IEEE, 2007.
Chen, X., Jia, S., and Xiang, Y. A review: Knowledge reasoning over knowledge graph. Expert Systems with Applications. 141:1 12948. 2020.
Dror, R., Wang, H., and Roth, D. Zero-shot on-the-fly event schema induction. arXiv preprint arXiv:2210.06254 2022.
Galárraga, L., Teffioudi, C., Hose, K., and Suchanek, M. Fast rule mining in ontological knowledge bases with amie +. The VLDB Journal, 24(6):707-730, 2015.
Ghosh, and Gilboa, A. What memory schema? a historical perspective on current neuroscience literature. Neuropsychologia, 53:104-114, 2014.
Goel, R., Kazemi, S. M., Brubaker, M., and Poupart, P. Diachronic embedding for temporal knowledge graph completion. In Proceedings of the AAAI Conference on Artificial Intelligence volume 34, pp. 398833989999 2020.
Goyal, A. and Bengio, Y. Inductive biases for deep learning of higher-level cognition. Proceedings of the Royal Society A, 478(2266):22110666. 2022.
Guan, S., Cheng, X., Bai, L.. Zhang. Li, Z.. Zeng, Y., Jin, X., and Guo. What event knowledge graph: A survey. IEEE Transactions on Knowledge and Data Engineering, 2022.
Han, Z., Chen, P., Ma, Y., and Tresp, V. xerte: Explainable reasoning on temporal knowledge graphs for forecasting future links. arXiv preprint arXiv:2012.15537, 2020.
Howard, M. W., MacDonald, C. J., Tiganj, Z., Shankar, K. H., Du, Q., Hasselmo, M. E., and Eichenbaum, H. A unified mathematical framework for coding time, space, and sequences in the hippocampal region. Journal of Neuroscience, 34(13):4692-4007 2014.
Huang. I.,, Cassidy, T., Feng, X., Ji, H., Voss, C., Han, J., and Sil, A. Liberal event extraction and event schema induction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 258-268, 2016.
Hudson, J. A., Fivush, R., and Kuebli, J. Scripts and episodes: The development of event memory. Applied Cognitive Psychology, 6(6):483-505 1992.

---

Jin, W., Jiang, H., Qu, M.. Chen, T., Zhang, C., Szekely, P., and Ren, X. Recurrent event network: Global structure inference over temporal knowledge graph. 2019.
Lacroix, T., Obozinski, G., and Usunier, N. Tensor decompositions for temporal knowledge base completion. arXiv preprint arXiv:2004.04920 2020.
Leblay, J. and Chekol, M. W. Deriving validity time in knowledge graph. In Companion Proceedings of the The Web Conference 2018, pp- 1771-1776, 2018.
Lectaru, K. and Schrodt, P. A. Gdelt: Global data on events, location, and tone, 1979-2012. In ISA annual convention, volume pp. 1-49. Citeseer, 2013.
Li, M., Li, S., Wang, Z., Huang, L.., Cho, K., Ji, H., Han, J., and Voss, C. The future is not dinedimensional: Complex event schema induction by graph modeling for event prediction. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing, pp. 5203 2013-221,,40 2021.
Mankin, E. A., Sparks, F. T., Slayyeh, B., Sutherland, R. J., Leutgeb, S., and Leutgeb, J. K. Neuronal code for extended time in the hippocampus. Proceedings of the National Academy of Sciences, (19(47))19462-1146) 2012.
McClelland, J. L., McNaughton, B. L., and O' O'Reilly, R. C. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
Messner, J., Abboud, R., and Ceylan, I. Temporal knowledge graph completion using box embeddings. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 7779-7787, 2022.
Namaki, H., Wu, Y., Song, Q.. Lin, P., and Ge, T. Discovering graph temporal association rules. In Proceedings of the 2017 ACM on Conference Information and Knowledge Management, pp. 1697-1706, 2017.
Noether, E. Invariante variationsprobleme. Nachrichten von der Gesellschaft der Wissenschaften zN Göttiigen, Muthermaticch-Plyyikkiiis Klasse, 1918:235-257, 1918. URL http://eudml. org/ doc//9024
Pudhiyidath, A., Roome, H. E., Coughlin, C., Nguyen, K. V., and Preston, A. R. Developmental differences temporal schema acquisition impact reasoning decisions. Cognitive neuropsychology. 37(1-2):25-45, 2020.
Schapiro, A. C., Turk-Browne, N. B., Botvinick. M. M., and Norman, K. A. Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning. Philosophical Transactions of the Royal Society B: Biological Sciences, 372( (11(1711):2016004, 60049, 2017.
Segura-Delgado, A., Gacto, M. J., Alcalá, R., and Alcalá-Fdez J. Temporal association rule mining: An overview considering the time variable as an integral implied component. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(4):e 2020.
Sun, 7.., Deng, Z.-H., Nie, J.-Y., and Tang, J. Rotate: Knowledge graph embedding by relational rotation in complex space, arXiv preprint urXiv:/902) 10197, 2019.
Tanaka, H. and Kunin, D. Noether's s learning dynamics: Role of symmetry breaking in neural networks. Advances in Neural Information Processing Systems, 34:25646 -56660, 2021.
Trivedi, R., Dai, H., Wang, Y., and Song, L. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In international conference on machine learning, pp. 3462-3471. PMLR, 2017.
Tsao, A., Sugar, J., Lu, L., Wang, C., Knierim, J. J., Moser, M.-.,, and Moser, E. I. Integrating time from experience in the lateral entorhinal cortex. Nature, 561(772)):5776) 2018.
Wang, J., Zhang, W., Chen, X., Lei, J., and Lai, 3drte: 3d rotation embedding in temporal knowledge graph. IEEE Access, 12277115-20752) 2020.
12

---

Wang, X., Lyu, S., Wang, X., Wu, X., and Chen, H. Temporal knowledge graph embedding via sparse transfer matrix. Information Sciences, 623:56-69, 2023.
Xiao, C., Sun, L., and Ji, Temporal knowledge graph incremental construction model for recommendation. In Asia-pacific web (apweb) and web-age information management (waim) joint international conference on web and big data, pp. 352-359. Springer, 2020.
Xu, C.., Nayyeri, M., Alkhoury, F., Yazdi, H., and Lehmann, J. Temporal knowledge graph completion based on time series gaussian embedding. In International Semantic Web Conference, pp. 654-671 Springer, 2020a.
Xu, C., Nayyeri, M., Alkhoury, F., Yazdi, H. S., and Lehmann, J. Tero: A time-aware knowledge graph embedding via temporal rotation. arXiv preprint arXiv:2010.01029, 2020b.
Xue, J.-R., Fang, J.-W., and Zhang, P. survey of scene understanding by event reasoning in autonomous driving. Machine Intelligence Research, (5(3):249-266 2018.
Yang. Y.. Wei, Z., Chen, Q and Wu, L. Using external knowledge for financial event prediction based on graph neural networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 2161-2166 2019.
Yoo, and Shekhar, S. Similarity-profiiee temporal association mining. IEEE Transactions on Knowledge and Data Engineering, 28) 147-1161, 2008.
Zhang, W., Paudel, B., Wang, L., Chen, J., Zhu, H., Zhang, W., Bernstein, A., and Chen, H. Iteratively learning embeddings and rules for knowledge graph reasoning. In The World Wide Web Conference, pp. 2366-236777 2377, 2019.
Zhao, L. Event prediction in the big data era: A systematic survey. ACM Computing Surveys (CSUR). 54(5):1-37, 2021.
13

---

Appendix
A Theoretical Illustrations
A.1 Interdisciplinary Correspondences
A.1.1 Physical Correspondence
Considering each event type as a partiele, the event embedding q(t: ev) can be viewed as its gen- eralized coordinate at time in the d-dimensional complex embedding space. Suppose that each two particles eva, e0,, connected by spring with the force linear their distance, their poten- tial energy can then be expressed as (l))(() qu (t + I)l?? where T is a parameter. As q(t; ev) changes with time, its kinetic energy also changes and can be viewed as being driven by external nonconservative forces. Using the extended Noether's theorem, we can see that the work of external forces and the kinetic energy offset, leading to a conserved quantity of time-translation symmetry that only includes the potential energy instead of the total energy. The inductive bias of Noether Em- bedding exactly enforces that such local potential energies are innately conserved, which means that Il (fx) qn (t& + Ty) = qm (f,) qn (ts + Ty)"l Veom, evn, E P, tg. tg E Ta. T, E Tr, where P. Ta, T, refer the set of event types, absolute time points, and relative time points, respectively.
A.1.2 Biological Correspondence
The formation of TRs relies on two central functions. namely measurement and integration. The measurement function involves comparing the temporal distance between each two events. It is claimed that Laplace transformation exists in the hippocampus for representing time (Howard et al. 2014). Specifically, the population of temporal context cells (or referred to as ramping cells (ssao et al.. 2018)) in the lateral entorhinal cortex is discovered to code time with a variety of rate constants (Bright et al. 2020p. r(f) in NE functionally corresponds to such cell population, which stores the rate distribution in vector of rt). The integration function aggregates event pairs separated in time to form TRs. Statistical learning for schema formation is reported to occur in the pathway connecting EC to CA1 in the hippocampus (Schapiro et al. 2017). In NE, we achieve the same function through the transsaannlttio symmetry introduced by r rt)): r(t r(t+ T) = rt)) rT)), r()) ot()) 1, vt T..
A.2 Revelation of TR Validity
A.2.1 Effectiveness in TR Query
NE an effective method for TR query due to its distributed storage of cross-correlations.
The score function f() describes the occurrences of each event type, where its value is mapped to Cp for time points implying event occurrences and Cn for those that do not. The support sp(r) of two event types etb, eon, can be then approximated by calculating the time-lagged cross-correlation Rs (7) Ex,treT. f.. +t+++++7 r) after training convergence.
Denote F(w) as the Fourier expansion of J((). Awaring that in NE, = Real(u e can see that the encoding stage enables d-dimensional Fourier-like expansion for each ft). The time vector w provides the expansion basis and u stores the coefficients as F(w)s.
By Fourier expansions, the corresponding correlation R, (T) = f 400 fo(l)Sa(t + r)dl =
1t Fx()))((eee dw. In NE, g(T) = ||/4, us or(r)) = 2 2 Real (wa o u. o e
Therefore, each g(T) reveals exactly the fitted time-lagged cross-correlation and is thus proportional to the support sp(r) of TR. This guarantees NE's effectiveness TR query.
A.2.2 Effectiveness in TR Detection
Notations Within all event samples trained for a potential TR, suppose that m event pairs share the same relative time of 7, where mi pairs are both positive both negative samples. In each remaining my m mi pair, one is positive sample while the other is a negative one. Denote that M {1. 2, ..., m) Mi, {dz, ig, ..., im; }. Ml, {j1.f2. .... Jmy]. }. There exists such least upper bound 17 that ( (t; ev) - C)2 < for the score functions of all 2m samples, where C' = Cp. Cn.
[1
14

---

Denote that a; = (cos(Wit;)) cos(wyts),.... cos cos(wat;) sin(wits), sin(wpt))) sin(wati))) M. where is the dimension NE vectors, t, is time of the ith body event in sample pairs. Denote that in the decoding function, иλд UA r(r) a iB3 where 3 are both real vectors, and x
(o11 o.2, Odd B2, ....... Ba)² Denote that 61 = mar(cco((n)) æ))² 02
min (eon((es())) (j x;))
JeM,
Using the trigonometric inequality, we can derive the following conclusions:
Theorem
Within the mi sample pairs, if c1 > 0, then g(T) < c
Theorem 2 Within the my sample pairs, if C2 0 and ICp - Cnl > 2//,, then g(T) >
Implications d, da w, Cp. Cn jointly result in the probability distributions of C1. C2. where c1 C2 > is generally guaranteed by experimental settings. Two conclusions can then be drawn from the two theorems. (1) Convergence. We
from theorem I that g(r) 0 as 17 - 0. This implies that the (()) of perfect TRs (gc(r) = 1, 1) will converge to its minimum of 0. (2) Competition. Since g(T) is invariant to t, comparing these of well-trained sample pairs for the value of g()), generally affected by the ratio mi These two conclusions, along with the fact that (()) is proportional to the support sp(T) (as illustrated in 1 jointly make g(T) reveal the TR validity ge(r) and thus guarantees NE's effectiveness in TR detection.
theorems also tells about the competing effect
worth noting that d is generally set as d > Ta to control the values of C1 C2. where Ta is the number of absolute time points of the whole event set. Otherwise, NE's TR detection performance will be interfered. For example, if d < To. then d < m in most cases. It will thus be very likely that C1 that theorem can be applied NE.
B Experimental Supplements
B.1 Training Details
All models are trained for 100 epochs on each dataset using the Adagrad optimizer (with a learning rate of 0.01) and the StepLR learning rate scheduler (with step size of 10 and gamma of 0.9). The experiments are conducted on single GPU (GeForce RTX 3090). The hyper-parameters of NE are fine-tuned using grid search achieve relatively optimal results.
B.2 Main Results with Error Bars
To ensure the reliability of the results, the experiments are repeated three times, and the error bars derived accordingly. Table show that the main results are quite stable with small error bars. Note that the precision and recall rates in corresoon exactly the Fl scores in Table The reason why the recall rate of NE is lower than that of TASTER is that we report the highest Fl score of each model in comparative studies by tuning their respective global threshold, denoted as gen As the Fl score is calculated as the harmonic mean of precision and recall rates, TASTER achieves its highest Fl score by reporting many false positives, resulting in a relatively high recall rate but an extremely low precision rate.
B.3 Additional Ablations
Unless otherwise specified, the experiments below adopt the original parameter settings as described in the main text.
B.3.1 Normalization of Event Type Vectors
Table 7 /emonstrates that normalized event type vectors slightly outperform unnormalized ones in the TR detection task.
15

---

Table 5: TR detection by Fl scores and TR query by confidence ratios
TR Defection (F1)
TR Query (r)
Embedding
D14
D18
GDELT
D14
D18
GDELT
TNTComplEx
0.26110.01
0.18
0.08
0.08+1000
0.08
0.01
DE-SimplE
0.22+0.00
0.20:10.00
0.0910.00
0.09=0.00
TASTER
0.1810.00
0.15+0.00
0.08
0.09-10.00
0.09=0.00
0.00
TeRo
0.43+0.02
0.64
0.1610.00
0.08+0.00
0.01 0.08+0.00
BoxTE
0,40=10.01
0.40=t0.02
0.18#0.01
0.08
0.08=0.00
0.01
ATISE
0.40##0.01
0.44
0.18#0.01
0.08
0.08=0.00
0.01
with o' g'(r) 0.7810000
0.79
0.48
0.85100.00
0.83
0.83:10.00
NE with (T)
0.82+0.00
0.83+0.00
0.51
0.87+0.00
0.86=0.00
0.85
Table 6: TR Detection by precision and recall rates
Precision DI8
Recall DI8
Embedding
D14
GDELT
D14
GDELT
TNTComplEx
0.22:+0.01
0.11
0.04
0.33+0.02
0.50=0.001
1.00=#.001
DE-Simpp DE-SimplE
0.16##0.0
0.14#0.00
0.35+00.3
0.33#0.00
TASTER
0.10#0.00
0.08+00.00
0.04
0.99+00001
0.99=0.01
0.97 0.03
TeRo
0.51+00.04
0.91
0.20+10.44
0.37+10.02
0.49+00.00
0.14
BoxTE
0.40+-0.22
0.39++0.05
0.15+0.01
0.41+00.2
0.41+0002
0.22++0.03
ATISE
0.35++0.01
0.49++0.03
0.15+0.01
0.47+00.01
0.40++000
0.21+0001
NE with g'(r) (T)
0.99++0.00
0.900
0.90++0.00
0.64+00.0
0.66+00.00
0.32+0.00
NE with (T))
0.99++0.00
0.99++0.00
0.83+0.00
0.70++0.0
0.72+0.00
0.37++0.00
B.3.2 Dimension of Embedding Vectors
Table 8 and demonstrate that d scarcely affects the performance of NE as long as is more than some certain value. It is worth noting that NE's detection performance in GDELT may be further improved with larger values of ds, as illustrated in A.2.2 because Ta = 2976 in GDELT, which is much larger than the tested dimensions.
B.3.3 Threshold for Valid TRs
In the main text, the threshold for distinguishing valid and invalid TRs is chosen as 0 = 0.8. Here we report NE's results on D14 with varying Os in Table I0] is shown that NE still has an overwhelming advantage over all baselines.
B.4 Mined TRs
Additional cases of mined TRs are shown in Table 11 as below.
16

---

Table 7: F1 of NE on ICEWS14 with different type vectors
Normalized
Unnormalized
F1
0.82
0.80
Table 8: NE on GDELT with different dimensions
d
100
200
300
400
500
TR Query (r)
0.83
0.84
0.85
0.85
0.84
TR Detection (F1)
0.22
0.45
0.50
0.51
0.51
Table 9: NE ICEWS14 with different dimensions
50
100
200
400
600
800
TR Query (r)
0.81
0.86
0.87
0.87
0.87
0.87
TR Detection (F1)
0.29
0.55
0.76
0.82
0.83
0.83
Table 10: Statistical results with different thresholds on ICEWS14
Detection(F1)
Query(r)
Embedding
0=0.7
6=0.8
0=0.9
0=0.7
0=0.8
6=0.9
TNTComplEx
0.29
0.26
0.25
0.07
0.08
0.06
DE-SimplE
0.28
0.22
0.21
0.09
0.09
0.21
TASTER
0.28
0.18
0.17
0.09
0.09
0.08
TeRo
0.35
0.43
0.40
0.08
0.08
0,06
BoXTE
0.38
0.40
0.41
0.08
0.18
0.06
ATISE
0.28
0.35
0.33
0.08
0.08
0.06
NE with (((
0.63
0.78
0.80
0.81
0.85
0.86
NE with (())
0.71
0.82
0.84
0.87
0.87
0.87
17

---

Table 1I: Cases of Mined TRs from ICEWSI8
Body Event
Head Event
ge
sp
9b
Pb
Ob
S/.
Ph
Oh
Threaten
Make em-
North Korea
with
South
North Korea
South Korea
pathetic comment
87
0.86
3
military
Korea
force
Reduce or break
Foreign Affairs (United States)
Foreign Affairs (United States)
Bahrain
diplo-
Bahrain
Consult
0.86
3
matic
relations
Host
Yoshitaka
Yoshitaka
Make visit
Japan
Japan
0
7
visit
Shindo
Shindo
Sign
Armed
Armed
Sign formal
Government (Syria)
Govermment (Syria)
formal
Rebel
Rebel
5
agreement
(Syria)
(Syria)
agreement
Defy norms, law
Protester (Thailand)
Make statement
Protester (Thailand)
Military (Thailand)
Thailand
129
0.86
3
Make pes- simistic comment
Host visit
Yasuo Fukuda
China
Japan
China
148
0.83
5
Court
Express
Citizen (India)
Citizen (India)
Villager (India)
Judge
intent to cooperate
Accuse
132
0.83
5
(India)
Appeal for diplo- matic coopera- tion (such as policy support)
Engage diplo- matic coopera- tion
Kim
South Korea
South Korea
Iran
27
0.86
3
Jong-Un
Make pes- simistic comment
Sign formal agreement
South Korea
North Korea
South Korea
Canada
68
0.86
3
Appeal for diplo-
Express intent settle dispute
matic
South Korea
China
coopera-
Malaysia
China
111
0.86
3
tion (such policy support)
Mohammad Javad Zarif
Host visit
Massoud Barzani
Massoud Barzani
Iraq
Consult
57
0.86
3
Appeal for diplo- matic coopera- tion (such as policy
Fight with small arms and light weapons
Citizen (Thailand)
Japan
Thailand
Thailand
118
0.86
3
support)
18


Reviews:
[{"review_id": "2EhEB38YLV", "reviewer": "Reviewer_bn9B", "summary": "In this paper, the authors introduce the concept of temporal regularities (TR), which indicates temporal associations invariant to time shifts between events. The authors claim that existing models are lack of the TR learning capability. Based on this idea, the authors define two tasks, TR detection and TR query, as well as their evaluation metrics. They also further develop a new framework to learn a set of event representations regularized by fixed time embeddings. Experiments on several benchmark datasets demonstrate the effectiveness of the proposed framework and its superiority on TR learning compared to existing methods.", "questions": "* In Line 166-168, each (s,p,o) combination corresponds to a specific event embedding. From the authors' code, it seems like the authors have tried different embedding strategy, such as encoding s, p and o separately, or encoding (s,p,o) as a whole. What's your choice among these different implementations?\n* In formula 7, the conserved local energy is only relevant two events. If the relative time is conditioned on other events, could this framework handle it?\n* I'm interested how much have the model learnt from statistic prior. Have you tried to compute the averaged relative time for every event pair in the training set and then use such averaged relative time for testing?\n", "limitations": "The authors have adequately address the limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The proposed temporal regularity problem is important and many existing models are lack of such TR learning capabilities.\n* The proposed TR detection and TR query tasks are well-designed. The corresponding evaluation metrics are also reasonable.\n* The proposed solution is simple and effective. Experiments demonstrate significant improvement on TR compared to previous methods.", "weaknesses": "* The event embedding implementation part is not very clear. Details are insufficient for reimplementation.\n* Font sizes in Figure 5 and 6 are too tiny.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for spending valuable time reviewing our manuscript and providing insightful comments. We have improved our paper accordingly and our responses are as below.\n\n**From ‘Weaknesses’**\n\n**Q1**: ‘The event embedding implementation part is not very clear. Details are insufficient for reimplementation.’\n\n**A1**: Original details of model implementation include: the determination of hyperparameters (in Section 4.1) and the training details (in Appendix C.1). We have added justifications of how NE decoding relates to TR detection and TR query (in Section 3.2) in the revised paper. Specifically, $\\mathop{\\min}\\limits _{\\tau\\in\\mathbb{T} _r} g(\\tau)$ is computed, which is compared with a global threshold $g _{th}$ to decide whether a potential TR is valid or not (for TR detection). For a valid TR, the $\\tau'$ which minimizes $g(\\tau), \\tau \\in \\mathbb{T} _r$ is selected as the model output of the relative time (for TR query).\n\n**Q2**: ‘Font sizes in Figure 5 and 6 are too tiny.’\n\n**A2**: We have taken note of this feedback and have made the necessary changes by enlarging the font sizes in Figure 5 and 6 in the revised paper.\n\n**From ‘Questions’**\n\n**Q1**: ‘From the authors' code, it seems like the authors have tried different embedding strategy, such as encoding s, p and o separately, or encoding (s,p,o) as a whole. What's your choice among these different implementations?’\n\n**A1**: We appreciate the reviewer's careful observation. Our choice of the current form of NE is based on its ability to fit large datasets effectively while still maintaining the efficient learning capability. We have found that encoding s, p, and o separately leads to a higher loss after training convergence. This reduces the capacity of NE for large datasets and subsequently affects its performance in both TR detection and TR query. Making the model smaller while maintaining effectiveness is an area that can be explored in future work.\n\n**Q2**: ‘If the relative time is conditioned on other events, could this framework handle it?’\n\n**A2**: One advantage of NE is exactly its ability to efficiently store large amounts of interlinked TRs. We attribute such a large capacity of NE storage to its Fourier-like representations. Specifically, the score function $f(t;ev)=\\sum _{j=1}^d Real(\\pmb{u} \\circ e^{i \\pmb{\\omega} t}) _j$ and the decoding function $g(\\tau;ev _b,ev _h)=2-2\\sum _{j=1}^d Real(\\overline{\\pmb{u} _b} \\circ \\pmb{u} _h \\circ e^{i \\pmb{\\omega} \\tau}) _j\\in [0,4]$ can be viewed as Fourier-like expansions. The global time vector $\\pmb{\\omega}$ provides the expansion basis, while the event type vectors $\\pmb{u}$s store the coefficients for $f(t)$ (revealing event occurrence) and compose $\\overline{\\pmb{u} _b} \\circ \\pmb{u} _h$ as the coefficients for $g(\\tau)$ (revealing TR validity).\n\n**Q3**: ‘I'm interested how much have the model learnt from statistic prior. Have you tried to compute the averaged relative time for every event pair in the training set and then use such averaged relative time for testing?’\n\n**A3**: While we have not specifically explored this setting, we have demonstrated the flexibility of NE through a grouped experiment. We have grouped valid TRs based on their golden relative time and showcased NE's performance in TR query. The results in Figure 4 (c) indicate that NE performs consistently well in learning TRs with varying $\\tau$s. We have not tested TR detection using grouped $\\tau$s as the golden relative time for invalid TRs does not hold meaning.\n\nOnce again, we would like to express our gratitude to the reviewer for the valuable feedback, which has helped us further improve our manuscript."}, {"review_id": "pxO1qv2RCV", "reviewer": "Reviewer_w2A9", "summary": "This paper introduces Noether Embedding (NE), a new model for efficient learning of temporal regularities with event embeddings. Experiments conducted on three datasets show the superior performance of this work compared to classic embeddings in detecting valid TRs and querying TR intervals.\n", "questions": "1. Table 2 in the Appendix shows that NE has extremely high accuracy, while the recall rate is not the highest. May I know the reason for this phenomenon?\n2. Could you further explain why NE has an overwhelming advantage over all baselines?\n", "limitations": "The author proposes the limitations of NE and solutions in terms of storage efficiency in lines 307-311. \n1. In the future, methods will be explored to store event occurrences and time patterns in different regions to improve the storage efficiency of NE. \n2. Future research will explore methods to compose 1-1 TRs into graphical and hierarchical event schemas and combine NE with deep learning and reinforcement learning methods. \n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors introduce Noether Embedding (NE), a new model that enables the data-efficient formation and rapid retrieval of temporal regularities simply through embedding each event sample. NE possesses the intrinsic time-translation symmetries of TRs, which facilitates TR encoding insensitive to sample size and TR retrieval in constant time complexity. This is a novel approach that has not been explored in previous works.\n2. The authors formally define complementary problems of TR detection and TR query, formulate their evaluation metrics, and evaluate NE on classic ICEWS14, ICEWS18, and GDELT datasets. This is a rigorous evaluation of the proposed model and provides evidence of its superior performance compared to classic embeddings with additional calculation efforts.\n3. The paper is well-written and clear, with a concise abstract and introduction that provide a good overview of the problem and the proposed solution. The authors also provide detailed explanations of the model and the evaluation metrics.\n", "weaknesses": "1. In Table 2 of the Appendix, the recall rate of NE is lower than that of TASTER. It would be better if the reason could be explained.\n2. It should be further explained why gc(τ) in line 79 has different input from gc(tr) in formula 3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for spending valuable time reviewing our manuscript and providing insightful comments. We have improved our paper accordingly and our responses are as below.\n\n**From ‘Weaknesses’**\n\n**Q1**: ‘In Table 2 of the Appendix, the recall rate of NE is lower than that of TASTER. It would be better if the reason could be explained.’\n\n**A1**: The reason is that we report the highest F1 score of each model in comparative studies by tuning their respective global threshold, denoted as $g _{th}$. As the F1 score is calculated using the formula $F1 = \\frac{2 * precision * recall}{precision + recall}$, TASTER achieves its highest F1 score by reporting many false positives, resulting in a relatively high recall rate but an extremely low precision rate. We have included this explanation in the revised Appendix and emphasized the ‘highest F1 score’ evaluation in the revised paper.\n\n**Q2**: ‘It should be further explained why gc(τ) in line 79 has different input from gc(tr) in formula 3.’\n\n**A2**: We would like to clarify that they refer to the same gc but with different emphases. On the one hand, the $gc(tr)$ in Formula 3 represents the definition of global confidence for each tr. On the other hand, since $tr: (ev _b, ev _h, \\bigtriangleup) = (ev _b, ev _h, \\tau, \\eta)$, we have $gc(tr)=gc(ev _b, ev _h, \\tau, \\eta)= gc(\\tau; ev _b, ev _h, \\eta)$. Therefore, the $gc(\\tau)$ in line 79 emphasizes the fact that $gc$ can be expressed as a function of $\\tau$ with fixed $ev _b, ev _h, \\eta$. We have clarified this further in the revised paper.\n\n**From ‘Questions’**\n\n**Q1**: ‘Table 2 in the Appendix shows that NE has extremely high accuracy, while the recall rate is not the highest. May I know the reason for this phenomenon?’\n\n**A1**: The reason is that we report the highest F1 score of each model in comparative studies by tuning their respective global threshold, denoted as $g _{th}$. As the F1 score is calculated using the formula $F1 = \\frac{2 * precision * recall}{precision + recall}$, NE achieves its highest F1 score by reporting few false positives, resulting in a relatively high precision rate but a relatively low recall rate. We have emphasized the ‘highest F1 score’ evaluation in the revised paper.\n\n**Q2**: ‘Could you further explain why NE has an overwhelming advantage over all baselines?’\n\n**A2**: Firstly, the 'efficient' learning capability is unique to NE, which is fundamental and qualitative, requiring no comparative experiments to demonstrate. Specifically, only NE can encode TRs from limited event items (as shown in Figure 4) and rapidly retrieve TRs (by applying $g$). We attribute this mainly to three structural biases inspired by Noether's theorem: (1) the event embedding $\\pmb{q}(t;ev)$ should be constructed to make each local energy $g$ invariant to $t$; (2) the training loss should be constructed to make the value of $g$ approximate TR validity; (3) we should use local energy $g$ as the decoding function. Without such structural biases, baseline embeddings can not learn TRs efficiently. \n\nWhen setting aside the 'efficient' requirement by added with the same interface $g'$, baseline models still can not learn TRs accurately as compared to NE, as demonstrated by the comparative experiments. The main reason is that baseline models over-apply the generalization capabilities of distributed representations, which hinders the fit of event occurrences, as discussed in Section 4.2 with Figure 3.\n\nOnce again, we would like to express our gratitude to the reviewer for the valuable feedback, which has helped us further improve our manuscript."}, {"review_id": "zJgme5RyCZ", "reviewer": "Reviewer_tTxw", "summary": "This paper introduced a new task, $\\textit{temporal regularity mining}$,  and proposed a Noether Embedding  to rapidly retrieval TR.", "questions": "Please see the Weaknesses.", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The argued temporal regularities sound interesting.\n- Good writing.", "weaknesses": "- The proposed temporal regularity mining is not a new task with a new paradigm, which the existing methods cannot well tackle.  I think it is more like an extended task based on event embedding using temporal knowledge graph data.\n- All existing event embedding methods can be used to tackle the proposed TR mining task. However, there is no discussion to clarify why these existing methods cannot well deal with this new task.\n- Mining general TR sounds interesting and useful, but the mined TR is not general. In other words, it is more like a temporal association between events. For example, as shown in the supplementary material, (China, Appeal for diplomatic cooperation, Malaysia) $\\rightarrow$ (South Korea, Express intent to settle dispute, China). An ideal TR should be no matter who does the body event, and then the head body will occur soon, which is invariant to the time.\n- The temporal range between the body and head events is large, from several days to several years. So how to set the $\\tau, \\eta$?  Please discuss this detail.\n- More experiments on traditional tasks in event embedding, such as event prediction, should be conducted to further demonstrate the effectiveness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for spending valuable time reviewing our manuscript. However, there do exist many factual errors, which are justified below.\n\n**From ‘Weaknesses’**\n\n**Q1**: ‘The proposed temporal regularity mining is not a new task with a new paradigm’\n\n**A1**: To our best knowledge, the problem is new. Our main problem is how to enable event embeddings with an efficient TR learning capability, which is distinct from those in the temporal rule mining field. Specifically, temporal rule mining is typically studied for practical applications, aiming to uncover event regularities in specific domains [1] [2] [3]. Our goal, instead, is to advance the representation learning field by enabling embedding models to efficiently learn the atomic structure (TR) of event schemas, similar to how humans learn [4] [5]. We, therefore, develop NE with such a novel capability and propose two complementary tasks for evaluating embedding capabilities. Our TR tasks are defined with evaluation metrics borrowed from the rule mining field [6] only to guarantee fair and reasonable evaluations. \n\n**Q2**: ‘I think it is more like an extended task based on event embedding using temporal knowledge graph data.’\n\n**A2**: We would like to clarify that we use temporal knowledge graph data because they are classic and authoritative, allowing for fair evaluations. However, our problem is targeted at the most basic form of events $(ev, t)$, which serves as the foundation for our definition of TR.  It is important to note that both of our proposed tasks and method can easily generalize to arbitrary forms of structured events, and temporal knowledge graph is just a special case. For example, by setting each (s,p,o) as an event type, our tasks and NE method can handle temporal knowledge graph data in the form of (s,p,o,t). Similarly, by setting (s,p) as an event type, our tasks and NE method can handle data in the form of (s,p,t), and so on.\n\n**Q3**: ‘All existing event embedding methods can be used to tackle the proposed TR mining task’\n\n**A3**: Existing embeddings, indeed, cannot tackle the proposed tasks in two levels. Firstly, the ‘efficient’ learning capability is unique to NE and not present in existing embeddings. We attribute this unique capability of NE to its structural biases inspired by Noether’s theorem. Specifically, (1) the event embedding $\\pmb{q}(t;ev)$ is constructed to make local energies $g$s invariant to $t$; (2) the training loss is constructed to make the value of $g$ approximate TR validity; (3) the local energy $g$ is used as the decoding function. Secondly, even when setting aside the ‘efficient’ requirement, existing embeddings still cannot learn TRs accurately, as demonstrated by experiments in Table 1 and evaluated by the proposed TR detection and query tasks. This is primarily because they over-apply the generalization capabilities of distributed representations, which hinders the fit of event occurrences, as discussed in Section 4.2. \n\n**Q4**: ‘the mined TR is not general’\n\n**A4**: We would like to clarify that NE can indeed learn ‘general’ TRs when we set (s,p,o,t) with the same p (predicate) to denote an event type, rather than with the same (s,p,o) as set in the paper. This further proves the wide range of potential applications for NE, and we appreciate the reviewer for suggesting this. We have emphasized this potential in the revised paper.\n\n**Q5**: ‘how to set the $\\tau, \\eta$? Please discuss this detail’\n\n**A5**: We have provided detailed explanations in Section 3.2 and Section 4.1 on how to set the values of $\\tau$ and $\\eta$. In our experiments, $\\tau$ is traversed through set $\\mathbb{T} _r$ of the relative time points such as $\\mathbb{T} _r: \\{-\\tau _{max},...,0, ..., \\tau _{max}\\}$ to plot the decoding results. We set $\\tau _{max}=T _a-1$. As for $\\eta$, we set it to 0.1 in $\\bigtriangleup$s for strict evaluations and take the upper integer $\\bigtriangleup=[\\tau-\\lceil \\tau\\eta \\rceil, \\tau+\\lceil\\tau\\eta\\rceil]$. It is important to note that even in extreme situations where both body and head event occurrences are equal to 2, stochastic noises are still unlikely to interfere with the evaluation of TR validity since $\\eta=0.1$ is strict.\n\n**Q6**: ‘More experiments on traditional tasks in event embedding, such as event prediction, should be conducted to further demonstrate the effectiveness of the proposed method’\n\n**A6**: We appreciate the reviewer for making such a suggestion. However, we have already shown that NE is the first efficient TR learner with event embeddings. Specifically, only NE can encode TRs from limited event items (as shown in Figure 4) and rapidly retrieve TRs (by applying $g(\\tau)$). Such a uniqueness is fundamental and qualitative, requiring no comparative experiments to demonstrate. Our comparative experiments have further demonstrated NE's superiority over existing embeddings when setting aside the 'efficient' requirement to only learn TRs accurately. Therefore, we believe that our experiments are sufficient to support the effectiveness of NE.\n\nThank you for your valuable feedback, and we have made sure to address the issues raised in your review.\n\n**References**\n\n[1] Segura‐Delgado A, Gacto M J, Alcalá R, et al. Temporal association rule mining: An overview considering the time variable as an integral or implied component. DMKD, 2020.\n\n[2] Chen M, Chen S C, Shyu M L. Hierarchical temporal association mining for video event detection in video databases. IEEE, 2007.\n\n[3] Yoo J S, Shekhar S. Similarity-profiled temporal association mining. IEEE, 2008.\n\n[4] Pudhiyidath A, Roome H E, Coughlin C, et al. Developmental differences in temporal schema acquisition impact reasoning decisions. Cognitive Neuropsychology, 2020.\n\n[5] Chaudhuri R, Fiete I. Computational principles of memory. Nature neuroscience, 2016.\n\n[6] Galárraga L, Teflioudi C, Hose K, et al. Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal, 2015."}, {"review_id": "o0p5Z1fQN7", "reviewer": "Reviewer_Uoii", "summary": "This paper defined the complementary problems of TR detection and TR query, formulated their evaluation metrics, and adopted classic datasets for evaluations. Towards the TR problem, this paper proposed Noether Embedding (NE), which for the first time, enabled both the data-efficient formation and rapid retrieval of TRs simply through embedding each event sample. ", "questions": "1) The paper emphasized \"distributed\" representation was an advantage of NE. Is it a general property of knowledge graph embedding?\n\n2) The paper also emphasized the proposed TR formulation was \"data-efficient\". Is this property brought by Fourier expansion? In opposite, what is the drawback or limitation?\n\n3) How to perform the tasks with NE during inference? It seemed unclear in the paper.", "limitations": "1) The limitation of Fourier expansion for real-world temporal regularities was not discussed.\n\n2) Potential negative societal impact was not discussed, e.g. causality of TR, privacy issue brought by TR detection.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "1) This paper targeted at detecting and encoding temporal regularities (TRs) in events, which are of great importance.\n\n2) This paper introduced Fourier basis to intrisincally model temporal regularities.", "weaknesses": "1) The authors overclaimed the first contribution.\nThe problems of TR detection and TR query are closely related to temporal assoication mining, with different evaluation metrics. However, these metrics were commonly adopted in other fields. So the problems were not new.\n\n2) No strong baselines were compared in the experiments.\nAll compared baselines were not targeted this problem or had different assumptions, and therefore did not support the superiority of NE.\n\n3) The paper is poorly presented. \nThe tasks were not presented clearly.\nIn addition this paper defined too many symbols but did not well explained them, which made readers lost.\n\n4) The connection between Noether’s theorem and the proposed method is unclear and weak.\nAs an evidence, keywords such as \"noether’s theorem\" and \"conservation law\" only appeared in two paragraphs and I can't find how \"noether’s theorem\" motivated the formulation of Noether Embedding in a concrete description.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for spending valuable time reviewing our manuscript and providing insightful comments. We have improved our paper accordingly but also discovered some misunderstandings concerning the content and contributions of the work. Our responses are as below.\n\n**From ‘Weaknesses’**\n\n**Q1**: ‘The authors overclaimed the first contribution… the problems were not new’\n\n**A1**: **To our best knowledge, the problem is new. The detailed answer is provided in the first section of our global response at the top of the webpage.**\n\n**Q2**: ‘No strong baselines were compared in the experiments…therefore did not support the superiority of NE.’\n\n**A2**: This is because our problem is new, so there are no existing baselines, to our best knowledge, that exactly match our approach in the available research fields. Besides, NE's main superiority of 'efficiency' is evident even without comparative experiments. **The detailed answer is provided in the second section of our global response.**\n\n**Q3**: ‘The paper is poorly presented ... too many symbols but did not well explained them, which made readers lost.’\n\n**A3**: We have accordingly revised our paper. For example, we provide intuitive descriptions of the defined terms, such as explaining that standard confidence can be viewed as the probability that the head event will occur at time $t+\\bigtriangleup$ once the body event occurs at time $t$.\n\n**Q4**: ‘I can't find how \"noether’s theorem\" motivated the formulation of Noether Embedding in a concrete description’\n\n**A4**: The motivation is described by the underlined text in Section 3.1. Specifically, Noether’s theorem inspires us with three structural biases when constructing NE: (1) the event embedding $\\pmb{q}(t;ev)$ is constructed to make each local energy $g$ invariant to $t$; (2) the training loss is constructed to make the value of $g$ approximate TR validity; (3) the local energy $g$ is used as the decoding function. We have made the connection clearer in the revised version of the paper.\n\n**Q5**: ‘The connection between Noether’s theorem and the proposed method is unclear and weak’\n\n**A5**: The connection is evident in three aspects. Firstly, Noether’s theorem directly motivates the construction of NE, as shown in Section 3.1. Secondly, we attribute NE’s unique efficient learning capability directly to the Noether-inspired structural biases. Specifically, the first bias enables the data-efficient formation of TRs, the second bias mainly contributes to accurate TR detection and query, and the third bias directly leads to the rapid retrieval of TRs. Detailed explanations are in Section 3.3 and Appendix B.2. Thirdly, a more strict correspondence between NE variables and those in a physical system is shown in Appendix B.1.1. \n\n**From ‘Questions’**\n\n**Q1**: ‘The paper emphasized \"distributed\" representation was an advantage of NE. Is it a general property of knowledge graph embedding’\n\n**A1**: NE and knowledge graph embeddings utilize distinct aspects of distributed representations. Knowledge graph embeddings leverage their generalization (by interpolation) capabilities to achieve good performance in the completion task. NE, instead, use complex vectors to apply Fourier-like expansions, thus fitting and storing both event occurrences and TR validities in the embedding space. Specifically, the score function $f(t;ev)=\\sum _{j=1}^d Real(\\pmb{u} \\circ e^{i \\pmb{\\omega} t}) _j$ and the decoding function $g(\\tau;ev _b,ev _h)=2-2\\sum _{j=1}^d Real(\\overline{\\pmb{u} _b} \\circ \\pmb{u} _h \\circ e^{i \\pmb{\\omega} \\tau}) _j\\in [0,4]$ can be seen as Fourier-like expansions. The global time vector $\\pmb{\\omega}$ serves as the expansion basis, and event type vector $\\pmb{u}$s store the coefficients for $f(t)$ (event occurrence) and composing $\\overline{\\pmb{u} _b} \\circ \\pmb{u} _h$ as the coefficients for $g(\\tau)$ (TR validity).\n\n**Q2**: ‘The paper also emphasized the proposed TR formulation was \"data-efficient\". Is this property brought by Fourier expansion?’\n\n**A2**: We attribute NE’s data-efficient capability mostly to the fact that $g$ is invariant to $t$, which is directly inspired by Noether’s theorem. Detailed explanations are in Section 3.3. Fourier-like expansion, instead, is necessary for allowing good fitting and large-capacity storage, serving as a prerequisite for NE to perform well on large-scale real-world datasets used in the paper.\n\n**Q3**: ‘How to perform the tasks with NE during inference’\n\n**A3**: As described in Section 3.2, $\\mathop{\\min}\\limits _{\\tau\\in\\mathbb{T} _r} g(\\tau)$ is computed, which is compared with a global threshold $g _{th}$ to decide whether a potential TR is valid or not (for TR detection). For a valid TR, the $\\tau'$ which minimizes $g(\\tau), \\tau \\in \\mathbb{T} _r$ is selected as the model output of the relative time (for TR query). We have provided clarifications on the relationship between NE decoding and TR detection and TR query in the revised paper.\n\n**From ‘Limitations’**\n\n**Q1**: ‘The limitation of Fourier expansion for real-world temporal regularities was not discussed’\n\n**A1**: In Appendix B.2.2, we have theoretically analyzed the requirement for the vector dimension $d$ to be larger than the number of absolute time points $T _a$ to avoid significant performance degradation of NE, as observed in the GDELT dataset. This limitation imposes a storage capacity constraint for large datasets. We have included notifications regarding this limitation in the revised paper.\n\n**Q2**: ‘Potential negative societal impact was not discussed, e.g. causality of TR, privacy issue brought by TR detection.’\n\n**A2**: Thanks for your reminding. We have accordingly revised our paper to address these concerns.\n\nWe believe that our work makes a nontrivial contribution to the representation learning community by developing NE as a first efficient TR learner with event embeddings and proposing tasks to fairly evaluate embeddings' TR learning capabilities."}, {"review_id": "njQgk5vsxW", "reviewer": "Reviewer_zRYw", "summary": "This paper defines the tasks of temporal regularity (TR) detection and query and their evaluation metrics, and proposes Noether Embedding (NE) that enables encoding TRs from limited event samples and rapid retrieval of TRs in constant time complexity. NE possesses time-translation symmetries of temporal regularities that are indicated by conserved local energies in the embedding space. NE is evaluated on ICEWS14, ICEWS18, and GDELT datasets, and achieves about double F1 scores for detecting valid TRs and over 10 times confidence scores for querying TR intervals compared with baseline embeddings with additional calculation. NE is further shown to be useful for social event prediction and personal decision-making scenarios.", "questions": "1. L279 argues that NE is qualitatively different from rule mining methods in that a search method may require fixing different relative time points before mining rules, while NE enables direct fitting and plotting of an approximate validity distribution of relative time points. But L150-151 suggests that users will select a set of relative time points. So how does NE differ? If users provide their selected set of relative time points, why couldn't we detect time regularities by counting? \n2. What structural biases do NE assume? Could you provide theoretical analyses for the capacity limitation of NE?\n3. Could you add to Section 3.2 the geometric interpretation of the Hardmard product in Eq. (4)?\n4. L116: Is the $\\omega$ vector a hyperparameter or learned, and what is the semantic implication of it being certain values?\n5. L122: Should $t^\\prime$ exclude all $\\hat{t}$ where $(ev,\\hat{t})$ is in the dataset?\n6. To confirm understanding, the second equal sign is L134 always hold true, and by construction, g being invariant to t is always true. Right?", "limitations": "The Conclusion section mentions limitations. The paper doesn't seem to discuss negative societal impacts. It could say something about when the method may fail to detect or query temporal regularities in applications and potential consequences.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- This work defines a pair of particularly novel tasks: temporal regularity detection and query. Both are critical to human-level intelligence.\n- It proposes Noether Embedding which for the first time enables learning temporal regularities directly from events and rapid retrieval of the regularities.\n- NE achieves promising performances on ICEWS14, ICEWS18, and GDELT datasets, and is shown to be useful for social event prediction and personal decision-making scenarios.", "weaknesses": "Please see Questions for doubts to be addressed in the next version of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for spending valuable time reviewing our manuscript and providing insightful comments. We have improved our paper accordingly, and our responses are as below.\n\n**Q1**: ‘So how does NE differ?’ & ‘why couldn't we detect time regularities by counting?’\n\n**A1**: The main difference lies in the transfer of time complexity. Specifically, the counting time in the search for each $(ev _b,ev _h,\\tau)$ is proportional to the number of relevant events. This time complexity is transferred to the training stage of NE so that decoding each $(ev _b,ev _h,\\tau)$ is only O(d), and even O(1) by applying GPUs. Therefore, the NE vectors after the training stage are functionally equivalent to an approximate memory of all TR validity results after the counting process for each $(ev _b,ev _h,\\tau)$. We have made this clearer in the revised paper.\n\n**Q2**: ‘What structural biases do NE assume?’\n\n**A2**: NE’s structural biases are directly inspired by Noether’s theorem. Specifically, (1) the event embedding $\\pmb{q}(t;ev)$ should be constructed to make each local energy $g$ invariant to $t$; (2) the training loss should be constructed to make the value of $g$ approximate TR validity; (3) we should use local energies $g$s as the decoding function. The detailed reasons for how these biases enable NE are explained in Section 3.3 and theoretically analyzed in Appendix B.2.\n\n**Q3**: ‘Could you provide theoretical analyses for the capacity limitation of NE?’\n\n**A3**: In Appendix B.2.2, we theoretically analyzed two inequality constraints that control NE’s performance. We discovered that the vector dimension $d$ is better to be larger than the number of relative time points $T _a$ to enable NE. Such a requirement is also confirmed by the ablation experiment in Table 5 in Appendix C.3.2. We will further provide a more strict proof of this capacity limitation in the next version of our paper.    \n\n**Q4**: ‘Could you add to Section 3.2 the geometric interpretation of the Hardmard product in Eq. (4)?’\n\n**A4**: The Hardmard product can be depicted as a rotation of event-type vectors by time in the d-dimensional complex space. We have added this interpretation in our revised paper.\n\n**Q5**: ‘Is the $\\omega$ vector a hyperparameter or learned, and what is the semantic implication of it being certain values?’\n\n**A5**: The $\\pmb{\\omega}$ vector is manually set as an exponential distribution in the paper. We also show in Section 4.4 (Ablation Studies -- Frequency Distribution) that the exponential distribution surpasses linear distribution for fitting larger datasets. \n\n$\\pmb{\\omega}$ provides the basis for Fourier-like expansions. Specifically, since the score function $f(t;ev)=\\sum _{j=1}^d Real(\\pmb{u} \\circ e^{i \\pmb{\\omega} t}) _j$ and the decoding function $g(\\tau;ev _b,ev _h)=2-2\\sum _{j=1}^d Real(\\overline{\\pmb{u} _b} \\circ \\pmb{u} _h \\circ e^{i \\pmb{\\omega} \\tau}) _j\\in [0,4]$ can be viewed as Fourier-like expansions, we can see that the global time vector $\\pmb{\\omega}$ provides the expansion basis and the event-type vectors $\\pmb{u}$s store the coefficients for $f(t)$ (revealing event occurrence) and composing $\\overline{\\pmb{u} _b} \\circ \\pmb{u} _h$ as the coefficients for $g(\\tau)$ (revealing TR validity). \n\n**Q6**: ‘Should $t’$ exclude all $t$ where (ev,t) is in the dataset?’\n\n**A6**: Your comment is reasonable. Exclusion is needed in the paper for rigor’s sake. However, it is also acceptable if it does not, as it trades a little performance drop for a faster implementation. This is because negative samples contribute much less ($\\frac{1}{N}$) than a positive sample ($1$) as long as the number of negative samples $N$ is much larger than $1$, which is the general case. Therefore, those positive samples ‘wrongly regarded’ as negative samples will make a minor difference in the loss score.\n\n**Q7**: ‘the second equal sign is L134 always hold true, and by construction, $g$ being invariant to $t$ is always true. Right?’\n\n**A7**: Yes, it does. This contributes to NE’s major structural bias as discussed in Answer 2. \n\n**Q8**: ‘The paper doesn't seem to discuss negative societal impacts’\n\n**A8**: NE fails if cannot fit the whole dataset well, which is generally when the vector dimension $d$ is set smaller than the number of absolute time points $T _a$. The potential consequence is that false positives may lead to inaccurate prediction and that false negatives may lead to untimely warnings. We have added this discussion to the revised paper.\n\nOnce again, we would like to express our gratitude to the reviewer for the valuable feedback, which has helped us further improve our manuscript."}, {"review_id": "IrzcpBSOai", "reviewer": "Reviewer_7yn1", "summary": "The paper presents a method for detecting, and embedding, temporal regularities from time-stamped event data, where events have a discrete type, and temporal regularities correspond to one base type preceding another head type by a characteristic relative time. A temporal regularity (TR) is defined in the following way. A temporal regularity associates a body event-type $b$ with a head-event type $h$, alongside a mean relative time $\\tau$, and a universal factor $\\eta$, such that events of type $b$ at time $t$ are \"regularly\" followed (or maybe also preceded) by an event of type $h$, with a mean time delay of $\\tau$ but with a time variation in the interval $\\Delta = [\\tau(1-\\eta), \\tau(1+\\eta)]$. One challenge of this is to discover these temporal regularities purely from time-stamped events, i.e. to abstract away from the specific time of the body event type. The principle contribution of this work is to train Noether Embeddings (NE) which the authors state \"enable both the data efficient formation and rapid retrieval of TRs simply through embedding each event sample\".\n\n\nThe authors define a collection of metrics for temporal regularities between any given body and head types (b,h). standard confidence (sc), head coverage (hc) and general confidence (gc), the last of which is a key measure of how strong the evidence is for a temporal regularity under a particular interval $\\Delta$ (which is itself a consequence of the mean time $\\tau$). They go on to specify two tasks, the first \"TR detection\" is to find the best value of general confidence for each pair of event types (for any value of relative-time) and specify this pair of event types as a TR if this value is above some threshold. The second task is TR Query which asks what the characteristic relative time between a pair of events is. Noether embeddings are then a composition of event embedding and time embedding, in such a way that the complex field is used to capture the time information withing just one part of the composition. TRs detection (or scoring) can then be efficiently calculated between any pair of embedded events. Associated methods are then used to determine the TR detection and query operations.\n\nThe authors compare their method of evaluating TR detection and querying with appropriately adapted methods from the literature. It isn't entirely clear from the paper, whether these other methods are trained to maximise scores between valid TRs in some way (but the assumption is that they are). The authors show that, on three datasets derived from large scale temporal knowledge graph data used elsewhere in the literature, their method performs substantially better at detecting and scoring (querying) temporal regularities in data.", "questions": "Why does the queried time set $\\mathbb{T}_r = \\{1-T_a,\\ldots, 0, \\ldots, T_a-1\\}$ mean that you only query one side of an asymmetric pair of events? Surely this means that the total number of relative time durations is $2T_a -2$, meaning that any pair of events $(ev_i, ev_j)$ at times $t_i$ and $t_j$ would be queried in both directions so long as $t_i\\neq 0$ or $t_j \\neq T_a$ and vice versa.\n\nWhat is the measure of TR Query (denoted r in table 1)?\n\nHow is the data in the experiments split into training and testing parts?\n\nWhy are there only 1 element per periodic time delay in the $\\omega$ vectors? Does this mean that you can only discover 1 TR (body-head type) per characteristic time delay $\\tau$? What if two distinct TRs (body-head types) had the same characteristic time delay?", "limitations": "The authors could be clearer about the limitations of this work. In particular, certain choices such as predefined fixed length interval factors for detection, predefined thresholds for detection, point estimates for TR Queries may be necessary to make the method tractable, but they do introduce a certain arbitrariness to the prediction frame work.\nEqually, the fact that the experiments are comparing the performance of their method in optimising against an objective that motivated the design of their approach is potentially problematic too. The ground truth general confidence of a TR is the best performing time delay for a pair of events, which is the same as the objective for their detection method. The argmax of this is both the ground-truth for TR Query and the training objective for TR Query on their model. The authors should justify why this is a fair comparator, and possibly develop alternative independent measures for evaluation too.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The method is well motivated in terms of Emmy Noether's work corresponding symmetries with conservation laws. The authors develop a neat way to encode things that lends itself to efficient training and their performance on three different datasets is substantially better than other methods. \nIt isn't entirely clear how to judge the significance of this work, as the authors define their own task and their own metrics, possibly because there are no suitable pre-existing candidates for this.", "weaknesses": "The paper could be a little clearer in parts. For instance:\n* It is unclear whether the authors' definition of temporal regularity (TR) is unique to the authors or is defined elsewhere. The authors relate this to prior work on temporal regularities, but the precise formulation appears to be the authors' own. It would be helpful to know whether and how this relates to other formal definitions of temporal regularities.\n* More effort could be made to give intuitive (informal) descriptions of the defined terms, e.g. standard confidence, TR Detection, TR Query, ground truth confidence, and so on. \n* How the data is partitioned into test and train, and how various methods are optimised from training data could be made clearer in the main body of the paper.\n* It isn't really clear until late in the paper that the time-stamps are discrete. I am not sure whether this method would be tractable for continuously valued time-stamps\n* The intuitive meaning of what is being established by temporal regularities and the associated metrics could be clearer too. For instance, what does it mean to have a ground-truth confidence greater than 0.8. Can that be given a (loose) statistical interpretation at all?\n* The notion of an interval $\\Delta$ which depends on the mean relative time $\\tau$ is not clearly discussed. Is this like saying that the TRs model events whose relative time is uniformly distributed in some range? Is this a little brittle compared to a method that \"smoothly\" traded off relative-time-clustering with observation frequency?\n\nThe notation is a little confusing at times. In particular, Equation 2 or the text around it, could make some things clearer:\n* that this metric applies to a specific pairing of event types b and h and a time interval (I realise this is wrapped up in the tr variable but I found the notation pretty opaque).\n* that the tr, sp, hc, and gc terms depend on $\\Delta$ but really more fundamentally depend on the mean relative time $\\tau$. Later the authors refer to  $gc$ as a function of $\\tau$ so this seems to me to be the fundamental variable. Would it be better to define tr as $(ev_b, ev_h, \\tau, \\eta)$ where $\\Delta$ is derived from these values?\n* that this metric is defined over an event set (by which I meant that the event set could appear formally in notation)\n* b(q;tr) and n(b(tr)) mean two slightly different things, the latter is a count over b(q;tr) for all q in the event set I think. Similarly replacing b with h.\n\nThe definition of TR Detection and TR Query seem to be the authors own, and so the fairness of the experimental comparison is difficult to judge. The performance is impressive on first appearance, but I am left wondering how to interpret this. The paper would be greatly strengthened by a clear justification of why this is the fairest available comparison between methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for spending valuable time reviewing our manuscript and providing insightful comments. We have improved our paper accordingly but discovered some misunderstandings concerning the content and contributions of the work. Our responses are provided below.\n\n**The foremost QA**\n\n**Q**: ‘The paper would be greatly strengthened by a clear justification of why this is the fairest available comparison between methods.’\n\n**A**: **The answer is in the second section of the global response at the top of the webpage.**\n\n**From ‘Weaknesses’**\n\n**Q1**: ‘whether TR is unique to the authors or is defined elsewhere’\n\n**A1**: Our definition is unique. A relevant definition for knowledge graph streams is in [1]. The main difference is that TR is suitable for arbitrary structured events, whereas the relevant definition in [1] applies only to knowledge graph streams. Additionally, TR introduces an adaptive $\\eta$ to allow vibrations of $\\tau$ in real-world data distributions.\n\n**Q2**: ‘intuitive (informal) descriptions of the defined terms…’ \n\n**A2**: We have revised our paper accordingly. For example, the intuitive description of standard confidence can be viewed as the probability that the head event will occur at time $t+\\bigtriangleup$ once the body event occurs at time $t$.\n\n**Q3**: ‘How the data is partitioned into test and train, and how various methods are optimised from training data…’\n\n**A3**: The train and test sets are of the same set. To enhance understanding, we can draw a comparison between TR learning and clustering. In this analogy, event items correspond to clustered samples, and TRs correspond to the clusters. Therefore, NE can be viewed as a memory with unsupervised learning capabilities due to its specific structural biases. \n\nThe loss functions of NE and existing embeddings all separate the score functions of positive and negative event samples, treating representation learning as a two-class classification problem. All the baseline embedding models adopt the log-softmax form of loss functions, as in their original settings. \n\n**Q4**: ‘whether this method would be tractable for continuously valued time-stamps’\n\n**A4**: It would be tractable because continuously valued time stamps can be discretized. \n\n**Q5**: ‘what does it mean to have a ground-truth confidence greater than 0.8.’\n\n**A5**: This implies that there is an 80% probability for the co-occurrence of a body event at time $t$ and a head event at time $t+\\bigtriangleup$.\n\n**Q6**: ‘The notion of an interval $\\bigtriangleup$ which depends on the mean relative time $\\tau$ is not clearly discussed’\n\n**A6**: Since such a definition is used for calculating metrics such as general confidence, it is required to be independent of specific data distributions. Intuitive meaning: the tolerance of noises (width of $\\bigtriangleup$) is proportional to the size of relative time $\\tau$.\n\n**Q7**: ‘The notation is a little confusing at times. In particular, Equation 2 or the text around it, could make some things clearer’\n\n**A7**: We have made improvements in the revised paper. For example, we have changed $tr: (ev _b,ev _h,\\bigtriangleup)$ into $tr: (ev _b,ev _h,\\tau,\\eta)$ as suggested.\n\n**From ‘Questions’**\n\n**Q1**: ‘Why does the queried time set mean that you only query one side of an asymmetric pair of events?’\n\n**A1**: Considering two event types $ev _i, ev _j$. Since $g(\\tau;ev _i,ev _j)=g(-\\tau;ev _j,ev _i)$, it is only necessary to calculate one decoding function between  $g(\\tau;ev _i,ev _j)$ and $g(-\\tau;ev _j,ev _i)$ at the decoding stage.\n\n**Q2**: ‘What is the measure of TR Query’\n\n**A2**: As stated in Section 2.2, TR query is to output the correct $\\tau'=\\tau _g$ for valid TRs, where the ground truth $\\tau_g$ is set as what maximizes gc(\\tau) in computing $gc_g$. For each tested query $(ev _b,ev _h)$, a ratio $r'=\\frac{gc(\\tau')}{gc _g}$ is computed. The averaged ratio $r$ of all queries is reported.\n\n**Q3**: ‘…training and testing parts?’\n\n**A3**: The same as in Answer 3 from ‘Weaknesses’.\n\n**Q4**: ‘Why are there only 1 element per periodic time delay in the $\\omega$ vectors?’ & ‘What if two distinct TRs (body-head types) had the same characteristic time delay?’ \n\n**A4**: One advantage of NE is exactly its ability to efficiently store large amounts of intertwined TRs. The specific reason is in 3(2) of the global response at the top of the webpage. \n\n**From ‘Limitations’**\n\n**Q1**: ‘they do introduce a certain arbitrariness to the prediction frame work’\n\n**A1**: We have conducted data analysis to rationalize the parameters as much as possible. For example, we have analyzed that TRs whose gc ∼ 0.8 constitute a small percentage of all tested TRs. We have also made reasonable adjustments. For example, we have conducted experiments in Appendix C.3.3 where the threshold for distinguishing valid and invalid TRs is chosen as 0.7,0.8,0.9, respectively. We emphasize that, similar to the classic clustering task, some degree of ‘arbitrariness’ is inevitable due to the unsupervised nature of TR learning.\n\n**Q2**: ‘justify why this is a fair comparator ... independent measures for evaluation too’\n\n**A2**: Fairness is ensured by treating both NE and existing embeddings equally as justified in the global response. Moreover, we disagree with the statement that ‘the argmax is … for TR Query on their model’. At the decoding stage for TR query, the argmax in evaluation is performed on the relative time. At the training stage, however, no information about the relative time is needed. The training loss is only relevant to the specific information $(ev, t)$ of each event sample and whether it is a positive or negative sample.\n\nWe have greatly improved the paper by the reviewer's valuable feedback. We emphasize that NE's advantage of 'efficiency' is evident even without the comparative experiments. \n\n**Reference**\n\n[1] Omran P G, Wang K, Wang Z. Learning Temporal Rules from Knowledge Graph Streams. AAAI, 2019."}]

Author rebuttal:
Three main justifications are provided below.

## 1. Novelty of the problem

**Q**: ‘The authors overclaimed the first contribution…the problems were not new’

**A**: To our best knowledge, the problem is new. Our main problem is how to enable event embeddings with an efficient TR learning capability, which is distinct from those in the temporal rule mining field. Specifically, temporal rule mining is typically studied for practical applications, aiming to uncover event regularities in specific domains [1] [2] [3]. Our goal, instead, is to advance the representation learning field by enabling embedding models to efficiently learn the atomic structure (TR) of event schemas, similar to how humans learn [4][5]. Our problem is not only new but also important because (1) embedding models are a promising technology field; (2) TRs are a basic world structure among events; (3) efficient learning is a long-pursued human-like capability. 

To address this problem, we have developed NE as the first efficient TR learner using event embeddings. Additionally, we have proposed two complementary tasks for evaluating the TR learning capabilities of embedding models. Our tasks are defined with evaluation metrics borrowed from the rule mining field [6], only to ensure fair and reasonable evaluations.

To avoid misunderstandings, we have reversed the order of our two contributions, making the development of NE the first contribution and the proposal of tasks the second. Both of these contributions aim to advance the representation learning field.

## 2. Fairness of comparisons

**Q**: ‘The paper would be greatly strengthened by a clear justification of why this is the fairest available comparison between methods.’

**A**: Our main contribution is developing NE as a first ‘efficient’ TR learner with event embeddings. The uniqueness of NE in terms of efficiency is fundamental and qualitative, requiring no comparative experiments to demonstrate. Specifically, only NE can encode TRs from limited event items (as shown in Figure 4) and rapidly retrieve TRs (by applying $g(\tau)$), thanks to its specific structural biases. Existing embedding models lack these capabilities.

Only when setting aside the ‘efficient’ requirement are comparative experiments necessary to compare the TR learning accuracy between NE and existing embeddings. This is secondary compared to the ‘efficient’ distinction. Since there are, to our best knowledge, no existing embedding baselines that exactly match our requirements in the available research fields, we have made efforts to ensure fairness in comparison through various means:  

(1)	Models are applied in the same way. We input the same event data during the training stage and add the same interface $g’(\tau)$ to the respective model outputs of score functions. This interface is applied to both NE and the embedding baselines, in the same manner, to indirectly compute TR validity from stored event occurrences. The excellent performance of NE with this interface validates its effectiveness. 

(2)	Evaluations are reliable. The evaluation metrics used in our proposed tasks are borrowed and adapted from the mature field of rule mining [6]. This guarantees a reliable evaluation of the TR learning capabilities of embedding models. 

(3)	Baselines are classic.  We have chosen baselines from the well-developed field of temporal knowledge graph embedding, which has a wide range of classic embeddings for structured events.

(4)	Dataset is convincing. Our main experiments are conducted on three classic real-world event datasets.

(5)	Performance is explainable. We provide detailed explanations in the paper regarding why NE works and why existing embeddings do not, both theoretically and experimentally.

We have further clarified these points in the revised paper.

## 3. Why NE works and overwhelms

The success of NE can be attributed to two main factors: the Noether-inspired structural biases and the Fourier-like memory representations. One contributes to NE’s efficient TR learning capability while the other enables NE’s large-capacity storage for both TR validity and event occurrences.

(1)	The Noether-inspired structural biases

The Noether-inspired structural biases can be summarized as below: (i) the event embedding $\pmb{q}(t;ev)$ is constructed to make each local energy $g$ remain invariant to $t$; (ii) the training loss is designed to make the value of $g$ approximate TR validity; (iii) the local energy $g$ is used as the decoding function. 

(2)	The Fourier-like memory representations

The score function $f(t;ev)=\sum _{j=1}^d Real(\pmb{u} \circ e^{i \pmb{\omega} t}) _j$ and the decoding function $g(\tau;ev _b,ev _h)=2-2\sum _{j=1}^d Real(\overline{\pmb{u} _b} \circ \pmb{u} _h \circ e^{i \pmb{\omega} \tau}) _j\in [0,4]$ can be viewed as Fourier-like expansions. The global time vector $\pmb{\omega}$ provides the expansion basis, while the event type vectors $\pmb{u}$s store the coefficients for $f(t)$ (revealing event occurrence) and compose $\overline{\pmb{u} _b} \circ \pmb{u} _h$ as the coefficients for $g(\tau)$ (revealing TR validity).

Detailed explanations can be found in Section 3.3 and Appendix B.2.

## Reference

[1] Segura‐Delgado A, Gacto M J, Alcalá R, et al. Temporal association rule mining: An overview considering the time variable as an integral or implied component. DMTD, 2020.

[2] Chen M, Chen S C, Shyu M L. Hierarchical temporal association mining for video event detection in video databases. IEEE, 2007.

[3] Yoo J S, Shekhar S. Similarity-profiled temporal association mining. IEEE, 2008.

[4] Pudhiyidath A, Roome H E, Coughlin C, et al. Developmental differences in temporal schema acquisition impact reasoning decisions. Cognitive Neuropsychology, 2020.

[5] Chaudhuri R, Fiete I. Computational principles of memory. Nature neuroscience, 2016.

[6] Galárraga L, Teflioudi C, Hose K, et al. Fast rule mining in ontological knowledge bases with AMIE+. VLDB, 2015.

Meta-review: This paper receives mixed reviews. The reviewers express concern on the novelty of the problem, fairness of comparisons, unclear on the important aspects of the proposed method. The authors provide very detailed responses these reviewers comments. After reading the author rebuttal, two reviewers are very active in the discussion with authors and give very detailed and specific comments to authors. The authors also give very detailed explanation and new results on the compression ratios of NE compared to exact counting, which is appreciated by the reviewers. Another active reviewers are also persuaded by the main strengths of the paper and think this paper can be accepted. The AC carefully check the paper, the reviewer comments and the author responses, and downgrade the score of some less active reviewers. Overall, the AC agrees with the contribution of this paper and makes accept recommendation to this paper. The authors are required to add the new results in the rebuttal and discussion with reviewers to the final version.
Decision: Accept (poster)