Original: D IRECT E MBEDDING OF T EMPORAL N ETWORK E DGES VIA T IME -D ECAYED L INE G RAPHS
Revised: D IRECT E MBEDDING OF T EMPORAL N ETWORK E DGES VIA T IME -D ECAYED L INE G RAPHS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Temporal networks model a variety of important phenomena involving timed interactions between entities.
Revised: Temporal networks model a variety of important phenomena involving timed interactions between entities.

Original: Existing methods for machine learning on temporal networks generally exhibit at least one of two limitations.
Revised: Existing methods for machine learning on temporal networks generally exhibit at least one of two limitations.

Original: First,  timeis assumed to be discretized, so if the time data is continuous, the user must determine the discretization and discard precise time information.
Revised: First, many methods assume time to be discretized, so if the time data is continuous, the user must determine the discretization and discard precise time information.

Original: Second, edge representations can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification.
Revised: Second, edge representations can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification.

Original: We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions.
Revised: We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions.

Original: From this derived graph, edge representations for the original network can be computed with efficient classical methods.
Revised: From this derived graph, edge representations for the original network can be computed with efficient classical methods.

Original: The simplicity of this approach facilitates explicit theoretical analysis: we can constructively show the effectiveness of our method’s representations for a natural synthetic model of temporal networks.
Revised: The simplicity of this approach facilitates explicit theoretical analysis: we can constructively show the effectiveness of our method’s representations for a natural synthetic model of temporal networks.

Original: Empirical results on real-world networks demonstrate our method’s efficacy and efficiency on both edge classification and temporal link prediction.
Revised: Empirical results on real-world networks demonstrate our method’s efficacy and efficiency on both edge classification and temporal link prediction.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Temporal networks, which are graphs augmented with a time value for each edge, model a variety of important phenomena involving timed interactions between entities, including financial transactions, flights, and web browsing.
Revised: Temporal networks, which are graphs augmented with a time value for each edge, model a variety of important phenomena involving timed interactions between entities, including financial transactions, flights, and web browsing.

Original: Common tasks for machine learning on temporal networks include classification of the temporal edges, as well as temporal link prediction, which involves predicting future links given some links in the past.
Revised: Common tasks for machine learning on temporal networks include classification of the temporal edges, as well as temporal link prediction, which involves predicting future links given some links in the past.

Original: These tasks yield various applications, such as recommendation systems (Zhou et al., 2021) and detection of illicit financial transactions (Pareja et al., 2020).
Revised: These tasks yield various applications, such as recommendation systems (Zhou et al., 2021) and detection of illicit financial transactions (Pareja et al., 2020).

Original: As with most machine learning for graphs, the key to learning for temporal networks is creating effective vector representations, also called embeddings, for the network’s components, namely the nodes and edges.
Revised: As with most machine learning for graphs, the key to learning for temporal networks is creating effective vector representations, also called embeddings, for the network’s components, namely the nodes and edges.

Original: These embeddings can either be made as part of an end-to-end framework, or created then passed to off-the-shelf classifiers for downstream tasks; for example, for the edge classification task, a logistic regression classifier can be trained using the training edges’ embedding vectors and class labels, then applied at inference time on the test edges’ vectors.
Revised: These embeddings can either be made as part of an end-to-end framework, or created then passed to off-the-shelf classifiers for downstream tasks; for example, for the edge classification task, a logistic regression classifier can be trained using the training edges’ embedding vectors and class labels, then applied at inference time on the test edges’ vectors.

Original: The node embedding task, in particular, has seen great interest, and many methods for ‘static’ (i.e., non-temporal) networks have been proposed over the years (Belkin & Niyogi, 2001;
Revised: The node embedding task, in particular, has seen great interest, and many methods for ‘static’ (i.e., non-temporal) networks have been proposed over the years (Belkin & Niyogi, 2001;

Original: Perozzi et al., 2014; Grover & Leskovec, 2016).
Revised: Perozzi et al., 2014; Grover & Leskovec, 2016).

Original: Edge embedding has seen less interest; there are some exceptions (Li et al., 2017b; Bandyopadhyay et al., 2019), but generally, edge embeddings are created by first making node embeddings, then aggregating them, e.g., by taking the entrywise product of the two endpoint nodes’ embeddings.
Revised: Edge embedding has seen less interest; there are some exceptions (Li et al., 2017b; Bandyopadhyay et al., 2019), but generally, edge embeddings are created by first making node embeddings, then aggregating them, e.g., by taking the entrywise product of the two endpoint nodes’ embeddings.

Original: There are a wide variety of methods for temporal network embedding, including ones based on matrix/tensor factorization (Dunlavy et al., 2011;
Revised: There are a wide variety of methods for temporal network embedding, including ones based on matrix/tensor factorization (Dunlavy et al., 2011;

Original: Li et al., 2017a; Zhang et al., 2018), random walks (Yu et al., 2018; Nguyen et al., 2018), graph convolutional networks (Pareja et al., 2020), and deep autoencoders (Goyal et al., 2018;
Revised: Li et al., 2017a; Zhang et al., 2018), random walks (Yu et al., 2018; Nguyen et al., 2018), graph convolutional networks (Pareja et al., 2020), and deep autoencoders (Goyal et al., 2018;

Original: Rahman et al., 2018; Goyal et al., 2020), but generally, these methods can be seen as variants of those for static networks.
Revised: Rahman et al., 2018; Goyal et al., 2020), but generally, these methods can be seen as variants of those for static networks.

Original: With some exceptions (Trivedi et al., 2017; Nguyen et al., 2018; Trivedi et al., 2019), these methods generally do not work directly with the continuous-valued times of temporal edges, but rather require that the times are discretized, yielding a sequence of static graphs.These prior methods present some limitations.
Revised: With some exceptions especially in more recent work (Nguyen et al., 2018; Trivedi et al., 2019; Rossi et al., 2020; Wang et al., 2021), these methods often do not work directly with the continuous-valued times of temporal edges, but rather require that the times are discretized, yielding a sequence of static graphs.

Original: First, since most datasets have continuous-timed edges, the assumption that time is discretized requires the user to manually determine the discretization
Revised: Further, nearly all of these works learn node embeddings rather than edge embeddings. Since the edges are timestamped, and not the nodes, it is more natural and meaningful to learn embeddings

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original:  and discard precise time information.
Revised: directly for the timestamped edges. It is also of practical importance since the downstream tasks of interest (such as edge classification) require edge embeddings as opposed to node embeddings. These prior methods overall present some limitations, which we now summarize. First, since most datasets have continuous-timed edges, the assumption that time is discretized requires the user to manually determine the discretization and discard precise time information.

Original: Second, prior methods generally return representations of nodes rather than edges, so edge representations can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification.
Revised: Second, prior methods generally return embeddings of nodes rather than edges, so edge embeddings can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification.

Original: Given that timestamps are associated with edges rather than nodes, and that there are few public datasets where the nodes rather than edges are associated with a time-series of attributes or classes, it is intuitively more natural to derive edge embeddings directly.
Revised: As noted above, given that timestamps are associated with edges rather than nodes, and that there are few public datasets where the nodes rather than edges are associated with a time-series of attributes or classes, it is intuitively more natural to derive edge embeddings directly.

Original: We present a simple but novel framework to address these issues: construct the line graph of the network, which converts each edge (timed interaction) of the network into a node, and connects interactions that share an endpoint node.
Revised: We present a simple but novel framework to address these issues: construct the line graph of the network, which converts each edge (timed interaction) of the network into a node, and connects interactions that share an endpoint node.

Original: Then, set the edge weights in this line graph based on differences in time between interactions, with interactions that occur closer together in time being connected with higher weights.
Revised: Then, set the edge weights in this line graph based on differences in time between interactions, with interactions that occur closer together in time being connected with higher weights.

Original: From this derived graph, which directly represents topological proximity (i.e., adjacency of edges) and temporal proximity , temporal edge representations for the original network can be computed and exploited with efficient classical methods.
Revised: From this derived graph, which directly represents topological proximity (i.e., adjacency of edges) and temporal proximity , temporal edge representations for the original network can be computed and exploited with efficient classical methods.

Original: To our knowledge, ours is the first method that directly forms embeddings for continuous-time temporal edges without supervision and without aggregating node embeddings - see Table 1.
Revised: To our knowledge, ours is the first method that directly forms embeddings for continuous-time temporal edges without supervision and without aggregating node embeddings - see Table 1.

Original: Our method is significantly simpler than recent prior work, particularly compared to deep methods, allowing for more direct theoretical analysis: we propose the temporal stochastic block model (TSBM), which naturally extends the well-known stochastic block model (SBM) for static networks to temporal networks, and we show that our method can exploit time and community information in TSBMs to form effective representations.
Revised: Our method is significantly simpler than recent prior work, particularly compared to deep methods, allowing for more direct theoretical analysis: we propose the union of Gaussian-timed stochastic block models (UGT-SBM), which naturally extends the well-known stochastic block model (SBM) for static networks to temporal networks, and we show that our method can exploit time and community information in UGT-SBMs to form effective representations.

Original: Practically, our method’s simplicity makes it easy to implement, yet it is accurate and efficient: in experiments on five benchmark real-world temporal networks, our approach achieves superior predictive and runtime performance relative to prior methods.
Revised: Practically, our method’s simplicity makes it easy to implement, yet it is accurate and efficient: in experiments on five benchmark real-world temporal networks, our approach achieves superior predictive and runtime performance relative to prior methods.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: Our approach starts with a sequence of timestamped edges:
Revised: Our approach starts with a sequence of timestamped edges:

Original: Definition 1 (Temporal Graph) .
Revised: Definition 1 (Temporal Graph) .

Original: A temporal graph G =
Revised: A temporal graph G =

Original: ( V, E ) is a set of vertices V and a set of temporal edges E , where E ⊆ V × V × R .
Revised: ( V, E ) is a set of vertices V and a set of temporal edges E , where E ⊆ V × V × R .

Original: t is the time of the temporal edge ( u, v, t ) .
Revised: t is the time of the temporal edge ( u, v, t ) .

Original: Our approach centers around our proposed notion of ‘time-decayed line graphs’ (TDLGs) which are derived from temporal graphs.
Revised: Our approach centers around our proposed notion of ‘time-decayed line graphs’ (TDLGs) which are derived from temporal graphs.

Original: Our notion of TDLGs extends the well-established idea of line graphs, which are derived from static (i.e., non-temporal), undirected graphs; the line graph of an undirected, unweighted graph is another undirected, unweighted graph that represents adjacencies between edges in the original graph.
Revised: Our notion of TDLGs extends the well-established idea of line graphs, which are derived from static (i.e., non-temporal), undirected graphs; the line graph of an undirected, unweighted graph is another undirected, unweighted graph that represents adjacencies between edges in the original graph.

Original: We propose to incorporate time information by using it to set the weights of the resulting line graph.
Revised: We propose to incorporate time information by using it to set the weights of the resulting line graph.

Original: Specifically, given a temporal graph G , we construct a TDLG, which is a static weighted line graph L TD ( G ) , as follows in Definition 2.
Revised: Specifically, given a temporal graph G , we construct a TDLG, which is a static weighted line graph L TD ( G ) , as follows in Definition 2.

Original: Definition 2 (Time-Decayed Line Graph) .
Revised: Definition 2 (Time-Decayed Line Graph) .

Original: Given a temporal graph G =
Revised: Given a temporal graph G =

Original: ( V, E )
Revised: ( V, E )

Original: , the associated timedecayed line graph is L TD ( G ) =
Revised: , the associated timedecayed line graph is L TD ( G ) =

Original: ( V L , E L , w ) , where V L = E , E L = { (( u, v, t 1 ) , ( v, z, t 2 )) : ( u, v, t 1 ) , ( v, z, t 2 ) ∈ E } , and the edge weight function w : E L → R + evaluated on edge (( u, v, t1 ) , ( v, z, t 2 )) is given by exp (cid:16) − 1 2 σ 2 t ( t 1 − t 2 ) 2 (cid:17) for some fixed σ t >0 .
Revised: ( V L , E L , w ) , where V L = E , E L = { (( u, v, t 1 ) , ( v, z, t 2 )) : ( u, v, t 1 ) , ( v, z, t 2 ) ∈ E } , and the edge weight function w : E L → R + evaluated on edge (( u, v, t ) , ( v, z, t 2 )) is given by exp (cid:16) − 12 σ 2 t ( t 1 − t 2 ) 2 (cid:17) for some fixed σ t >

Original: Thus, proximity of two temporal edges in the TDLG incorporates both topological proximity in the original graph as well as proximity in time.
Revised: Thus, proximity of two temporal edges in the TDLG incorporates both topological proximity in the original graph as well as proximity in time.

Original: The parameter σ t controls how quickly proximity in the
Revised: The parameter σ t controls how quickly proximity in the

Original: TDLG decays as difference in time grows.
Revised: TDLG decays as difference in time grows.

Original: For a graph with n nodes and m edges, we can construct the weighted adjacency matrix A ∈ R m × m + of the TDLG as follows.
Revised: For a graph with n nodes and m edges, we can construct the weighted adjacency matrix A ∈ R m × m + of the TDLG as follows.

Original: Given the incidence matrix
Revised: Given the incidence matrix

Original: B ∈ { 0 , 1 } n × m , each column vector of which is n -dimensional, corresponds to an edge, and is 1 for the edge’s two endpoint nodes and 0 elsewhere; and also given the vector of times of the temporal edges
Revised: B ∈ { 0 , 1 } n × m , each column vector of which is n -dimensional, corresponds to an edge, and is 1 for the edge’s two endpoint nodes and 0 elsewhere; and also given the vector of times of the temporal edges

Original: t ∈ R m 
Revised: t ∈ R m :

Original: [Equation]
Revised: [Equation]

Original: where b i and b j denote columns i and j of B .
Revised: where b i and b j denote columns i and j of B .

Original: Note that A is symmetric.
Revised: Note that A is symmetric.

Original: While self-loops in line graphs are generally removed, we keep them here for simplicity.
Revised: While self-loops in line graphs are generally removed, we keep them here for simplicity.

Original: The choice of Gaussian weight decay is somewhat arbitrary, and, in theory, the TDLG concept would also work with, e.g., Laplacian weight decay.
Revised: The choice of Gaussian weight decay is somewhat arbitrary, and, in theory, the TDLG concept would also work with, e.g., Laplacian weight decay.

Original: We use Gaussian decay since it is well known and effective in practice.
Revised: We use Gaussian decay since it is well known and effective in practice.

Original: We now describe our approach for temporal edge embedding, classification, and link prediction.
Revised: We now describe our approach for temporal edge embedding, classification, and link prediction.

Original: Temporal edge embedding and classification For downstream tasks, we seek temporal edge embeddings, that is, an informative real-valued vector for each temporal edge that can be provided to a classifier.
Revised: Temporal edge embedding and classification For downstream tasks, we seek temporal edge embeddings, that is, an informative real-valued vector for each temporal edge that can be provided to a classifier.

Original: For temporal edge i , we simply return the i th row of the TDLG adjacency matrix A .
Revised: For temporal edge i , we simply return the i th row of the TDLG adjacency matrix A .

Original: This returns an m -dimensional sparse vector as the feature vector.
Revised: This returns an m -dimensional sparse vector as the feature vector.

Original: Note that we could reduce the dimensionality of these vectors, e.g., via eigendecomposition, but we find that this is not necessary.
Revised: Note that we could reduce the dimensionality of these vectors, e.g., via eigendecomposition, but we find that this is not necessary.

Original: We use the standard supervised learning pipeline for edge classification.
Revised: We use the standard supervised learning pipeline for edge classification.

Original: Given training data comprising a set of temporal edges and the class labels of some fraction of these edges, the TDLG matrix A is created using all of the edges, then a logistic regression classifier is trained using the edges for which the classes are known and the corresponding rows of A as feature vectors.
Revised: Given training data comprising a set of temporal edges and the class labels of some fraction of these edges, the TDLG matrix A is created using all of the edges, then a logistic regression classifier is trained using the edges for which the classes are known and the corresponding rows of A as feature vectors.

Original: After this, the classifier can be used on the remaining rows to make predictions for the remaining edges.
Revised: After this, the classifier can be used on the remaining rows to make predictions for the remaining edges.

Original: Temporal link prediction We describe the full experimental setup for temporal link prediction in
Revised: Temporal link prediction We describe the full experimental setup for temporal link prediction in

Original: Section 6, but essentially, it amounts to binary temporal edge classification where the classes are + for true/positive edges ( u i , v i , t i ) and − 1 for false/negative edges ( u ′ i , v ′
Revised: Section 6, but essentially, it amounts to binary temporal edge classification where the classes are + for true/positive edges ( u i , v i , t i ) and − 1 for false/negative edges ( u ′ i , v ′

Original: i , t ′ i ) .
Revised: i , t ′ i ) .

Original: The difference is that for temporal link prediction, we must additionally be able to form representations for negative edges, as well as for test edges (which are not given when training the classifier).
Revised: The difference is that for temporal link prediction, we must additionally be able to form representations for negative edges, as well as for test edges (which are not given when training the classifier).

Original: Let subscripts r and e denote training and test edges, respectively.
Revised: Let subscripts r and e denote training and test edges, respectively.

Original: We deal with negative edges exactly the same as positive edges besides the class labels: the positive and negative training edges are concatenated, producing the training incidence matrix B r ∈ { 0 , 1 } n × m r and times vector t r ∈ R m r , from which we construct the training TDLG adjacency matrix A rr ∈ R m r × m r
Revised: We deal with negative edges exactly the same as positive edges besides the class labels: the positive and negative training edges are concatenated, producing the training incidence matrix B r ∈ { 0 , 1 } n × m r and times vector t r ∈ R m r , from which we construct the training TDLG adjacency matrix A rr ∈ R m r × m r

Original: according to Equation 1.
Revised: according to Equation 1.

Original: The classifier can then be trained using the rows of A rr as feature vectors and the edge labels (i.e., positive/negative edge).
Revised: The classifier can then be trained using the rows of A rr as feature vectors and the edge labels (i.e., positive/negative edge).

Original: Now feature vectors for the test edges must each be m r -dimensional and consider only proximity to training edges.
Revised: Now feature vectors for the test edges must each be m r -dimensional and consider only proximity to training edges.

Original: Letting the test incidence matrix and test times vector be B e ∈ { 0 , 1 } n × m e and t e ∈ R m e , the test edge feature vectors will be the rows of the matrix A er ∈ R m e × m r , the ( i, j ) -th entry of which is given by
Revised: Letting the test incidence matrix and test times vector be B e ∈ { 0 , 1 } n × m e and t e ∈ R m e , the test edge feature vectors will be the rows of the matrix A er ∈ R m e × m r , the ( i, j ) -th entry of which is given by

Original: [Equation]
Revised: [Equation]

Original: which can be passed to the classifier for inference.
Revised: which can be passed to the classifier for inference.

Original:  D EMONSTRATIVE E XAMPLE
Revised:  D EMONSTRATIVE E XAMPLE

Original: To demonstrate the power of our TDLG method at capturing latent structure in temporal graphs, we introduce a simple, natural model for temporal networks based on the stochastic block model (SBM) (Holland et al., 1983).
Revised: To demonstrate the power of our TDLG method at capturing latent structure in temporal graphs, we introduce a simple, natural model for temporal networks based on the stochastic block model (SBM) (Holland et al., 1983).

Original: Our model, calledtemporal SBM, comprises the union of several SBMs.
Revised: Our model is called the union of Gaussian-timed SBMs (UGT-SBM).

Original: A normal distribution is attached to each SBM, and the edges drawn from an SBM are given a time which is sampled from its distribution.
Revised: As the name suggests, it comprises the union of several SBMs, where a normal distribution is attached to each SBM, and the edges drawn from an SBM are given a time which is sampled from its distribution.

Original: The formal defintion follows in Definition 3.
Revised: The formal defintion follows in Definition 3.

Original: Definition 3 (Temporal SBM) .
Revised: Definition 3 (UGT-SBM) .

Original: Consider a set of n nodes partitioned into two equal-sized communities { U, V } .
Revised: Consider a set of n nodes partitioned into two equal-sized communities { U, V } .

Original: For some integer ∆ > 0 and real numbers 0 < α 1 , α 2 < 1 , construct a random temporal edge set over this graph as follows: Let there be ∆ · n 2 temporal edges in each of two time periods; the times associated with edges in each time period are drawn from the normal distributions N ( µ 1 , σ 21 ) and N ( µ 2 , σ 22 ) , respectively.
Revised: For some integer ∆ > 0 and real numbers 0 < α 1 , α 2 < 1 , construct a random temporal edge set over this graph as follows: Let there be ∆ · n 2 temporal edges in each of two time periods; the times associated with edges in each time period are drawn from the normal distributions N ( µ 1 , σ 21 ) and N ( µ 2 , σ 22 ) , respectively.

Original: Of the edges in time period 1, let (1 − α 1 ) · ∆ · n 2 edges be drawn uniformly at random from U × V , and let the remaining α 1 · ∆ · n 2 edges be drawn half each from
Revised: Of the edges in time period 1, let (1 − α 1 ) · ∆ · n 2 edges be drawn uniformly at random from U × V , and let the remaining α 1 · ∆ · n 2 edges be drawn half each from

Original: U × U and V × V .
Revised: U × U and V × V .

Original: Similarly, of the edges in time period 2, let (1 − α 2 ) · ∆ · n 2 edges be drawn from
Revised: Similarly, of the edges in time period 2, let (1 − α 2 ) · ∆ · n 2 edges be drawn from

Original: U × V , and the remaining α 2 · ∆ · n 2 edges be drawn half each from U × U and V × V .
Revised: U × V , and the remaining α 2 · ∆ · n 2 edges be drawn half each from U × U and V × V .

Original: Here ∆ is the expected degree of each node in each of the two SBMs (so the total degree of each node is 2 · ∆ ), and α 1 , α 2 represent the fraction of edges in each SBM which are intra-community as opposed to inter-community.
Revised: Here ∆ is the expected degree of each node in each of the two SBMs (so the total degree of each node is 2 · ∆ ), and α 1 , α 2 represent the fraction of edges in each SBM which are intra-community as opposed to inter-community.

Original: Note that this definition can generalize straightforwardly to a union of an arbitrary number of SBMs, potentially with unequal community sizes.
Revised: This definition can generalize straightforwardly to a union of an arbitrary number of SBMs, potentially with unequal community sizes.

Revised: We note some related prior models with more sophisticated temporal dependencies based on point processes: the Hawkes IRM (Blundell et al., 2012), the block point process (BPP) model (Junuthula et al., 2019), and the community Hawkes independent pairs (CHIP) model (Arastuie et al., 2020); the latter two in particular are also based on SBMs.

Revised: Also related is the model from Barbillon et al.

Revised: (2017), which extends SBMs not to the temporal setting, but rather to the multiplex setting (i.e., multiple graphs over the same set of nodes).

Original: We first construct a TSBM where it is not possible to distinguish any community structure if one ignores temporal information.
Revised: We first construct a UGT-SBM where it is impossible to distinguish any community structure if one ignores temporal information.

Original: This TSBM is visualized in Figure 1.
Revised: This UGT-SBM is visualized in Figure 1.

Original: The graph has n = 100 nodes, with expected degrees ∆ = 40 , and the time distributions of the two SBMs are N (cid:0) − 1 , ( 1 / 2 ) 2 (cid:1) and N (cid:0) +1 , ( 1 / 2 ) 2 (cid:1) ; the distributions have the same variance, but one occurs earlier on average.
Revised: The graph has n =nodes, with expected degrees ∆ = 40 , and the time distributions of the two SBMs are N (cid:0) − 1 , ( 1 / 2 ) 2 (cid:1) and N (cid:0) +1 , ( 1 / 2 ) 2 (cid:1) ; the distributions have the same variance, but one occurs earlier on average.

Original: Further, we set α 1 = 9 / 10 and α 2 = 1 / 10 , so that the mean fraction of intra-community edges is α 1 + α 2= 1 / 2 ; this ensures that it is impossible to retrieve community structure without considering time, as is best illustrated in Figure 1 in the union of the two SBMs, which is simply an Erd˝os–R´enyi graph.
Revised: We set α 1 = 9 / 10 and α 2 = 1 / 10 , so that the mean fraction of intra-community edges is α 1 + α 2= 1 / 2 ; this ensures that it is impossible to retrieve community structure without considering time, as is best illustrated in Figure 1 in the union of the two SBMs, which is simply an Erd˝os–R´enyi graph.

Original: [Figure]
Revised: [Figure]

Original: We will show that our TDLG method can utilize the time information to recover the node communities.
Revised: We will show that our TDLG method can utilize the time information to recover the node communities.

Original: Since our method produces embeddings for temporal edges, to demonstrate recovery of node communities, we must aggregate the edge embeddings into node embeddings.
Revised: Since our method produces embeddings for temporal edges, to demonstrate recovery of node communities, we must aggregate the edge embeddings into node embeddings.

Original: Node embeddings can be straightforwardly constructed from edge embeddings as follows: for each node, return the mean of the embeddings of all edges incident on that node.
Revised: Node embeddings can be straightforwardly constructed from edge embeddings as follows: for each node, return the mean of the embeddings of all edges incident on that node.

Original: This is described formally in Definition 4:
Revised: This is described formally in Definition 4:

Original: Definition 4 (Mean-Edge Node Embeddings) .
Revised: Definition 4 (Mean-Edge Node Embeddings) .

Original: Suppose a network has n nodes and m edges, with incidence matrix B ∈ R n × m .
Revised: Suppose a network has n nodes and m edges, with incidence matrix B ∈ R n × m .

Original: Let ˜ B be the matrix that results from dividing each row of B by its sum.
Revised: Let ˜ B be the matrix that results from dividing each row of B by its sum.

Original: Given a matrix of k -dimensional edge embeddings Y ∈ R m × k , produce a matrix of node embeddings X ∈ R n × k via the matrix product X = ˜ BY .
Revised: Given a matrix of k -dimensional edge embeddings Y ∈ R m × k , produce a matrix of node embeddings X ∈ R n × k via the matrix product X = ˜ BY .

Original: In Figure 2, we show the result of applying our method to this TSBM.
Revised: In Figure 2, we show the result of applying our method to this UGT-SBM.

Original: In particular, we construct the TDLG with σ t = 1 / 2 and show the TDLG adjacency matrix, temporal edge embeddings generated from this matrix, and node embeddings generated by Definition 4.
Revised: In particular, we construct the TDLG with σ t = 1 / 2 and show the TDLG adjacency matrix, temporal edge embeddings generated from this matrix, and node embeddings generated by Definition 4.

Original: Note that while we generally use the raw rows of the TDLG adjacency matrix as edge embeddings (which produces sparse vectors), only in this section, for visualization purposes, we instead use dense eigenvector embeddings.
Revised: Note that while we generally use the raw rows of the TDLG adjacency matrix as edge embeddings (which produces sparse vectors), only in this section, for visualization purposes, we instead use dense eigenvector embeddings.

Original: The embedding visualizations show that our method is recovering the community structure:
Revised: The embedding visualizations show that our method is recovering the community structure:

Original: The edge embeddings roughly separate the 6 kinds of edges (i.e., edges in U × U , V × V , and U × V occurring in each of the two time periods), and further, the node embeddings linearly separate the two node communities U and V .
Revised: The edge embeddings roughly separate the 6 kinds of edges (i.e., edges in U × U , V × V , and U × V occurring in each of the two time periods), and further, the node embeddings linearly separate the two node communities U and V .

Original:  T HEORETICAL D ISCUSSION
Revised:  T HEORETICAL D ISCUSSION

Original: We now discuss our theoretical results, which concern the action of our TDLG method on the proposed TSBM family of synthetic graphs.
Revised: We now discuss our theoretical results, which concern the action of our TDLG method on the proposed UGT-SBM family of synthetic graphs.

Original: We present these results not only to show the power of the TDLG approach, but also to contrast with deep methods, the complexity of which often precludes explicit analysis.
Revised: We present these results not only to show the power of the TDLG approach, but also to contrast with deep methods, the complexity of which often precludes explicit analysis.

Original: In particular, under certain strong assumptions, our approach’s simplicity allows us to give the exact expectation for the TDLG adjacency matrix A for TSBMs.
Revised: In particular, under certain strong assumptions, our approach’s simplicity allows us to give the exact expectation for the TDLG adjacency matrix A for UGT-SBMs.

Original: Further,we can show constructively that the TDLG method is able to preserve and disentangle the node community structure of the TSBM: specifically, taking the temporal edge embeddings returned by our method and aggregating them into node embeddings via averaging over a node’s incident edges (as in Definition 4) yields node embeddings which can distinguish the two node communities. The assumptions for these two results are as follows: we analyze using the expected TDLG adjacency matrix and the
Revised: Further,

Original: [Figure]
Revised: [Figure]

Revised: we can show constructively that the TDLG method is able to preserve and disentangle the node community structure of the UGT-SBM: specifically, taking the temporal edge embeddings returned by our method and aggregating them into node embeddings via averaging over a node’s incident edges (as in Definition 4) yields node embeddings which can distinguish the two node communities.

Original:  expected incidence matrix, and we assume zero time variance within time periods of the TSBM; these assumptions all simplify the analysis by reducing variance, though perhaps via concentration bounds one could extend such results to the sampled setting (Spielman, 2012).
Revised: The assumptions for these two results are as follows: we analyze using the expected TDLG adjacency matrix and the expected incidence matrix, and we assume zero time variance within time periods of the UGT-SBM; these assumptions all simplify the analysis by reducing variance, though perhaps via concentration bounds one could extend such results to the sampled setting (Spielman, 2012).

Original: Also, we will consider TSBMs in the limit as n → ∞ for simplicity of the resulting constant values, but the following analysis would carry through in general with different values.
Revised: Also, we will consider UGT-SBMs in the limit as n → ∞ for simplicity of the resulting constant values, but the following analysis would carry through in general with different values.

Original: Proposition 4.1 (Structure of TDLG Adjacency Matrix for Temporal SBMs) .
Revised: Proposition 4.1 (Structure of TDLG Adjacency Matrix for UGT-SBMs) .

Original: Suppose A ∈ R m × m is the expected TDLG adjacency matrix of a Temporal SBM graph with node communities { U, V } , time periods { 1 , 2 } , and zero time variance ( σ 21 =
Revised: Suppose A ∈ R m × m is the expected TDLG adjacency matrix of a UGT-SBM graph with node communities { U, V } , time periods { 1 , 2 } , and zero time variance ( σ 21 =

Original: σ 22 = 0 ).
Revised: σ 22 = 0 ).

Original: A has a 6 × 6 block structure where each block is constant-valued, that is, a multiple of an all-ones matrix J of some dimensionality.
Revised: A has a 6 × 6 block structure where each block is constant-valued, that is, a multiple of an all-ones matrix J of some dimensionality.

Original: Letting subscripts denote time period, the block matrix is indexed by 6 edge sets: ( U × U ) 1 , ( V × V ) 1 , ( U × V ) 1 , ( U × U ) 2 , ( V × V ) 2 , and ( U × V ) 2 .
Revised: Letting subscripts denote time period, the block matrix is indexed by 6 edge sets: ( U × U ) 1 , ( V × V ) 1 , ( U × V ) 1 , ( U × U ) 2 , ( V × V ) 2 , and ( U × V ) 2 .

Original: In the limit as n → ∞ , the constant values of each block are given by the Kronecker product 
Revised: In the limit as n → ∞ , the constant values of each block are given by the Kronecker product

Original: U × U 8 / n 4 / n V × V 8 / n 4 / n
Revised: [Equation]

Original: U × V 4 / n 4 / n 4 / n   ⊗ (cid:18) λ λ (cid:19) , where γ = exp (cid:16) − ( µ 1 − µ 2 ) 2 2 σ 2 t (cid:17) .
Revised: , where γ = exp (cid:16) − ( µ 1 − µ 2 ) 2 2 σ 2 t (cid:17) .

Original: Note that writing A as a Kronecker product involves some abuse of notation, and the actual sizes of the edge sets (and hence the sizes of the blocks) are as given in Definition 3.
Revised: Note that writing A as a Kronecker product involves some abuse of notation, and the actual sizes of the edge sets (and hence the sizes of the blocks) are as given in Definition 3.

Original: In the order of
Revised: In the order of

Original: Proposition 4.1, these sizes are α 1 · ∆ · n 4 , α 1 · ∆ · n 4 , (1 − α 1 ) · ∆ · n 2 , α 2 · ∆ · n 4 , α 2 · ∆ · n 4 , and (1 − α 2 ) · ∆ · n 2 .
Revised: Proposition 4.1, these sizes are α 1 · ∆ · n 4 , α 1 · ∆ · n 4 , (1 − α 1 ) · ∆ · n 2 , α 2 · ∆ · n 4 , α 2 · ∆ · n 4 , and (1 − α 2 ) · ∆ · n 2 .

Original: Proof.
Revised: Proof.

Original: We calculate an entry A ij of this matrix.
Revised: We calculate an entry A ij of this matrix.

Original: We first consider the effect of the time decay, then the effect of edge adjacency (i.e., b ⊤ i b j ).
Revised: We first consider the effect of the time decay, then the effect of edge adjacency (i.e., b ⊤ i b j ).

Original: Given the assumption of zero time variance, there are only two possible values for each temporal edge’s time, µ 1 and µ 2 .
Revised: Given the assumption of zero time variance, there are only two possible values for each temporal edge’s time, µ 1 and µ 2 .

Original: This means effect of time decay on each entry of A is multiplication by either 1 , if i and j occur at the same time, or otherwise γ .
Revised: This means effect of time decay on each entry of A is multiplication by either 1 , if i and j occur at the same time, or otherwise γ .

Original: Second, we calculate the expected value of b ⊤ i b j , that is, the expected number of endpoint nodes shared between edges i and j .
Revised: Second, we calculate the expected value of b ⊤ i b j , that is, the expected number of endpoint nodes shared between edges i and j .

Original: Suppose both i and j are in U × U .
Revised: Suppose both i and j are in U × U .

Original: Since there are n / 2 nodes in U , the probability of any two random nodes in U being the same is 2 / n .
Revised: Since there are n / 2 nodes in U , the probability of any two random nodes in U being the same is 2 / n .

Original: There are 4 pairs of endpoints in U between i and j ; in the assumed n → ∞ limit, the 4 events of these pairs each having the same two nodes tend toward independence, so b i · b j = 4 · 2 / n = 8 ⁄ n .
Revised: There are 4 pairs of endpoints in U between i and j ; in the assumed n → ∞ limit, the 4 events of these pairs each having the same two nodes tend toward independence, so b i · b j = 4 · 2 / n = 8 ⁄ n .

Original: By similar logic, if i is in U × U and j is in U × V , there are 2 pairs of endpoints that could be identical (each of i ’s nodes in U with the single node of j in U ), so b i · b j = 2 · 2 / n = 4 ⁄ n .
Revised: By similar logic, if i is in U × U and j is in U × V , there are 2 pairs of endpoints that could be identical (each of i ’s nodes in U with the single node of j in U ), so b i · b j = 2 · 2 / n = 4 ⁄ n .

Original: The same holds if both i and j are in U × V .
Revised: The same holds if both i and j are in U × V .

Original: Finally, if i is in U × U and j is in V × V , there is zero chance of a shared endpoint.
Revised: Finally, if i is in U × U and j is in V × V , there is zero chance of a shared endpoint.

Original: Remaining combinations follow by symmetry.
Revised: Remaining combinations follow by symmetry.

Original: Combining the terms for time decay and edge adjacency, the expected TDLG adjacency matrix has the specified 6 × 6 block structure.
Revised: Combining the terms for time decay and edge adjacency, the expected TDLG adjacency matrix has the specified 6 × 6 block structure.

Original: ■
Revised: ■

Original: We now state the second result, about the ability to distinguish TSBM node communities using the TDLG method.
Revised: We now state the second result, about the ability to distinguish UGT-SBM node communities using the TDLG method.

Original: This result requires that it is not the case that α 1 = α 2 = 1 / 2 ; in this case of TSBM, the SBMs at both time periods are Erd˝os–R´enyi, so there is no community structure to recover.
Revised: This result requires that it is not the case that α 1 = α 2 = 1 / 2 ; in this case of UGT-SBM, the SBMs at both time periods are Erd˝os–R´enyi, so there is no community structure to recover.

Original: Proposition 4.2 (TDLG Embedding Recovers Communities from Temporal SBMs) .
Revised: Proposition 4.2 (TDLG Embedding Recovers Communities from UGT-SBMs) .

Original: Suppose A ∈ R m × m is the expected TDLG adjacency matrix of a Temporal SBM graph with node communities { U, V } , time periods { 1 , 2 } , and zero time variance ( σ 21 = σ 22 = 0 ).
Revised: Suppose A ∈ R m × m is the expected TDLG adjacency matrix of a UGT-SBM graph with node communities { U, V } , time periods { 1 , 2 } , and zero time variance ( σ 21 = σ 22 = 0 ).

Original: Let X ∈ R n × m be the mean-edge node embeddings resulting from treating the rows of A as edge embeddings.
Revised: Let X ∈ R n × m be the mean-edge node embeddings resulting from treating the rows of A as edge embeddings.

Original: Assuming that γ ̸ = 1 and it is not the case that α 1 = α 2 = 1 / 2 , the node communities are distinguishable in X .
Revised: Assuming that γ ̸ = 1 and it is not the case that α 1 = α 2 = 1 / 2 , the node communities are distinguishable in X .

Original: For brevity, the proof is deferred to Appendix A.1.
Revised: For brevity, the proof is deferred to Appendix A.1.

Original: It essentially involves the same flavor of countingbased arguments as Proposition 4.1, though it is much more involved.
Revised: It essentially involves the same flavor of countingbased arguments as Proposition 4.1, though it is much more involved.

Original: As part of this proof, we provide a description of part of the expected block structure of the resulting node embeddings X , similar to the preceding description of the TDLG adjacency matrix A .
Revised: As part of this proof, we provide a description of part of the expected block structure of the resulting node embeddings X , similar to the preceding description of the TDLG adjacency matrix A .

Original: When time decay is not applied (i.e., by setting γ = 1 ), we find that nodes in U and V become indistinguishable in the derived embeddings for TSBMs with α 1 + α 2 = 1 , as opposed to
Revised: When time decay is not applied (i.e., by setting γ = 1 ), we find that nodes in U and V become indistinguishable in the derived embeddings for UGT-SBMs with α 1 + α 2 = 1 , as opposed to

Original: just when α 1 = α 2 = 1 / 2 .
Revised: just when α 1 = α 2 = 1 / 2 .

Original: These are TSBMs like the one from Figure 1, which do have community structure when considering time, but become Erd˝os–R´enyi graphs when ignoring time.
Revised: These are UGT-SBMs like the one from Figure 1, which do have community structure when considering time, but become Erd˝os–R´enyi graphs when ignoring time.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Node embeddings Many methods have been proposed over the years for the node embedding task.
Revised: Node embeddings Many methods have been proposed over the years for the node embedding task.

Original: The best understood are the classical ‘spectral’ methods based on eigendecomposition of the graph’s adjacency matrix or Laplacian (Roweis & Saul, 2000; Belkin & Niyogi, 2001).
Revised: The best understood are the classical ‘spectral’ methods based on eigendecomposition of the graph’s adjacency matrix or Laplacian (Roweis & Saul, 2000; Belkin & Niyogi, 2001).

Original: Perhaps the most famous non-spectral method is DeepWalk (Perozzi et al., 2014), which takes random walks on the graph, then, using a nonlinear, probabilistic objective, increases the similarity of the embeddings of nodes which frequently co-occur in the walks.
Revised: Perhaps the most famous non-spectral method is DeepWalk (Perozzi et al., 2014), which takes random walks on the graph, then, using a nonlinear, probabilistic objective, increases the similarity of the embeddings of nodes which frequently co-occur in the walks.

Original: Similar methods include node2vec (Grover & Leskovec, 2016), which adds bias to the random walks, and LINE (Tang et al., 2015), which increases scalability.
Revised: Similar methods include node2vec (Grover & Leskovec, 2016), which adds bias to the random walks, and LINE (Tang et al., 2015), which increases scalability.

Original: GraRep (Cao et al., 2015) and NetMF (Qiu et al., 2018) incorporate both matrix factorization like the classical methods and nonlinear objectives like the recent ones, yielding good speed and performance.
Revised: GraRep (Cao et al., 2015) and NetMF (Qiu et al., 2018) incorporate both matrix factorization like the classical methods and nonlinear objectives like the recent ones, yielding good speed and performance.

Original: Unsupervised deep learning approaches, like VGAE (Kipf & Welling, 2016) and SDNE (Wang et al., 2016), involve deep autoencoders; besides this, there is a wide array of supervised deep models which implicitly form node representations, including graph convolutional networks (GCNs) (Kipf &
Revised: Unsupervised deep learning approaches, like VGAE (Kipf & Welling, 2016) and SDNE (Wang et al., 2016), involve deep autoencoders; besides this, there is a wide array of supervised deep models which implicitly form node representations, including graph convolutional networks (GCNs) (Kipf &

Original: Welling, 2017) and graph attention networks (GATs) (Veli ˇ ckovi ´ c et al., 2018).
Revised: Welling, 2017) and graph attention networks (GATs) (Veli ˇ ckovi ´ c et al., 2018).

Original: Edge embeddings The task of creating vector representations for each edge, rather than each node, has seen less exploration.
Revised: Edge embeddings The task of creating vector representations for each edge, rather than each node, has seen less exploration.

Original: A common strategy is to simply create node embeddings, then process them into edge embeddings by aggregating the embeddings of the two endpoints of an edge.
Revised: A common strategy is to simply create node embeddings, then process them into edge embeddings by aggregating the embeddings of the two endpoints of an edge.

Original: In particular, this is done by taking the mean, entrywise product, cross product, or concatenation of the two node embedding vectors.
Revised: In particular, this is done by taking the mean, entrywise product, cross product, or concatenation of the two node embedding vectors.

Original: Notably, node2vec uses this strategy for link prediction; Shi et al.
Revised: Notably, node2vec uses this strategy for link prediction; Shi et al.

Original: (2018) and Verma et al.
Revised: (2018) and Verma et al.

Original: (2019) are more examples where this is part of a larger framework.
Revised: (2019) are more examples where this is part of a larger framework.

Original: More along the lines of our method, though still not involving temporal networks, Bandyopadhyay et al.
Revised: More along the lines of our method, though still not involving temporal networks, Bandyopadhyay et al.

Original: (2019) produce edge representations in an unsupervised manner by embedding the nodes of the line graph of the original network.
Revised: (2019) produce edge representations in an unsupervised manner by embedding the nodes of the line graph of the original network.

Original: The method of Li et al.
Revised: The method of Li et al.

Original: (2017b) implicitly takes a similar approach by taking a random walk on the edges and iteratively updating edge embeddings, as DeepWalk does for node embeddings.
Revised: (2017b) implicitly takes a similar approach by taking a random walk on the edges and iteratively updating edge embeddings, as DeepWalk does for node embeddings.

Original: Some graph deep learning approaches (Monti et al., 2018;
Revised: Some graph deep learning approaches (Monti et al., 2018;

Original: Gong & Cheng, 2019) include layers which form edge embeddings, but do so as part of a supervised framework; by contrast,
Revised: Gong & Cheng, 2019) include layers which form edge embeddings, but do so as part of a supervised framework; by contrast,

Original: Zhou et al.
Revised: Zhou et al.

Original: (2018b) propose an unsupervised deep method based on generative adversarial networks, though their approach is fairly complicated.
Revised: (2018b) propose an unsupervised deep method based on generative adversarial networks, though their approach is fairly complicated.

Original: Also of note, though not directly applicable to the standard or temporal edge setting, several works in the area of knowledge graphs form embeddings of relations between entities (Bordes et al., 2013; Yang et al., 2014; Chen & Quirk, 2019).
Revised: Also of note, though not directly applicable to the standard or temporal edge setting, several works in the area of knowledge graphs form embeddings of relations between entities (Bordes et al., 2013; Yang et al., 2014; Chen & Quirk, 2019).

Original: Embeddings for temporal networks Many methods for temporal network embedding can be seen as variants of those for static graphs.
Revised: Embeddings for temporal networks Many methods for temporal network embedding can be seen as variants of those for static graphs.

Original: Dunlavy et al.
Revised: Dunlavy et al.

Original: (2011) propose a variant of factorization-based approaches, intended for a dynamic network comprising snapshots of static graphs at consecutive intervals of time.
Revised: (2011) propose a variant of factorization-based approaches, intended for a dynamic network comprising snapshots of static graphs at consecutive intervals of time.

Original: It directly incorporates time information by stacking adjacency matrices of different time steps and employing tensor factorization.
Revised: It directly incorporates time information by stacking adjacency matrices of different time steps and employing tensor factorization.

Original: DANE (Li et al., 2017a) is another factorization approach, though it only factorizes matrices.
Revised: DANE (Li et al., 2017a) is another factorization approach, though it only factorizes matrices.

Original: To boost efficiency, rather than computing node embeddings from scratch for each snapshot, DANE iteratively updates the embeddings from the previous time step according to the change in the network; TIMERS (Zhang et al., 2018) is a similar approach which analyzes the growth of the error over iterative updates to determine when it isnecessary to re-embed from scratch. NetWalk (Yu et al., 2018) is a variant of random walk-based
Revised: To boost efficiency, rather than computing node embeddings from scratch for each snapshot, DANE iteratively updates the embeddings from the previous time step according to the change in the network; TIMERS (Zhang et al., 2018) is a similar approach which analyzes the growth of the error over iterative updates to determine when it is

Original: [Table]
Revised: [Table]

Original:  methods like DeepWalk; along the same lines, it saves time by efficiently updating embeddings at each snapshot using warm starts and reservoir sampling.
Revised: necessary to re-embed from scratch. NetWalk (Yu et al., 2018) is a variant of random walk-based methods like DeepWalk; along the same lines, it saves time by efficiently updating embeddings at each snapshot using warm starts and reservoir sampling.

Original: CTDNE (Nguyen et al., 2018) is another random walk method which incorporates time information using the constraint that the walks must obey time.
Revised: CTDNE (Nguyen et al., 2018) is another random walk method which incorporates time information using the constraint that the walks must obey time.

Original: Notably, like our method, CTDNE is designed for networks with continuous time rather than just snapshots of different time intervals.
Revised: Notably, like our method, CTDNE is designed for networks with continuous time rather than just snapshots of different time intervals.

Original: Among unsupervised deep approaches, DynGEM (Goyal et al., 2018) is an autoencoder method which again uses warm starts for each snapshot, both for speed and to encourage embeddings to be approximately preserved across time steps; DynGraph2Vec (Goyal et al., 2020) more directly allows information at different snapshots to be integrated by employing an RNN to evolve representations across time steps.
Revised: Among unsupervised deep approaches, DynGEM (Goyal et al., 2018) is an autoencoder method which again uses warm starts for each snapshot, both for speed and to encourage embeddings to be approximately preserved across time steps; DynGraph2Vec (Goyal et al., 2020) more directly allows information at different snapshots to be integrated by employing an RNN to evolve representations across time steps.

Original: The similarly-named DyLink2Vec (Rahman et al., 2018) is another deep autoencoder approach, which also integrates information across time steps; notably, like our method, it yields embeddings for links rather than nodes, though it works on snapshots rather than continuous time.
Revised: The similarly-named DyLink2Vec (Rahman et al., 2018) is another deep autoencoder approach, which also integrates information across time steps; notably, like our method, it yields embeddings for links rather than nodes, though it works on snapshots rather than continuous time.

Original: More recently, EvolveGCN (Pareja et al., 2020) proposes the interesting idea of using an RNN to evolve not the node embeddings, but the weights of a GCN model which generates embeddings for each snapshot.
Revised: More recently, EvolveGCN (Pareja et al., 2020) proposes the interesting idea of using an RNN to evolve not the node embeddings, but the weights of a GCN model which generates embeddings for each snapshot.

Original: Some deep methods also handle continuous time: using point processes, Know-Evolve (Trivedi et al., 2017), HTNE (Zuo et al., 2018), and DyRep (Trivedi et al., 2019) model the occurrence of temporal edges, while Dynamic-Triad (Zhou et al., 2018a) aims to capture structural information by modeling the closure of wedges into triangles over time. 
Revised: Some deep methods also handle continuous time: using point processes, Know-Evolve (Trivedi et al., 2017), HTNE (Zuo et al., 2018), and DyRep (Trivedi et al., 2019) model the occurrence of temporal edges, while Dynamic-Triad (Zhou et al., 2018a) aims to capture structural information by modeling the closure of wedges into triangles over time. More recent methods handling continuous-time include TGAT (Xu et al., 2020), TGNs (Rossi et al., 2020),

Revised: MeTa (Wang et al., 2021), and PINT (Souza et al., 2022); these methods adapt and integrate various recent deep learning modules and concepts into the temporal network setting, such as self-attention, memory, data augmentation, and positional embedding.

Revised: The PINT paper in particular performs an analysis of the continuous-time and discrete-time settings, proving that the latter setting reduces to the former without loss of information, whereas the converse is not always true, and hence the class of methods handling continuous-time directly is theoretically more powerful in this sense.

Original: Most of these approaches  are principally node embedding methods from which edge embeddings can be generated by aggregation; to our knowledge, ours is the first work which directly forms embeddings for continuous-time temporal edges without supervision and without aggregating node embeddings.
Revised: Of the many approaches for temporal networks that we have discussed, most are principally node embedding methods from which edge embeddings can be generated by aggregation; to our knowledge, ours is the first work which directly forms embeddings for continuous-time temporal edges without supervision and without aggregating node embeddings.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate our method on two kinds of tasks, edge classification and temporal link prediction, on a collection of five real-world temporal network datasets.
Revised: We evaluate our method on two kinds of tasks, edge classification and temporal link prediction, on a collection of five real-world temporal network datasets.

Original: The statistics for these networks are provided in Table 2, but we defer discussion of these datasets to Appendix A.2.
Revised: The statistics for these networks are provided in Table 2, but we defer discussion of these datasets to Appendix A.2.

Original: 6. E DGE CLASSIFICATION
Revised: 6. E DGE CLASSIFICATION

Original: Experimental setup We use a logistic regression classifier to which the input is a feature vector for each edge of the network and the target is a binary edge class.
Revised: Experimental setup We use a logistic regression classifier to which the input is a feature vector for each edge of the network and the target is a binary edge class.

Original: We make random 70%/30% splits of training/test data, and report test AUC of binary classification across 10 trials with 10 random splits.
Revised: We make random 70%/30% splits of training/test data, and report test AUC of binary classification across 10 trials with 10 random splits.

Original: In addition to results for our TDLG method’s sparse embeddings, we report results for several other methods.
Revised: In addition to results for our TDLG method’s sparse embeddings, we report results for several other methods.

Original: We compare to results for some prior methods applied to these datasets: 1) CTDNE (Nguyen et al., 2018); 2) TIMERS (Zhang et al., 2018); 3) EvolveGCN
Revised: We compare to results for some prior methods applied to these datasets: 1) CTDNE (Nguyen et al., 2018); 2) TIMERS (Zhang et al., 2018); 3) EvolveGCN

Original: (Pareja et al., 2020); 4) DynGEM (Goyal et al., 2018); 5) GCRN (Seo et al., 2018); and 6) VGRNN (Hajiramezanali et al., 2019).
Revised: (Pareja et al., 2020); 4) DynGEM (Goyal et al., 2018); 5) GCRN (Seo et al., 2018); and 6) VGRNN (Hajiramezanali et al., 2019).

Original: Note that these methods output 128 -dimensional dense embeddings, unlike our TDLG method, which outputs m -dimensional sparse embeddings; for an additional, direct comparison, we also report results  for using the top 128 eigenvectors (i.e., those with highest magnitude eigenvalues) of the TDLG adjacency matrix (‘TDLG Dense’).
Revised: Note that these methods output 128 -dimensional dense embeddings, unlike our TDLG method, which outputs m -dimensional sparse embeddings; for an additional, direct comparison, we also report results [Figure] for using the top 128 eigenvectors (i.e., those with highest magnitude eigenvalues) of the TDLG adjacency matrix (‘TDLG Dense’).

Original: With the exception of the methods we introduce and CTDNE, all other methods do not directly handle the continuous time stamps of the datasets; for these methods, we slot the edges into discrete snapshots by evenly dividing the full time span into 20 intervals.
Revised: With the exception of the methods we introduce and CTDNE, all other methods do not directly handle the continuous time stamps of the datasets; for these methods, we slot the edges into discrete snapshots by evenly dividing the full time span into 20 intervals.

Original: All experiments are run on an Xeon Gold 6130 CPU and Tesla v100 16GB GPU; only the latter four methods (i.e., the deep methods) use the GPU, and we are unable to run these four methods on
Revised: All experiments are run on an Xeon Gold 6130 CPU and Tesla v100 16GB GPU; only the latter four methods (i.e., the deep methods) use the GPU, and we are unable to run these four methods on

Original: E PINIONS due to GPU memory limitations.
Revised: E PINIONS due to GPU memory limitations.

Original: We include code in the form of a Jupyter notebook (P´erez & Granger, 2007) demo in the supplementary material.
Revised: We include code in the form of a Jupyter notebook (P´erez & Granger, 2007) demo in the supplementary material.

Original: Hyperparameter selection For all embedding methods, we use the scikit-learn (Pedregosa et al., 2011) implementation of logistic regression; we increase the maximum iterations to 10 3 , and, since the edge classes are generally imbalanced across the datasets, we set the class weight option to ‘balanced,’ which adjusts loss weights inversely in proportion to class frequency.
Revised: Hyperparameter selection For all embedding methods, we use the scikit-learn (Pedregosa et al., 2011) implementation of logistic regression; we increase the maximum iterations to 10 3 , and, since the edge classes are generally imbalanced across the datasets, we set the class weight option to ‘balanced,’ which adjusts loss weights inversely in proportion to class frequency.

Original: We keep otherwise default options.
Revised: We keep otherwise default options.

Original: For our TDLG method, we set the time scale hyperparameter σ t as ratio of the standard deviation of the edges’ times; calling this standard deviation be σ T , we use σ t = 10 − 1 · σ T , which is chosen by informal tuning.
Revised: For our TDLG method, we set the time scale hyperparameter σ t as ratio of the standard deviation of the edges’ times; calling this standard deviation be σ T , we use σ t = 10 − 1 · σ T , which is chosen by informal tuning.

Original: In Appendix A.3, we explore the impact of varying σ t .We use the implementation of Liu et al.
Revised: In Appendix A.3, we explore the impact of varying σ t .We use the implementation of Liu et al.

Original: (2020) for TIMERS and all deep methods, all with default hyperparameters.
Revised: (2020) for TIMERS and all deep methods, all with default hyperparameters.

Original: For the E PINIONS dataset only, for our TDLG method, we modify the solver for sparse logistic regression to the ‘saga’ solver, which is more scalable than the default ‘lbfgs’ solver, and reduce the maximum number of iterations to 100 (the default value).
Revised: For the E PINIONS dataset only, for our TDLG method, we modify the solver for sparse logistic regression to the ‘saga’ solver, which is more scalable than the default ‘lbfgs’ solver, and reduce the maximum number of iterations to 100 (the default value).

Original: Results We first plot mean AUC and 95% confidence intervals for selected methods - our TDLG,
Revised: Results We first plot mean AUC and 95% confidence intervals for selected methods - our TDLG,

Original: CTDNE, TIMERS, EvolveGCN, and DynGEM - on all datasets in Figure 3.
Revised: CTDNE, TIMERS, EvolveGCN, and DynGEM - on all datasets in Figure 3.

Original: Our TDLG method achieves higher performance than the comparison methods on all datasets, often by a large margin.
Revised: Our TDLG method achieves higher performance than the comparison methods on all datasets, often by a large margin.

Original: [Figure]
Revised: [Figure]

Original: , TIMERS, EvolveGCN, and DynGEM.

Original: To compactly compare all 9 of the methods across these datasets, we aggregate the performance across the datasets as follows: we plot the mean across all datasets, excluding E PINIONS , of each method’s test AUC.
Revised: To compactly compare all 9 of the methods across these datasets, we aggregate the performance across the datasets as follows: we plot the mean across all datasets, excluding E PINIONS , of each method’s test AUC.

Original: We exclude E PINIONS since we are unable to run some methods on this dataset.
Revised: We exclude E PINIONS since we are unable to run some methods on this dataset.

Original: See Figure 4 (left).
Revised: See Figure 4 (left).

Original: TDLG achieves the highest performance out of all these methods on all datasets; the dense variant of TDLG, which perhaps provides more direct comparison with other methods, sees reduced performance, but also outperforms all prior methods.
Revised: TDLG achieves the highest performance out of all these methods on all datasets; the dense variant of TDLG, which perhaps provides more direct comparison with other methods, sees reduced performance, but also outperforms all prior methods.

Original: Tables of full experimental results are deferred to Appendix A.4.
Revised: Tables of full experimental results are deferred to Appendix A.5.

Original: [Figure]

Original: Runtime We evaluate the time efficiency of our approach against the selected baseline methods.
Revised: Runtime We evaluate the time efficiency of our approach against the selected baseline methods.

Original: For this, we report the mean runtime in seconds for one trial of edge classification.
Revised: For this, we report the mean runtime in seconds for one trial of edge classification.

Original: Runtime constitutes one trial of learning embeddings, learning a logistic regression classifier on training data, and predicting on test data.
Revised: Runtime constitutes one trial of learning embeddings, learning a logistic regression classifier on training data, and predicting on test data.

Original: See Figure 5.
Revised: See Figure 5.

Original: Across all five datasets, our TDLG method is the fastest, often by a significant margin.
Revised: Across all five datasets, our TDLG method is the fastest, often by a significant margin.

Revised: We discuss the scalability of our method in Appendix A.4.

Revised: [Figure]

Original: 6. L INK P REDICTION
Revised: 6. L INK P REDICTION

Original: Experimental setup Given the dataset of actual, ‘positive’ edge tuples ( u i , v i , t i ) , we generate an equal-sized set of fake, ‘negative’ edges by independently shuffling the three columns (i.e., independently shuffling the set of first nodes, the set of second nodes, and the times).
Revised: Experimental setup Given the dataset of actual, ‘positive’ edge tuples ( u i , v i , t i ) , we generate an equal-sized set of fake, ‘negative’ edges by independently shuffling the three columns (i.e., independently shuffling the set of first nodes, the set of second nodes, and the times).

Original: Note that this preserves the distribution nodes/times in of each of the columns.
Revised: Note that this preserves the distribution nodes/times in of each of the columns.

Original: The task is to distinguish positive[Figure] edges from negative ones.
Revised: The task is to distinguish positive edges from negative ones.

Original: We evaluate two different settings for link prediction.
Revised: We evaluate two different settings for link prediction.

Original: In the first, which we call temporal link outlier detection, the test edges are in the same time interval as the training edges.
Revised: In the first, which we call temporal link outlier detection, the test edges are in the same time interval as the training edges.

Original: In the second, which we call temporal link prediction, the test edges are in an interval of time following that of the training data.
Revised: In the second, which we call temporal link prediction, the test edges are in an interval of time following that of the training data.

Original: We also call these settings ‘interpolative’ and ‘extrapolative,’ respectively.
Revised: We also call these settings ‘interpolative’ and ‘extrapolative,’ respectively.

Original: Prior work generally focuses on the latter setting, since it is more applicable, e.g., for predicting future events/behavior, but we also evaluate the former mainly out of scientific interest.
Revised: Prior work generally focuses on the latter setting, since it is more applicable, e.g., for predicting future events/behavior, but we also evaluate the former mainly out of scientific interest.

Original: Specifically, given that we are splitting the data into 20 intervals, we consider intervals 1-19 to be ‘early’ data and interval 20 to be ‘late’ data.
Revised: Specifically, given that we are splitting the data into 20 intervals, we consider intervals 1-19 to be ‘early’ data and interval 20 to be ‘late’ data.

Original: To share computational effort across the two settings, we train both classifiers on 70% of edges (both positive and negative) from the early intervals.
Revised: To share computational effort across the two settings, we train both classifiers on 70% of edges (both positive and negative) from the early intervals.

Original: For the interpolative setting, the test data comprises the remaining 30% of edges from the early intervals.
Revised: For the interpolative setting, the test data comprises the remaining 30% of edges from the early intervals.

Original: For the extrapolative setting, the test data comprises all edges from the late interval.
Revised: For the extrapolative setting, the test data comprises all edges from the late interval.

Original: Evaluating the link prediction task has greater computational cost compared to edge classification since the input graph changes across trials rather than just train/test splits of labels, requiring new embeddings to be made for each trial.
Revised: Evaluating the link prediction task has greater computational cost compared to edge classification since the input graph changes across trials rather than just train/test splits of labels, requiring new embeddings to be made for each trial.

Original: For this reason, we only test the 5 selected methods from the previous section.
Revised: For this reason, we only test the 5 selected methods from the previous section.

Original: We report mean AUC and 95% confidence intervals across 5 trials, each trial having different random shuffles producing different negative edges, as well as different 70%/30% train/test splits.
Revised: We report mean AUC and 95% confidence intervals across 5 trials, each trial having different random shuffles producing different negative edges, as well as different 70%/30% train/test splits.

Original: Hyperparameter selection We generally use the same hyperparameters as for edge classification, with one exception: since there is an equal number of positive and negative edges, we no longer set the class weight option to ‘balanced,’ and all edges are given equal weight.
Revised: Hyperparameter selection We generally use the same hyperparameters as for edge classification, with one exception: since there is an equal number of positive and negative edges, we no longer set the class weight option to ‘balanced,’ and all edges are given equal weight.

Original: Results As for edge classification, we plot the aggregated (mean) performance across the datasets in Figure 4 (right), for both the interpolative and extrapolative settings of link prediction.
Revised: Results As for edge classification, we plot the aggregated (mean) performance across the datasets in Figure 4 (right), for both the interpolative and extrapolative settings of link prediction.

Original: For brevity, we defer non-aggregated results to Figure 7 in Appendix A.4, where we also include results for link prediction in tabular form.
Revised: For brevity, we defer non-aggregated results to Figure 8 in Appendix A.5, where we also include results for link prediction in tabular form.

Original: In both settings, across all datasets, our TDLG method achieves the highest performance, often by a large margin.
Revised: In both settings, across all datasets, our TDLG method achieves the highest performance, often by a large margin.

Original: We do find that the deep methods EvolveGCN and DynGEM generally perform better than CTDNE and TIMERS, and EvolveGCN in particular is competitive with TDLG on some datasets.
Revised: We do find that the deep methods EvolveGCN and DynGEM generally perform better than CTDNE and TIMERS, and EvolveGCN in particular is competitive with TDLG on some datasets.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We present a novel framework for temporal edge embedding called the time-decayed line graph (TDLG).
Revised: We present a novel framework for temporal edge embedding called the time-decayed line graph (TDLG).

Original: Unlike prior methods, our method works directly with continuous timestamps rather than requiring discretization of times, and directly generates temporal edge embeddings rather than requiring aggregation of node embeddings.
Revised: Unlike prior methods, our method works directly with continuous timestamps rather than requiring discretization of times, and directly generates temporal edge embeddings rather than requiring aggregation of node embeddings.

Original: Also in contrast to prior methods, ours has a very simple concept and high ease of implementation, since it is essentially based on creating a sparse matrix of proximities between temporal edges.
Revised: Also in contrast to prior methods, ours has a very simple concept and high ease of implementation, since it is essentially based on creating a sparse matrix of proximities between temporal edges.

Original: We show that, despite its simplicity, our method achieves superior performance on several downstream tasks on benchmark temporal network datasets, while requiring less runtime.
Revised: We show that, despite its simplicity, our method achieves superior performance on several downstream tasks on benchmark temporal network datasets, while requiring less runtime.

Original: Its simplicity also facilitates theoretical demonstrations of its effectiveness on our proposed TSBM family of graphs.
Revised: Its simplicity also facilitates theoretical demonstrations of its effectiveness on our proposed UGT-SBM family of graphs.

Original: An interesting direction for future work is further increasing scalability via reduction of the number of features. The TDLG method returns m features per temporal edge based on proximities to all other temporal edges. Perhaps it is possible to efficiently reduce the feature space by only considering proximities to certain edges, e.g., edges with higher effective resistances. Along the same lines, it may also be interesting to explore ways of reducing the feature space by efficiently forming a low-rank representation of the TDLG adjacency matrix, for example, via a stochastic method.
Revised: Future directions include extensions of our approach to more general scenarios involving, for example, streaming input data or transductive learning.

Original: A A PPENDIX A. P ROOF OF P ROPOSITION 4. 
Revised: A A PPENDIX A. P ROOF OF P ROPOSITION 4.

Original: We restate Proposition 4.2 from Section 4 and provide the deferred proof.
Revised: We restate Proposition 4.2 from Section 4 and provide the deferred proof.

Original: Note that we make a slight change to the notation to facilitate the proof.
Revised: Note that we make a slight change to the notation to facilitate the proof.

Original: As in Section 4, we assume the limit as number of nodes n → ∞ for simplicity of the constant values, though it is not crucial to the analysis.
Revised: As in Section 4, we assume the limit as number of nodes n → ∞ for simplicity of the constant values, though it is not crucial to the analysis.

Original: Proposition A.1 (TDLG Embedding Recovers Communities from Temporal SBMs) .
Revised: Proposition A.1 (TDLG Embedding Recovers Communities from UGT-SBMs) .

Original: Suppose
Revised: Suppose

Original: A ∈ R m × m is the expected TDLG adjacency matrix of a Temporal SBM graph with node communities { U, V } , time periods { 1 , 2 } , and zero time variance ( σ 21 = σ 22 = 0 ).
Revised: A ∈ R m × m is the expected TDLG adjacency matrix of a UGT-SBM graph with node communities { U, V } , time periods { 1 , 2 } , and zero time variance ( σ 21 = σ 22 = 0 ).

Original: Let ˜ X ∈ R n × m be the meanedge node embeddings resulting from treating the rows of A as edge embeddings.
Revised: Let ˜ X ∈ R n × m be the mean-edge node embeddings resulting from treating the rows of A as edge embeddings.

Original: Assuming that γ ̸ = 1 and it is not the case that α 1 = α 2 = 1 ⁄ 2 , the node communities are distinguishable in X .
Revised: Assuming that γ ̸ = 1 and it is not the case that α 1 = α 2 = 1 ⁄ 2 , the node communities are distinguishable in X .

Original: Proof.
Revised: Proof.

Original: Recalling Definition 4 for mean-edge node embeddings, the node embeddings ˜ X ∈ R n × m are given by the matrix product ˜ X =
Revised: Recalling Definition 4 for mean-edge node embeddings, the node embeddings ˜ X ∈ R n × m are given by the matrix product ˜ X =

Original: ˜
Revised: ˜

Original: BA , where ˜ B results from dividing each row of the incidence matrix B ∈ R n × m by its sum.
Revised: BA , where ˜ B results from dividing each row of the incidence matrix B ∈ R n × m by its sum.

Original: Since the expected graph A is (2∆) -regular by Definition 3 , we have ˜ X
Revised: Since the expected graph A is (2∆) -regular by Definition 3 , we have ˜ X

Original: =
Revised: =

Original: ( 1 / (2∆) ) BA .
Revised: ( 1 / (2∆) ) BA .

Original: For simplicity, for most of this proof, we will skip this division and take
Revised: For simplicity, for most of this proof, we will skip this division and take

Original: X = BA ; note that X are ‘sum-edge node embeddings,’ which result from summing each the edge embeddings of each node’s incident edge rather than taking the mean.
Revised: X = BA ; note that X are ‘sum-edge node embeddings,’ which result from summing each the edge embeddings of each node’s incident edge rather than taking the mean.

Original: We will consider the sum-edge node embeddings of two nodes, u ∈ U and v ∈ V .
Revised: We will consider the sum-edge node embeddings of two nodes, u ∈ U and v ∈ V .

Original: Let U ′ =
Revised: Let U ′ =

Original: U \ { u } and V ′ =
Revised: U \ { u } and V ′ =

Original: V \ { v } .
Revised: V \ { v } .

Original: We calculate entries of the embeddings corresponding to U ′ ×
Revised: We calculate entries of the embeddings corresponding to U ′ ×

Original: U ′ , V ′ × V ′ , and
Revised: U ′ , V ′ × V ′ , and

Original: U ′ × V ′ for the two time periods, and show that the two nodes’ embeddings are distinct under the assumptions on α 1 , α 2 and γ .
Revised: U ′ × V ′ for the two time periods, and show that the two nodes’ embeddings are distinct under the assumptions on α 1 , α 2 and γ .

Original: We start with the expectation of node u ’s embedding x u , which is a single row of X given by b u A , where b u is the incidence vector of node u .
Revised: We start with the expectation of node u ’s embedding x u , which is a single row of X given by b u A , where b u is the incidence vector of node u .

Original: Hence, x u is a sum of rows of A corresponding to edges which are incident on node u .
Revised: Hence, x u is a sum of rows of A corresponding to edges which are incident on node u .

Original: Consider the row vector of A corresponding to such an edge u − w between distinct nodes u and w : this vector has a nonzero entry exactly over columns of A corresponding to edges incident to this edge u − w ; in particular, by the assumption of zero time variance, the nonzero entries are 1 if the incident edges are in the same time period, otherwise γ .
Revised: Consider the row vector of A corresponding to such an edge u − w between distinct nodes u and w : this vector has a nonzero entry exactly over columns of A corresponding to edges incident to this edge u − w ; in particular, by the assumption of zero time variance, the nonzero entries are 1 if the incident edges are in the same time period, otherwise γ .

Original: These incident edges are of the form w − z or u − z for some node z .
Revised: These incident edges are of the form w − z or u − z for some node z .

Original: Since we ignore entries of the embedding vectors corresponding to columns of the latter type (due to only considering columns involving edges between U ′ , V ′ rather than U, V ), the expectation of entries of x u can be calculated by counting the contributions of paths of length 2 of the form u − w − z with u ̸ = w, z .
Revised: Since we ignore entries of the embedding vectors corresponding to columns of the latter type (due to only considering columns involving edges between U ′ , V ′ rather than U, V ), the expectation of entries of x u can be calculated by counting the contributions of paths of length 2 of the form u − w − z with u ̸ = w, z .

Original: First, consider the ( U ′ ×
Revised: First, consider the ( U ′ ×

Original: U ′ ) 1 block of x u .
Revised: U ′ ) 1 block of x u .

Original: Contributions to this block must come from paths of the form u 1 − u i 1 − u j or u 2 − u i 1 − u j with u i , u j ∈ U ′ , where the number above the dash denotes the time period of the edge.
Revised: Contributions to this block must come from paths of the form u 1 − u i 1 − u j or u 2 − u i 1 − u j with u i , u j ∈ U ′ , where the number above the dash denotes the time period of the edge.

Original: For the former possibility of u 1 − u i 1 − u j , there are α 1 ∆ such edges u 1 − u
Revised: For the former possibility of u 1 − u i 1 − u j , there are α 1 ∆ such edges u 1 − u

Original: i (i.e., intra-community edges from u in period 1), and for each node u i ∈ U ′ , there are also α 1 ∆ edges of the form u i 1 − u j to some node u j ∈ U ′ .
Revised: i (i.e., intra-community edges from u in period 1), and for each node u i ∈ U ′ , there are also α 1 ∆ edges of the form u i 1 − u j to some node u j ∈ U ′ .

Original: Hence the total number of u 1 − u i 1 − u j paths is the product α 21 ∆ 2 , and since both edges in such paths are in period 1, there is no time decay.
Revised: Hence the total number of u 1 − u i 1 − u j paths is the product α 21 ∆ 2 , and since both edges in such paths are in period 1, there is no time decay.

Original: For the latter possibility of u 2 − u i 1 − u j , there are α 2 ∆ edges u 2 −
Revised: For the latter possibility of u 2 − u i 1 − u j , there are α 2 ∆ edges u 2 −

Original: u
Revised: u

Original: i (intra-community edges from u in period 2), and through each of these edges, there are α 1 ∆ such paths u 2 − u i 1 − u j ; hence the total number of such paths is α 1 α 2 ∆ 2 , and due to time decay, their total contribution is γα 1 α 2 ∆ 2 .
Revised: i (intra-community edges from u in period 2), and through each of these edges, there are α 1 ∆ such paths u 2 − u i 1 − u j ; hence the total number of such paths is α 1 α 2 ∆ 2 , and due to time decay, their total contribution is γα 1 α 2 ∆ 2 .

Original: Therefore, the sum of the ( U ′ × U ′ ) 1 block of x u is (cid:0)
Revised: Therefore, the sum of the ( U ′ × U ′ ) 1 block of x u is (cid:0)

Original: α 

Revised: α

Original: + γα 1 α 2 (cid:1) ∆ 2 .
Revised: + γα 1 α 2 (cid:1) ∆ 2 .

Original: It follows that the sum of the ( U ′ ×
Revised: It follows that the sum of the ( U ′ ×

Original: U ′ ) 1 block of ˜ x u is ( 1 / (2∆) )
Revised: U ′ ) 1 block of ˜ x u is ( 1 / (2∆) )

Original: (cid:0)
Revised: (cid:0)

Original: α 

Revised: α

Original: + γα 1 α 2 (cid:1) ∆ 2 = (cid:0)
Revised: + γα 1 α 2 (cid:1) ∆ 2 = (cid:0)

Original: α 21 + γα 1 α 2 (cid:1) ∆2 .
Revised: α 21 + γα 1 α 2 (cid:1) ∆2 .

Original: Since there are α 1 ∆ nedges in ( U ′ ×
Revised: Since there are α 1 ∆ nedges in ( U ′ ×

Original: U ′ ) 1 , the expected value of an entry in this block is ( α 1 + γα 2 ) ( 2 / n ) .
Revised: U ′ ) 1 , the expected value of an entry in this block is ( α 1 + γα 2 ) ( 2 / n ) .

Original: Second, consider the ( V ′ × V ′ ) 1 block of x u .
Revised: Second, consider the ( V ′ × V ′ ) 1 block of x u .

Original: Contributions to this block must come from paths of the form u 1 − v i 1 − v j or u 2 − v i 1 − v j with v i , v j ∈ V ′ .
Revised: Contributions to this block must come from paths of the form u 1 − v i 1 − v j or u 2 − v i 1 − v j with v i , v j ∈ V ′ .

Original: For the former possibility of u 1 − v i 1 − v j , there are (1 − α 1 )∆ such edges u 1 − v i (i.e., inter-community edges from u in period 1), and for each node v i ∈ V ′ , there are α 1 ∆ edges of the form v i 1 − v j to some node v j ∈ V ′ (intra-community edges from v i in period 1).
Revised: For the former possibility of u 1 − v i 1 − v j , there are (1 − α 1 )∆ such edges u 1 − v i (i.e., inter-community edges from u in period 1), and for each node v i ∈ V ′ , there are α 1 ∆ edges of the form v i 1 − v j to some node v j ∈ V ′ (intra-community edges from v i in period 1).

Original: Hence the total number of u 1 − v i 1 − v j paths is the product
Revised: Hence the total number of u 1 − v i 1 − v j paths is the product

Original: α 1 (1 − α 1 )∆ 2 , and since both edges in such paths are in period 1, there is no time decay.
Revised: α 1 (1 − α 1 )∆ 2 , and since both edges in such paths are in period 1, there is no time decay.

Original: For the latter possibility of u 2 − v i 1 − v j , there are (1 − α 2 )∆ edges u 2 − v i (inter-community edges from u in period 2), and through each of these edges, there are α 1 ∆ such paths u 2 − v i 1 − v j ; hence the total number of such paths is α 1 (1 − α 2 )∆ 2 , and due to time decay, their total contribution is γα 1 (1 − α 2 )∆ 2 .
Revised: For the latter possibility of u 2 − v i 1 − v j , there are (1 − α 2 )∆ edges u 2 − v i (inter-community edges from u in period 2), and through each of these edges, there are α 1 ∆ such paths u 2 − v i 1 − v j ; hence the total number of such paths is α 1 (1 − α 2 )∆ 2 , and due to time decay, their total contribution is γα 1 (1 − α 2 )∆ 2 .

Original: Therefore, the sum of the ( V ′ × V ′ ) 1 block of x u is ( α 1 (1 − α 1 ) + γα 1 (1 − α 2 ))
Revised: Therefore, the sum of the ( V ′ × V ′ ) 1 block of x u is ( α 1 (1 − α 1 ) + γα 1 (1 − α 2 ))

Original: ∆ 2 .
Revised: ∆ 2 .

Original: As before, it follows that the expected value of an entry in the ( V ′ × V ′ ) 1 block of ˜ x u , which also contains α 1 ∆ n 4 edges/entries, is ((1 − α 1 ) + γ (1 − α 2 )) ( 2 / n ) .
Revised: As before, it follows that the expected value of an entry in the ( V ′ × V ′ ) 1 block of ˜ x u , which also contains α 1 ∆ n 4 edges/entries, is ((1 − α 1 ) + γ (1 − α 2 )) ( 2 / n ) .

Original: Third, consider the ( U ′ × V ′ ) 1 block of x u .
Revised: Third, consider the ( U ′ × V ′ ) 1 block of x u .

Original: Contributions to this block must come from paths of the form u 1 − u i 1 − v i , u 1 − v i 1 − u i , u 2 − u i 1 − v i , or u 2 − v i 1 − u i , with u i ∈
Revised: Contributions to this block must come from paths of the form u 1 − u i 1 − v i , u 1 − v i 1 − u i , u 2 − u i 1 − v i , or u 2 − v i 1 − u i , with u i ∈

Original: U ′ and v i ∈ V ′ .
Revised: U ′ and v i ∈ V ′ .

Original: As before, we sum the contributions each kind of path, yielding
Revised: As before, we sum the contributions each kind of path, yielding

Original: [Equation]
Revised: [Equation]

Original: for the sum of the ( U ′ × V ′ ) 1 block of x u .
Revised: for the sum of the ( U ′ × V ′ ) 1 block of x u .

Original: It follows that the expected value of an entry in the ( U ′ × V ′ ) 1 block of ˜ x u , which contains (1 − α 1 ) ∆ n 2 edges/entries, is (1 + γ ) ( 1 / n ) .
Revised: It follows that the expected value of an entry in the ( U ′ × V ′ ) 1 block of ˜ x u , which contains (1 − α 1 ) ∆ n 2 edges/entries, is (1 + γ ) ( 1 / n ) .

Original: Note that the expected values of entries in the ( U ′ ×
Revised: Note that the expected values of entries in the ( U ′ ×

Original: U ′ ) 2 , ( V ′ ×
Revised: U ′ ) 2 , ( V ′ ×

Original: V ′ )
Revised: V ′ )

Original: 2 , and ( U ′ × V ′ )
Revised: 2 , and ( U ′ × V ′ )

Original: 2 follow from these results by symmetry, by simply swapping α 

Revised: 2 follow from these results by symmetry, by simply swapping α

Original: and α 2 .
Revised: and α 2 .

Original: The embedding ˜ x v for the node in V also follows by symmetry.
Revised: The embedding ˜ x v for the node in V also follows by symmetry.

Original: We can finally write the block structure of the mean-edge node embeddings ˜ x u and ˜ x v for these nodes.
Revised: We can finally write the block structure of the mean-edge node embeddings ˜ x u and ˜ x v for these nodes.

Original: As with the expected adjacency matrix Proposition 4.1, the blocks are constant in value, and these values are given by 2 / n times
Revised: As with the expected adjacency matrix Proposition 4.1, the blocks are constant in value, and these values are given by 2 / n times

Original: [Equation]
Revised: [Equation]

Original: ( V ′ × V ′ ) ( V ′ × V ′ )
Revised: ( V ′ × V ′ ) ( V ′ × V ′ )

Original: ( U ′ × V ′ ) ( U ′ × V ′ )
Revised: ( U ′ × V ′ ) ( U ′ × V ′ )

Original: u
Revised: u

Original: α 1 + γαα 2 + γα(1 − α 1 ) + γ (1 − α 2 ) (1 − α 2 ) + γ (1 − α 1 ) 12 (1 + γ ) 12 (1 + γ ) v (1 − α 1 ) + γ (1 − α 2 ) (1 − α 2 ) + γ (1 − α 1 ) α 1 + γαα 2 + γα12 (1 + γ ) 12 (1 + γ )   .
Revised: α 1 + γαα 2 + γα(1 − α 1 ) + γ (1 − α 2 ) (1 − α 2 ) + γ (1 − α 1 ) 12 (1 + γ ) 12 (1 + γ ) v (1 − α 1 ) + γ (1 − α 2 ) (1 − α 2 ) + γ (1 − α 1 ) α 1 + γαα 2 + γα12 (1 + γ ) 12 (1 + γ )   .

Original: As desired, the embeddings for nodes u and v are distinct as long as both
Revised: As desired, the embeddings for nodes u and v are distinct as long as both

Original: γ ̸ = 1 and it is not the case
Revised: γ ̸ = 1 and it is not the case

Original: that α 1 = α
Revised: that α 1 = α

Original: 2 = 1 ⁄ 2 .
Revised: 2 = 1 ⁄ 2 .

Original: Note that, as discussed in Section 4, the embeddings are not distinct if γ = 1 and α 1 + α 2 = 1 .
Revised: Note that, as discussed in Section 4, the embeddings are not distinct if γ = 1 and α 1 + α 2 = 1 .

Original: ■
Revised: ■

Original: A. D ATASETS
Revised: A. D ATASETS

Original: We use a collection of 5 temporal network datasets which are hosted on the repositories of Rossi & Ahmed (2015) and Leskovec & Sosiˇc (2016).
Revised: We use a collection of 5 temporal network datasets which are hosted on the repositories of Rossi & Ahmed (2015) and Leskovec & Sosiˇc (2016).

Original: The statistics for these networks are provided in
Revised: The statistics for these networks are provided in

Original: Table 2.
Revised: Table 2.

Original: BITCOINALPHA and BITCOINOTC (Kumar et al., 2016; 2018) are who-trust-whom networks of people who trade Bitcoin on the Bitcoin Alpha and Bitcoin OTC platforms.
Revised: (Kumar et al., 2016; 2018) are who-trust of people who trade Bitcoin on the Bitcoin Alpha and Bitcoin OTC platforms.

Original: Each temporal edge involves one user rating another user’s trustworthiness on a scale of -10 to +10 at some point in time; we binarize these ratings to positive and nonpositive to produce binary edge labels for classification.
Revised: Each temporal edge involves one user rating another user’s trustworthiness on a scale of -10 to +10 at some point in time; we binarize these ratings to positive and nonpositive to produce binary edge labels for classification.

Original: ESCORTS is a bipartite network of sexual escorts and buyers; the edges are timed instances of a buyer ratingan escorts from -1 to +1, which we again binarize as before.
Revised: ESCORTS is a bipartite network of sexual escorts and buyers; the edges are timed instances of a buyer rating escorts from -1 to +1, which we again binarize as before.

Original: WIKIELECT (Leskovec et al., 2010) is a network of Wikipedia users voting in administrator elections for or against other users to be promoted to administrator.
Revised: WIKIELECT (Leskovec et al., 2010) is a network of Wikipedia users voting in administrator elections for or against other users to be promoted to administrator.

Original: Finally, EPINIONS (Leskovec et al., 2010) is a who-trusts-whom network for the now-closed consumer review website Epinions.com.
Revised: Finally, EPINIONS (Leskovec et al., 2010) is a who-trusts-whom network for the now-closed consumer review website Epinions.com.

Original: A. H YPERPARAMETER S ENSITIVITY
Revised: A. H YPERPARAMETER S ENSITIVITY

Original: Time scale hyperparameter As described in the main body of this paper, for our TDLG method, we set the hyperparameter σ t as ratio of the standard deviation of the edges’ times.
Revised: Time scale hyperparameter As described in the main body of this paper, for our TDLG method, we set the hyperparameter σ t as ratio of the standard deviation of the edges’ times.

Original: Letting this standard deviation be σ T , we use σ t = σ ′ t · σ T .
Revised: Letting this standard deviation be σ T , we use σ t = σ ′ t · σ T .

Original: For all results in the main body, we use σ′ t = 10 − 1 ,  which  we choose by informal tuning.
Revised: In initial experiments on small graphs, we tried some natural values for σ t : 10 − 2 , 10 − 1 , and 10 0 . We found 10 − 1 to appear to work well in general, and all subsequent experiments on the 5 datasets of the paper, which appear in the paper’s main body, use this value. Here, we now conduct experiments to evaluate this setting.

Original: Figure 6 shows how test performance is impacted when varying σ ′ t ∈ [10 − 3 , 10 +1 ] .
Revised: Figureshows how test performance is impacted when varying σ ′ t ∈ [10 − 3 , 10 +1 ] .

Original: Across the tested datasets (B ITCOIN A LPHA , B ITCOIN OTC, E SCORTS , and W IKI E LECT ), we find that the highest test performance indeed occurs near σ ′ t = 10 − 1 , though there is some variance in the optimal value across datasets.
Revised: Across the tested datasets (B ITCOIN A LPHA , B ITCOIN OTC, E SCORTS , and W IKI E LECT ), we find that the highest test performance indeed occurs near σ ′ t = 10 − 1 , though there is some variance in the optimal value across datasets.

Original: Performance  alsoseems fairly robust with respect to this hyperparameter: for all tested datasets except E SCORTS , performance does not fall more than 5% relative to the best value of σ ′ t as it is varied across the tested range.
Revised: Performance is also fairly robust with respect to this hyperparameter: for all tested datasets except E SCORTS , performance does not fall more than 5% relative to the best value of σ ′ t as it is varied across the tested range.

Original: TDLG normalization It is common to somehow normalize the adjacency matrix
Revised: TDLG normalization It is common to somehow normalize the adjacency matrix

Original: A prior to use in downstream tasks.
Revised: A prior to use in downstream tasks.

Original: We forego this procedure for all results in the main body of this paper, but here we test the impact of two kinds of normalization on the edge classification test.
Revised: We forego this procedure for all results in the main body of this paper, but here we test the impact of two kinds of normalization on the edge classification test.

Original: Let D ∈ R m × m +  denote the diagonal degree matrix, the diagonal entries of which are the row sums of A (which are equivalent to the column sums).
Revised: Let D ∈ R m × m + [Figure] denote the diagonal degree matrix, the diagonal entries of which are the row sums of A (which are equivalent to the column sums).

Original: A common form of normalization, used in the spectral clustering[Figure] algorithm of Shi & Malik (2000) and many other works, is to set the normalized matrix
Revised: A common form of normalization, used in the spectral clustering algorithm of Shi & Malik (2000) and many other works, is to set the normalized matrix

Original: [Equation]
Revised: [Equation]

Original: this normalization has useful spectral properties, bounding eigenvalues within [ − 1 , +1] .
Revised: this normalization has useful spectral properties, bounding eigenvalues within [ − 1 , +1] .

Original: We also test a second normalization procedure which aims to roughly preserve the total amount of edge weight associated with each edge.
Revised: We also test a second normalization procedure which aims to roughly preserve the total amount of edge weight associated with each edge.

Original: Since we assume that the original temporal graph is unweighted, meaning each edge weight is 1 , we seek for each row/column sum of the TDLG adjacency matrix to also be approximately 1 .
Revised: Since we assume that the original temporal graph is unweighted, meaning each edge weight is 1 , we seek for each row/column sum of the TDLG adjacency matrix to also be approximately 1 .

Original: In particular, we set
Revised: In particular, we set

Original: [Equation]
Revised: [Equation]

Original: that is, we return the mean of the two matrices that result from dividing the rows/columns of A by the row/column sums.
Revised: that is, we return the mean of the two matrices that result from dividing the rows/columns of A by the row/column sums.

Original: Note that this normalization preserves symmetry of the adjacency matrix.
Revised: Note that this normalization preserves symmetry of the adjacency matrix.

Original: We observe that this second normalization scheme guarantees rough preservation of edge weight as follows.
Revised: We observe that this second normalization scheme guarantees rough preservation of edge weight as follows.

Original: Because each row of D − 1 A sums to 1 , each row of ˜
Revised: Because each row of D − 1 A sums to 1 , each row of ˜

Original: A sums to at least 1 / 2 .
Revised: A sums to at least 1 / 2 .

Original: Further, since there are m rows, the total sum of all entries in D − 1 A is m ; similarly, AD − 1 sums to m as well, so ˜
Revised: Further, since there are m rows, the total sum of all entries in D − 1 A is m ; similarly, AD − 1 sums to m as well, so ˜

Original: A sums to m .
Revised: A sums to m .

Original: Therefore, each edge, which in the original temporal graph was given exactly fraction/ m of the total edge weight, is assigned at least fraction 1 / 2 m of the total weight in the normalized
Revised: Therefore, each edge, which in the original temporal graph was given exactly fraction/ m of the total edge weight, is assigned at least fraction 1 / 2 m of the total weight in the normalized

Original: TDLG.
Revised: TDLG.

Original: As shown in Figure 6 (right), across the tested datasets, we find that both normalization schemes (’Spectral’ and ’Edge,’ respectively) generally have a negative impact on performance, though the drop in performance is never more than around 6%.
Revised: As shown in Figure 6 (right), across the tested datasets, we find that both normalization schemes (’Spectral’ and ’Edge,’ respectively) generally have a negative impact on performance, though the drop in performance is never more than around 6%.

Original: These results suggest that the simplest approach of skipping normalization is generally also the most performant. 
Revised: These results suggest that the simplest approach of skipping normalization is generally also the most performant. A. S CALABILITY

Revised: We now discuss the scalability of our approach.

Original: A. F ULL E XPERIMENTAL R ESULTS
Revised: Scaling in theory The number of edges in a line graph L ( G ) of a graph G scales with the sum of squared degrees of the nodes in G . Hence the number of nonzero entries in the adjacency matrix A of the TDLG scales with the same quantity (cid:80) i d i ; this determines both the memory requirement of storing the TDLG, as well as the asymptotic runtime of methods based on multiplying by A , including training a logistic regression classifier and truncated SVD (i.e., calculating the top eigenvectors), since the time complexity of matrix multiplication by a sparse matrix scales linearly with the number of nonzero entries. Scaling in practice In theory, this could be very expensive, but real-world degree distributions generally seem to make this practical, including up to the fairly large E PINIONS dataset of over 800K edges. We are able to run our method on E PINIONS on a standard 16GB laptop, which is not true of many of the comparison methods.

Revised: In Figure 7, we plot the number of nonzero entries in the embeddings generated by each method for each dataset, when using the hyperparameter settings described in Section 6.1.

Revised: Generally, the numbers for our TDLG method are comparable to those for

Revised: TIMERS and deep methods.

Revised: CTDNE generally has significantly lighter embeddings, since it stores a single dense vector per node.

Revised: Scaling strategies To scale our method to much larger datasets of millions of edges, there are several possible strategies, and we now discuss a few of these.

Revised: To be clear, we do not employ any of these strategies in this work since they are not necessary for the datasets we consider, but this discussion may be of interest for future extensions.

Revised: First, some edges will share an endpoint but be very far apart in time.

Revised: Hence the entry corresponding to such pairs of edges will be nonzero but very small; these entries could be ignored.

Revised: Second, and perhaps most importantly, it is not necessary to hold the entire TDLG matrix in memory at once: given the closed-form formula, it is possible to construct some rows and supply them to a stochastic solver on-the-fly.

Revised: Third, as we discuss in the conclusion, since each column can be thought of as a feature, it may be possible to sample only certain columns, e.g., by leverage score sampling, without much drop in performance.

Revised: Third, and most interestingly, is increasing scalability via reduction of the number of features.

Revised: The TDLG method returns m features per temporal edge based on proximities to all other temporal edges.

Revised: Perhaps it is possible to efficiently reduce the feature space by only considering proximities to certain edges, e.g., edges with higher effective resistances.

Revised: Along the same lines, it may also be interesting to explore ways of reducing the feature space by efficiently forming a low-rank representation of the TDLG adjacency matrix, for example, via a stochastic method.

Revised: [Figure]

Revised: A. F ULL E XPERIMENTAL R ESULTS

Original: As stated in Section 6.
Revised: As stated in Section 6.

Original: [Figure]
Revised: [Figure]

Original: we include plots for link prediction performance for each dataset, for both the interpolative and extrapolative settings, in Figure 7.
Revised: we include plots for link prediction performance for each dataset, for both the interpolative and extrapolative settings, in Figure 8.

Original: Tables 3,
Revised: Tables 3,

Original: [Figure]
Revised: [Figure]

Original: and 5 report the full performance results for edge classification, temporal link outlier detection (interpolative)
Revised: and 5 report the full performance results for edge classification, temporal link outlier detection (interpolative)

Original: [Figure]
Revised: [Figure]

Original: and temporal link prediction (extrapolative), respectively, which were deferred in Section
Revised: and temporal link prediction (extrapolative), respectively, which were deferred in Section

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]