Original: Generalized Laplacian Eigenmaps
Revised: Generalized Laplacian Eigenmaps

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Graph contrastive learning attracts/disperses node representations for similar/dissimilar node pairs under some notion of similarity.
Revised: Graph contrastive learning attracts/disperses node representations for similar/dissimilar node pairs under some notion of similarity.

Original: It may be combined with alow-dimensional embedding of nodes to preserve intrinsic and structural propertiesof a graph.
Revised: It may be combined with a low-dimensional embedding of nodes to preserve intrinsic and structural properties of a graph.

Original: Some recent graph contrastive methods combine traditional graphembedding and negative sampling into one framework, which minimizes the tracedifference between the within-class scatter matrix encapsulating the graph connectivity and the total scatter matrix encapsulating negative sampling.
Revised: COLES, a recent graph contrastive method combines traditional graph embedding and negative sampling into one framework. COLES in fact minimizes the trace difference between the within-class scatter matrix encapsulating the graph connectivity and the total scatter matrix encapsulating negative sampling.

Original: In this paper,we propose a more essential framework for graph embedding, called Generalized Laplacian EigeNmaps (GLEN), to learn graph representation by maximizing therank difference between the total scatter matrix and the within-class scatter matrix,resulting in the minimum class separation guarantee.
Revised: In this paper, we propose a more essential framework for graph embedding, called Generalized Laplacian EigeNmaps (GLEN), which learns a graph representation by maximizing the rank difference between the total scatter matrix and the within-class scatter matrix, resulting in the minimum class separation guarantee.

Original: However, the rank differenceminimization is an NP-hard problem.
Revised: However, the rank difference minimization is an NP-hard problem.

Original: Herein, we replace the trace differencethat corresponds to the difference of nuclear norms by the difference of logdetexpressions, which we argue is a more accurate surrogate for the NP-hard rankdifference than the trace difference.
Revised: Thus, we replace the trace difference that corresponds to the difference of nuclear norms by the difference of LogDet expressions, which we argue is a more accurate surrogate for the NP-hard rank difference than the trace difference.

Original: We show that the logdet loss can be interpretedas an upper bound of the Jensen-Bregman LogDet Divergence (JBLD), and the Affine-invariant Riemannian metric (AIRM) while enjoying a lesser computationalburden.
Revised: We show that the LogDet loss can be interpreted as an upper bound of the Jensen-Bregman LogDet Divergence (JBLD), and the Affine-invariant Riemannian metric (AIRM) while enjoying a lesser computational burden.

Original: We show on popular benchmarks/backbones that GLEN offers favourableaccuracy/scalability compared to state-of-the-art baselines.
Revised: We show on popular benchmarks/backbones that GLEN offers favourable accuracy/scalability compared to state-of-the-art baselines.

Original:  Introduction
Revised:  Introduction

Original: Laplacian Eigenmaps [4] and IsoMap [30] arepopular graph embedding methods that minimise thedimensionality of data which exists on a low-dimensional manifold.
Revised: Laplacian Eigenmaps [3] and IsoMap [32] are graph embedding methods that reduce the dimensionality of data by assuming the data exists on a low-dimensional manifold.

Original: The objective function in suchmodels encourages node embeddings to lie near each other in the embedding space if they are closein the original space.
Revised: The objective function in such models encourages node embeddings to lie near each other in the embedding space if nodes are close to each other in the original space.

Original: However, classical methods focus solely on modeling the related node pairs andignore modelingof unrelated node pairs.
Revised: While the classical methods capture the related node pairs, they neglect modeling unrelated node pairs.

Original: In contrast, modern graph embedding models  areoften unified under the (sampled) Noise Contrastive Estimation (NCE) framework [24, 20] and extended to network embedding [29, 10, 37] and Graph Contrastive Learning (GCL) [31, 46, 11, 47, 45].
Revised: In contrast, modern graph embedding models such as [31, 9, 39] and Graph Contrastive Learning (GCL) [33, 52, 10, 53, 51] are unified under the (Sampled) Noise Contrastive Estimation framework, called (Sampled)NCE [26, 22].

Original: Most  GCL methods do not incorporate graphinformation into the loss but follow the setting from computer vision, i.e ., they assume that randomlydrawn pairs should be dissimilar and the original sample and its augmentations should be similar [33].
Revised: Most of GCL methods do not incorporate the graph information into the loss but follow the setting from computer vision, i.e ., they assume that randomly drawn pairs should be dissimilar, whereas the original sample and its augmentations should be similar [35].

Original: In contrast, Contrastive Laplacian Eigenmaps [45] propose a framework to learn (graph) neuralnetworks with Laplacian eigenmaps utilizing  graph information within a contrastive loss.
Revised: In contrast, COntrastive Laplacian EigenmapS (COLES) [51] is a framework which combines a (graph) neural network with Laplacian eigenmaps utilizing the graph Laplacian matrix within a contrastive loss.

Original: However,their method minimizes the trace difference of Laplacians which follows the NCE framework.
Revised: Based on the NCE framework, COLES minimizes the trace difference of Laplacians.

Original: In this paper, we analyze the relation among within-class, between-class and total scatter matricesunder the rank inequality, and prove that, under a simple assumption, the distance between anydissimilar (negative) samples would be greater/equal than the inter-class distance between theircorresponding class centres.
Revised: In this paper, we analyze the relation among within-class, between-class and total scatter matrices under the rank inequality, and prove that, under a simple assumption, the distance between any dissimilar (negative) samples would be greater/equal than the inter-class distance between their corresponding class centers.

Original: Based on this condition, we derive GLEN, a reformulation of graphembedding into a rank difference problem, which is a more essential framework than other graph embedding  frameworks.
Revised: Based on such a condition, we derive GLEN, a reformulation of graph embedding into a rank difference problem, which is a more general framework than other graph embedding frameworks, i.e ., under specific relaxations of the rank difference problem, we can recover different frameworks.

Original: Furthermore, we demonstrate how to optimize the rank difference problemwith a difference of logdet expressions, a differentiable function which is easy to use within (graph)neural networks.
Revised: To that end, we demonstrate how to optimize the rank difference problem with a difference of LogDet expressions, a differentiable relaxation suitable for use with (graph) neural networks.

Revised: We consider other surrogates of the rank difference problem, based on the Nuclear norm, γ -nuclear norm, Schatten norm, and the Geman norm.

Original: Moreover, we provide theoretical considerations regarding the low-rank optimizationand connection to the Riemannian manifold in order to interpret our approach.
Revised: Moreover, we provide theoretical considerations regarding the low-rank optimization and connection to the Riemannian manifold in order to interpret our approach.

Original: In summary, our contributions are threefold:
Revised: In summary, our contributions are threefold:

Original: i. We propose a rank-based condition connecting within-class, between-class and total scatter matrices under which we can guarantee the worst margin on the  class separation.
Revised: i. We propose a rank-based condition connecting within-class, between-class and total scatter matrices under which we provide the minimum class separation guarantee.

Original: We propose aloss function, Generalized Laplacian EigenNaps (GLEN), that realizes this condition.ii.
Revised: We propose a loss function, Generalized Laplacian EigenNaps (GLEN), that realizes this condition. ii.

Original: However, as the rank difference problem is NP-hard, we turn it into a difference of logdet expressions to learn node embeddings, as opposed to the trace difference (an upper bound of the difference of logdet terms and a poorer surrogate) emerging in traditional graph embeddingmodels.
Revised: As the rank difference problem is NP-hard, we consider a difference of LogDet surrogate to learn node embeddings, as opposed to the trace difference (an upper bound of the difference of LogDet terms) used by other graph embedding models. We also consider other surrogates.

Original: iii.
Revised: iii.

Original: We study the distance between Symmetric Positive Definite (SPD) matrices realized by the logdet-based implementation of GLEN which is lower-bounded by Jesen-Bregman LogDet Divergence (JBLD) and the Affine-invariant Riemmanian Metric (AIRM).
Revised: We study the distance between Symmetric Positive Definite (SPD) matrices realized by the LogDet-based implementation of GLEN which is lower-bounded by Jesen-Bregman LogDet Divergence (JBLD) and the Affine-invariant Riemmanian Metric (AIRM).

Original: We also explain howGLEN connects to other graph embedding methods.
Revised: We also explain how GLEN connects to other graph embedding methods.

Original:  Related Works
Revised:  Related Works

Original: Graph Embeddings.
Revised: Graph Embeddings.

Original: By assuming that the data lies on a low-dimensional manifold, graph embedding methods such as Laplacian Eigenmaps [4] and IsoMap [30] optimize low-dimensionaldata embeddings.
Revised: By assuming that the data lies on a low-dimensional manifold, graph embedding methods such as Laplacian Eigenmaps [3] and IsoMap [32] optimize low-dimensional data embeddings.

Original: These methods [6] construct a similarity graph by measuring the similarity ofhigh-dimensional feature vectors and embed the nodes into a low-dimensional space.
Revised: These methods [5] construct a similarity graph by measuring the similarity of high-dimensional feature vectors and embed the nodes into a low-dimensional space.

Original: However, thesimilarity graph does not contain the structural graph information.

Original: DeepWalk [27] uses truncated random walks to explore the graph structure and the skip-gram modelfor word embedding to determine the embedding vectors of nodes to address the above shortcomings.
Revised: DeepWalk [28] uses truncated random walks to explore the graph structure, and the skip-gram model for word embedding to determine the embedding vectors of nodes.

Original: By setting the walk length to one and using negative sampling [23], LINE [29] explores a similaridea with an explicit objective function.
Revised: By setting the walk length to one and using negative sampling [25], LINE [31] explores a similar idea with an explicit objective function while REFINE [48] imposes additional orthogonality constraints which deem REFINE extremely fast.

Original: To capture different graph neighborhoods, Node2Vec [9]interpolates between breadth- and depth-first sampling.
Revised: Node2Vec [8] interpolates between breadth- and depth-first sampling.

Original: Contrastive Laplacian Eigenmaps (COLES)unifies the traditional graph embedding and negative sampling by introducing a  contrastive term thatcaptures the graph structure, whereas the connectivity for negative sampling is arranged randomly.
Revised: COLES [51] unifies traditional graph embedding and negative sampling by introducing a positive contrastive term that captures the graph structure, and a negative contrastive random sampling.

Original: COLES solves the trace difference problem akin to traditional graph embedding models [36].
Revised: COLES solves the trace difference problem akin to traditional graph embedding models [38].

Original: In this paper, we propose a more essential lossfunction for graph embedding. COLES is a specialcase of GLEN, i.e ., COLES uses the trace difference (difference of nuclear norms) which is a specificrelaxation of generic GLEN that encapsulates a variety of contrastive properties.
Revised: In this paper, we propose a more general loss for graph embedding, i.e ., COLES solves the trace difference (Nuclear norms difference) relaxation of GLEN.

Original: Graph embedding techniques [36] provide a general framework for dimensionality reduction such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Locality Preserving Projections (LPP) [12].
Revised: Graph embedding techniques [38] provide a general framework for dimensionality reduction such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Locality Preserving Projections (LPP) [11].

Original: All methods within this category can be considered as solving the same problem under varying assumptions, i.e ., maximising the intra- and inter-class separation by optimizingthe trace difference, also used in metric learning [19].
Revised: All methods within this category can be considered as solving the same problem under varying assumptions, i.e ., maximising the intra- and inter-class separation by optimizing the trace difference, also used in metric learning [20].

Original: However, such a family of objective functions is not motivated by the guarantee on the minimummargin between feature vectors from different categories.
Revised: However, such a family of objective functions is not motivated by the guarantee on the minimum class separation between feature vectors from different categories.

Original: GLEN, in its purest NP-hard form, providesthe minimum margin guarantee and can be realised by several formulations depending on  trade-offs.
Revised: GLEN, in its purest NP-hard form, provides the minimum class separation guarantee and can be realised by several formulations depending on chosen trade-offs.

Original: Unsupervised Representation Learning for Graph Neural Networks (GNN).
Revised: Unsupervised Representation Learning for Graph Neural Networks (GNN).

Original: Broadly, unsupervised GNN training can be reconstruction based, contrastive based or diffusion based.
Revised: Unsupervised GNN training can be reconstruction-, contrastive- or diffusion-based.

Original: To traina graph encoder in an unsupervised manner, GCN [17] minimizes a reconstruction error whichonly considers the similarity matrix and ignores the dissimilarity information.
Revised: To train a graph encoder in an unsupervised manner, GCN [16] minimizes a reconstruction error which only considers the similarity matrix and ignores the dissimilarity information.

Original: At various scales ofthe graph, contrastive methods determine the positive and negative sets.
Revised: At various scales of the graph, contrastive methods determine the positive and negative sets.

Original: For example, local-local CL and global-local CL strategies are highly popular.
Revised: For example, local-local CL and global-local CL strategies are highly popular.

Original: GraphSAGE [10] incorporates objectivesinspired by DeepWalk [27], i.e ., the contrastive loss  encourages neighbor nodes to have similarrepresentations, while preserving dissimilarity between representations of disparate nodes.
Revised: GraphSAGE [9], inspired by DeepWalk [28], uses the contrastive loss which encourages neighbor nodes to have similar representations, while preserving dissimilarity between representations of disparate nodes.

Original: DGI [31],inspired by Deep InfoMax (DIM) [13], proposes an objective with global-local sampling strategy,which maximizes the Mutual Information (MI) between global and local graph embeddings.
Revised: DGI [33], inspired by Deep InfoMax (DIM) [12], uses an objective with global-local sampling strategy to maximize the Mutual Information (MI) between global and local graph embeddings.

Original: Incontrast, Augmented Multiscale Deep InfoMax (AMDIM) [2] maximizes MI between multiple views [Figure] of data.
Revised: Augmented Multiscale Deep InfoMax (AMDIM) [2] maximizes MI between multiple data views.

Original: MVRLG [11] contrasts encodings from first-order neighbors and a graph diffusion.
Revised: MVRLG [10] contrasts encodings from first-order neighbors and a graph diffusion.

Original: MVRLGuses GCNs to learn node embeddings for different views.
Revised: Fisher-Bures Adversary GCN [30] treats the graph as generated w.r.t. some observation noise. COSTA [46] constructs the views by injecting features with the random noise. [Figure]

Original: However, such contrastive approaches often require thousands of epochs to converge and performwell.
Revised: However, such contrastive approaches often require thousands of epochs to converge and perform well.

Original: In addition, many contrastive losses have an exponential increase in memory overhead w.r.t.
Revised: In addition, many contrastive losses have an exponential increase in memory overhead w.r.t.

Original: thenumber of nodes.
Revised: the number of nodes.

Original: In contrast, our method does not explicitly use the local-local setting but the totalscatter matrix, and thus saves computational and storage cost.
Revised: In contrast, our method does not explicitly use the local-local setting but the total scatter matrix, and thus saves computational and storage cost.

Original: Moreover, some linear GNNs, e.g ., SGC [35] and S 2 GC [44], capture the neighborhood and increasingly larger neighborhoods of each node by diffusion, respectively.
Revised: Linear GNNs, i.e ., SGC [37] and S 2 GC [49], capture the neighborhood and increasingly larger neighborhoods of each node due to the diffusion, respectively.

Original: SGC and S 2 GC have noprojection layer, which results in embeddings of size  equal to the input dimension.
Revised: SGC and S 2 GC have no projection layer, thus the size of embeddings is equal to the input dimension.

Original: Our methodis able to learn a projection layer in an unsupervised manner with a linear function or MultiLayer Perceptron (MLP) for these diffusion-based GNNs.
Revised: GLEN can learn a projection layer in an unsupervised manner with a linear function or Multi-Layer Perceptron (MLP) applied to linear GNNs or any other GNN models [17, 30, 45], etc .

Original:  Preliminaries
Revised:  Preliminaries

Original: Notations.
Revised: Notations.

Original: Let G =( V, E ) be a simple, connected and undirected graph with n = | V | nodes andm = | E | edges.
Revised: Let G =( V, E ) be a simple, connected and undirected graph with n = | V | nodes and m = | E | edges.

Original: Let i ∈ { 1 , · · · , n } be the node index of G , and d j be the degree of node j of G .
Revised: Let i ∈ { 1 , · · · , n } be the node index of G , and d j be the degree of node j of G .

Original: Let W be the adjacency matrix, and D be the diagonal matrix containing degrees of nodes.
Revised: Let W be the adjacency matrix, and D be the diagonal matrix containing degrees of nodes.

Original: Let
Revised: Let

Original: X ∈ R n × d denote the node feature matrix where each node v is associated with a feature vectorx v ∈ R d .
Revised: X ∈ R n × d denote the node feature matrix where each node v is associated with a feature vector x v ∈ R d .

Original: Let the normalized graph Laplacian matrix be defined as L = I − ˜ W ∈ S n + , a symmetricpositive semi-definite matrix and ˜ W
Revised: Let the normalized graph Laplacian matrix be defined as L = I − ˜ W ∈ S n + , a symmetric positive semi-definite matrix and ˜ W

Original: = D − 1 / 2 WD − 1 / 2 .
Revised: = D − 1 / 2 WD − 1 / 2 .

Revised: S m + ( + ) is a set of symmetric positive (semi- )definite matrices.

Original: Let Z = f Θ ( X ) ∈ R n × m be a generalizednode embedding, i.e .,
Revised: Let Z = f Θ ( X ) ∈ R n × m be a generalized node embedding, i.e .,

Original: X could be identity matrix ( e.g ., no attributes for nodes), f Θ ( X ) could be GNN or a linear function with parameters Θ .
Revised: X could be identity matrix ( e.g ., no node attributes), f Θ ( X ) could be GNN or a linear function with parameters Θ .

Original: Finally, scalars and vectors are denoted by lowercaseregular and bold fonts, respectively. Matrices are denoted by uppercase bold fonts.
Revised: Scalars/vectors/matrices are denoted by lowercase regular/lowercase bold/uppercase bold fonts.

Original: 3.1 Scatter Matrices
Revised: 3.1 Scatter Matrices

Original: Let us write the standard definitions of scatter matrices, including the total scatter  S t  ,  within-class  S w  , and between-class  S b matrices:
Revised: Below are given standard definitions of scatter matrices, including the total scatter matrix S t ∈ S m + ( + ) , the within-class matrix S w ∈ S m + ( + ) , and between-class matrix S b ∈ S m + ( + ) :

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: where˜ W
Revised: where

Original: [Equation]
Revised: [Equation]

Original: = [Equation] Let e be an n -dimensional vector with all coefficients equal one, I be an identity matrix, S t be thetotal scatter (covariance) matrix, and ¯ z  be the mean of all samples.
Revised: Let e be an n -dimensional vector with all coefficients equal one, I be an identity matrix, S t be the total scatter (covariance) matrix, and ¯ z ∈ R m be the mean of all samples.

Original: Let µ y i  be the class center of thei-th sample and µ c  be the c -th class center.
Revised: Let µ y i ∈ R m be the class center of the i -th sample and µ c ∈ R m be the c -th class center.

Original: Let the total number of categories be given by C , whereas n c be the number of samples for the c -th category.
Revised: Let the total number of categories be given by C , whereas n c be the number of samples for the c -th category.

Original: Let e c be an n -dimensional vector where a givencoefficient is equal one if itscorresponding node is of class c , otherwise it is equal zero.
Revised: Let e c ∈ R n be a vector where a given coefficient indexed by node is equal one if its node is of class c , otherwise it is equal zero.

Original: We note that both S t and S w can take a form akin to Laplacian eigenmaps such that ˜ W t and ˜ W ware the corresponding normalized adjacent matrices.
Revised: We note that both S t and S w can take a form akin to Laplacian eigenmaps such that ˜ W t and ˜ W w are the corresponding normalized adjacent matrices.

Original: Let us also define graph Laplacian matrices
Revised: Let us also define graph Laplacian matrices

Original: L t = I − ˜ W t  and L w =
Revised: L t = I − ˜ W t ∈ S n + and L w =

Original: I − ˜ W w  which will be used in the sequel.
Revised: I − ˜ W w ∈ S n + which will be used in the sequel.

Original: Importantly, let us assume that agraph Laplacian matrix L containing graph links could be seen as a noisy version of L w in which allnodes of a given class c connect under  weight equal 1 /c c .
Revised: Importantly, let us assume that a graph Laplacian matrix L containing graph links could be seen as a noisy version of L w in which all nodes of a given class c connect under the weight equal 1 /n c .

Original: Observe that S t =
Revised: Observe that S t =

Original: S w + S b .
Revised: S w + S b .

Original: Thus, Rank( S t ) ≤ Rank( S w ) + Rank( S b ) due to the rank inequality.
Revised: Thus, Rank( S t ) ≤ Rank( S w ) + Rank( S b ) due to the rank inequality.

Original: Here, wewant to highlight the condition underpinning the subsequent motivation:
Revised: Below we highlight the condition underpinning the subsequent motivation:

Original: Condition 1.
Revised: Condition 1.

Original: Rank( S t ) =
Revised: Rank( S t ) =

Original: Rank( S w ) + Rank( S b ) .
Revised: Rank( S w ) + Rank( S b ) .

Original: 3.2 Motivation
Revised: 3.2 Motivation

Original: Figure 1 shows  threepossible optimal solutions for Condition 1.
Revised: Figure 1 shows some three optimal solutions for Condition 1.

Original: The rank of between-class scattermatrix S b for the whole data set is at most C − 1 (where C is the number of classes).
Revised: The rank of between-class scatter matrix S b for the whole dataset is at most C − 1 (where C is the number of classes).

Original: Since
Revised: Since

Original: Rank( AB ) ≤ min(Rank( A ) , Rank( B )) , we have  Rank( S − 1 w S b ) ≤ Rank( S b ) ≤ C − 1 .
Revised: Rank( AB ) ≤ min(Rank( A ) , Rank( B )) , we have † Rank( S − 1 w S b ) ≤ Rank( S b ) ≤ C − 1 .

Original: Rankequals the number of non-zero eigenvalues of a matrix which means that S − 1 w S b has at most C − 1non-zero eigenvalues (and corresponding eigenvectors).
Revised: The rank is the number of non-zero eigenvalues of a matrix so S − 1 w S b has at most C − 1 non-zero eigenvalues.

Original: Condition 1 implies that Rank( S − 1 w S b ) = 0which leads us to the minimum class separation guarantee under that condition.
Revised: Condition 1 implies that Rank( S − 1 w S b ) = 0 results in the minimum class separation guarantee under that condition.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let the feature dimension be larger than the class number ( i.e ., m > C) and Condition 1hold.
Revised: Let the feature dimension be larger than the class number ( i.e ., m > C ) and Conditionhold.

Original: Then, the minimum class separation is equal to the  class centre distance.
Revised: Then, the minimum class separation is equal to the distance between class centers.

Original: In other words, thedistance between any two vectors z i and z j with labels y iand y j from different classes, i.e ., y i ̸
Revised: In other words, the distance between any two vectors z i and z j with labels y i ̸

Original: = y j ,is greater/equal the distance between class centers µ y i and µ y j :
Revised: = y j is greater/equal the distance between class centers µ y i and µ y j :

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: As S w is the orthogonal complement of S b , i.e ., S − 1 w S b = 0 , S w + S b = U Σ U ⊤ , S w =
Revised: As S w is the orthogonal complement of S b , i.e ., S − 1 w S b = 0 , S w + S b = U Σ U ⊤ , S w =

Original: U 1: k Σ 1: k U ⊤ 1: k and S b =
Revised: U 1: k Σ 1: k U ⊤ 1: k and S b =

Original: U k
Revised: U k

Original: +1: m Σ k
Revised: +1: m Σ k

Original: +1: m U ⊤ k
Revised: +1: m U ⊤ k

Original: +1: m where 1 ≤ k < m .
Revised: +1: m where 1 ≤ k < m .

Original: Let z i = µ y i +
Revised: Let z i = µ y i +

Original: U ⊤ ϵ
Revised: U ⊤ ϵ

Original: i where140 ϵ i is the representation under the basis U and ϵ ( k +1: m ) ,i = [-
-] because only top k components 1: krepresent S w .
Revised: i where ϵ i is the representation under the basis U and ϵ ( k +1: m ) ,i = [-
-]0 because only top k components 1: k represent S w .

Original: Thus orthogonal projection U k +1: m fulfills ∥
Revised: Thus, the orthogonal projection U k +1: m fulfills ∥

Original: U k +1: m ( z i − z j ) ∥ 2 ≤ ∥
Revised: U k +1: m ( z i − z j ) ∥ 2 ≤ ∥

Original: z
Revised: z

Original: i − z j ∥ 2 .
Revised: i − z j ∥ 2 .

Original: Moreover, U k +1: m ( z i − µ y i )
Revised: Moreover, U k +1: m ( z i − µ y i )

Original: =
Revised: =

Original: U k
Revised: U k

Original: +1: m ( U ⊤ ϵ i )
Revised: +1: m ( U ⊤ ϵ i )

Original: = 0 .
Revised: = 0 .

Original: That is, all { z i : y i =
Revised: That is, all { z i : y i =

Original: c } are projected ontothe mean µ c 
Revised: c } are projected onto the mean µ c .

Original: Thus, the inequality in Eq.
Revised: Thus, the inequality in Eq.

Original: 2 holds.
Revised: 2 holds.

Original: Theorem 2 is attractive as it guarantees the worst inter-class distance (no known graph embeddingmethods that maximize inter-class distances and minimize intra-class distances provide such guarantees).
Revised: Theorem 1 guarantees the worst inter-class distance § .

Original: Figure 1 shows possible cases that meet Condition 1.
Revised: Figure 1 shows some cases that meet Condition 1.

Original: However, Figure 1a shows the casefor which the class centers collapse to a single point and thus the inter-class distance equals zero(collapse of  feature space).
Revised: Figure 1a shows the case for which the class centers collapse to a single point and thus the inter-class distance equals zero (collapse of the feature space).

Original: Figures 1b and 1c showtwo othervalid cases.
Revised: Figures 1b and 1c show other cases.

Original:  Methodology
Revised:  Methodology

Original: Condition 1 points to a promising research direction in learning discriminative feature spaces.
Revised: Condition 1 points to a promising research direction in learning discriminative feature spaces.

Original: However, optimizing over the rank is NP-hard and non-differentiable.
Revised: However, optimizing over the rank is NP-hard and non-differentiable.

Original: In what follows, we provide theformulation of Generalized Laplacian EigeNmaps (GLEN) and its relaxation, which is differentiable.
Revised: In what follows, we provide the formulation of Generalized Laplacian EigeNmaps (GLEN) and its relaxation, which is differentiable.

Original: 4.1 Generalized Laplacian Eigenmaps
Revised: 4.1 Generalized Laplacian Eigenmaps

Original: As solving Condition 1 is NP-hard, we propose a relaxation where Rank( S t ) is encouraged to be aslarge as possible (bounded by the feature dimension m ).
Revised: As solving Condition 1 is NP-hard, we propose a relaxation where Rank( S t ) is encouraged to be as large as possible (bounded by the feature dimension m ).

Original: On the contrary, if Rank( S t ) ≈ Rank( S b )then the small Rank( S w ) limits the feature diversity.
Revised: On the contrary, if Rank( S t ) ≈ Rank( S b ) then the small Rank( S w ) limits the feature diversity.

Original: In the extreme case, if Rank( S w ) = 0 , thefeature representation collapses.
Revised: In the extreme case, if Rank( S w ) = 0 , the feature representation collapses.

Original: Larger 

Revised: Larger

Original: <
Revised: <

Original: Rank( S b ) ≤ C − 1 improves the inter-class diversity.
Revised: Rank( S b ) ≤ C − 1 improves the inter-class diversity.

Original: In this paper, we propose a new Generalized Laplacian EigeNmaps (GLEN) framework for unsupervised network embedding.
Revised: We propose a new Generalized Laplacian EigeNmaps (GLEN) framework for unsupervised network embedding.

Original: In the most general form, GLEN maximizes the difference of rank terms:
Revised: In the most general form, GLEN maximizes the difference of rank terms:

Original: [Equation]
Revised: [Equation]

Original: However, the general matrix Rank Minimization Problem (RMP) [8] is NP-hard and the differenceof rank terms in Eq.
Revised: However, the general matrix Rank Minimization Problem (RMP) [7] is NP-hard and the difference of rank terms in Eq.

Original: 3 is possibly a harderproblem to solve.
Revised: 3 is even harder to solve.

Original: Thus, we relax this problem by thedifference of logdet terms that serve as a surrogate of the NP-hard problem. 
Revised: Thus, we relax this problem by the difference of LogDet terms that serve as a surrogate of the NP-hard problem. GLEN (LogDet relaxation).

Original: We define:
Revised: I. Define:

Original: [Equation]
Revised: [Equation]

Original: where λ ≥ 0 controls the impact of log det( S w ) term.
Revised: where λ ≥ 0 controls the impact of log det( S w ) .

Original: If λ = 0 , δ rf encourages Rank( f Θ ( X ))
Revised: If λ = 0 , δ rf encourages Rank( f Θ ( X ))

Original: = m .
Revised: = m .

Original: Moreover, let S t = f Θ ( X ) ⊤ L t f Θ ( X ) and S
Revised: II. Let S t = f Θ ( X ) ⊤ L t f Θ ( X ) and S

Original: w = f Θ ( X )
Revised: w = f Θ ( X )

Original: ⊤ L w f Θ ( X ) which yields:
Revised: ⊤ L w f Θ ( X ) . Then the LogDet relaxation becomes:

Original: [Equation]
Revised: [Equation]

Original: where I ensures I + αf Θ ( X ) ⊤ L f Θ ( X ) > 0 as f Θ ( X ) ⊤ L f Θ ( X ) may be semi-SPD matrix leadingto det( f Θ ( X )
Revised: where I ensures I + αf Θ ( X ) ⊤ L f Θ ( X ) > 0 as f Θ ( X ) ⊤ L f Θ ( X ) may be S m + leading to det( f Θ ( X )

Original: ⊤ L f Θ ( X ))
Revised: ⊤ L f Θ ( X ))

Original: = 0 .
Revised: = 0 .

Original: Thus, we use log det( I + α Y ) as a smooth surrogate for Rank( Y ) .
Revised: Thus, we use log det( I + α Y ) as a smooth surrogate for Rank( Y ) .

Original: Proposition 1.
Revised: Proposition 1.

Original: Let σ ( X ) be the vector of eigenvalues of SPD matrix X , and Eig( X ) be a diagonalmatrix with σ ( X ) as its diagonal.
Revised: Let σ ( X ) be the vector of eigenvalues of SPD matrix X , and Eig( X ) be a diagonal matrix with σ ( X ) as its diagonal.

Original: Let X , X ′ ∈ 
Revised: Let X , X ′ ∈ S

Original: P m (the set of all SPD matrices of size m × m ), andα > 0 .
Revised: m ++ , and α > 0 .

Original: Then, δ rf ( X , X ′ ; α ) =
Revised: Then, δ rf ( X , X ′ ; α ) =

Original: δ rf (Eig( X ) , Eig( X ′ ); α ) , i.e ., δ rf ( · ) depends on eigenvalues ratherthan eigenvectors of X and X ′ . 
Revised: δ rf (Eig( X ) , Eig( X ′ ); α ) , i.e ., δ rf ( · ) depends on eigenvalues rather than eigenvectors of X and X ′ . Proof. The proof follows from the equality det( I + α X ) = (cid:81)

Revised: i σ i

Revised: ( I + α X ) =

Original: Theoretical Analysis
Revised: (cid:81) i (1 + ασ i ( X )) = det( I + α Eig( X )) . Thus δ rf ( X , X ′ ; α ) = log det( I + α X ) − λ log det( I + α X ′ )

Revised: = log det( I + α Eig( X ))

Revised: − λ log det( I + α Eig( X ′ ))

Original:  Below, we demonstrate the comparison between our approach and other methods by looking at (i) thelow-rank optimization and (ii) the non-Euclidean distances between SPD matrices.
Revised: = δ rf (Eig( X ) , Eig( X ′ ); α ) .  Theoretical Analysis Below, we demonstrate the comparison between our approach and other methods by looking at (i) the low-rank optimization and (ii) the non-Euclidean distances between SPD matrices.

Original: 5.1 Nuclear Norm vs . Logdet for Rank Minimization
Revised: 5.1 Nuclear Norm vs . LogDet for Rank Minimization

Original: Claim 1.
Revised: Claim 1.

Original: COLES [45] is a convex relaxation (using the nuclear norm) of the rank difference in Eq.
Revised: COLES [51] is a convex relaxation (using the nuclear norm) of the rank difference in Eq.

Original: 3: Θ [Equation] = arg max Θ
Revised: 3: Θ ∗ = arg max Θ

Original: [Equation]
Revised: [Equation]

Original: where Tr (cid:0) f Θ ( X ) ⊤ L t f Θ ( X ) (cid:1) =
Revised: where Tr (cid:0) f Θ ( X ) ⊤ L t f Θ ( X ) (cid:1) =

Original: ∥
Revised: ∥

Original: S t ∥ ∗ and Tr (cid:0) f Θ ( X )
Revised: S t ∥ ∗ and Tr (cid:0) f Θ ( X )

Original: ⊤ L w f Θ ( X ) (cid:1) =
Revised: ⊤ L w f Θ ( X ) (cid:1) =

Original: ∥
Revised: ∥

Original: S w ∥ ∗ .
Revised: S w ∥ ∗ .

Original: The nuclear norm ∥ · ∥ ∗ can be regarded as the ℓ 1 norm over singular values.
Revised: The nuclear norm ∥ · ∥ ∗ can be regarded as the ℓ 1 norm over singular values.

Original: As the ℓ 1 norminduces sparsity, the nuclear norm encourages sparse singular values leading to low-rank solutions.
Revised: As the ℓ 1 norm induces sparsity, the nuclear norm encourages sparse singular values leading to low-rank solutions.

Original: If f Θ ( X ) ⊤ f Θ ( X ) is restricted to be diagonal, ∥ f Θ ( X )
Revised: If f Θ ( X ) ⊤ f Θ ( X ) is restricted to be diagonal, ∥ f Θ ( X )

Original: ⊤ f Θ ( X ) ∥ ∗ =
Revised: ⊤ f Θ ( X ) ∥ ∗ =

Original: ∥
Revised: ∥

Original: Diag (cid:0) f Θ ( X ) ⊤ f Θ ( X ) (cid:1)∥
Revised: Diag (cid:0) f Θ ( X ) ⊤ f Θ ( X ) (cid:1)

Original: 1and the nuclear norm surrogate for the rank minimization reduces to the ℓ 1 norm surrogate for thecardinality (rank) minimization.
Revised: ∥and the nuclear norm surrogate for the rank minimization reduces to the ℓ 1 norm surrogate for the cardinality (rank) minimization.

Original: However, for the m -dimensional embedding, the solution of tracedifference lies on a subspace of dimension less than m − 1 [4].
Revised: However, for the m -dimensional embedding, the solution of trace difference lies on a subspace of dimension less than m − 1 [3].

Original: Thus, the constraint Ω( f Θ ( X )) =
Revised: Thus, the constraint Ω( f Θ ( X )) =

Original: Bprevents the dimensional collapse, i.e ., f Θ ( X ) ⊤ f Θ ( X ) =
Revised: B prevents the dimensional collapse, i.e ., f Θ ( X ) ⊤ f Θ ( X ) =

Original: I .
Revised: I .

Original: Compared with the trace-based relaxation, logdet is more suitable for cardinality minimization as itis less sensitive to large singular values.
Revised: Compared with the trace-based relaxation, LogDet is more suitable for cardinality minimization as it is less sensitive to large singular values.

Original: Also, the difference of logdet terms does not require decorrelation of features to prevent the dimensional collapse.
Revised: Also, the difference of LogDet terms does not require decorrelation of features to prevent the dimensional collapse.

Original: We discuss this matter in the supplementarymaterial.
Revised: We discuss this matter in the supplementary material.

Original: In our case, the difference of logdet terms is always bounded by the difference of traceterms as follows.
Revised: In our case, the difference of LogDet terms is always bounded by the difference of trace terms as follows.

Original: Proposition 2.
Revised: Proposition 2.

Original: Given an embedding matrix f Θ ( X ) ∈ R n × m , a fixed small constant α > 0 , we havethe following inequality:
Revised: Given an embedding matrix f Θ ( X ) ∈ R n × m , a fixed small constant α > 0 , we have the following inequality:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: Proposition 2 is important because Rank( Y ) ≤ log det(  Y ) ≤
Revised: Proposition 2 follows Rank( Y ) ≤ log det( I + Y ) ≤

Original: Tr( Y ) [8].
Revised: Tr( Y ) [7].

Original: 5.2 Distance between Positive Definite Matrices
Revised: 5.2 Distance between Positive Definite Matrices

Original: Below, we provide a perspective on non-Euclidean distances between SPD matrices to compare theproposed method with other graph embeddings, e.g ., Laplacian Eigenmaps [4] and COLES [45].
Revised: Below, we provide a perspective on non-Euclidean distances between SPD matrices to compare the proposed method with other graph embeddings, e.g ., Laplacian Eigenmaps [3] and COLES [51].

Original: Forthe sake of comparison, we reformulate the Laplacian eigenmaps and COLES into forms in Prop. 3and 4.
Revised: For the sake of comparison, we reformulate the Laplacian eigenmaps and COLES into forms in Prop.and 4.

Original: Proposition 3.
Revised: Proposition 3.

Original: Laplacian Eigenmaps [4] equals to maximizing the Frobenius norm:
Revised: Laplacian Eigenmaps [3] equals to maximizing the Frobenius norm:

Original: [Equation]
Revised: [Equation]

Original: 200 Proposition 4.
Revised: Proposition 4.

Original: Contrastive Laplacian Eigenmaps [45] equals to maximizing the difference of
Revised: Contrastive Laplacian Eigenmaps [51] equals to maximizing the difference of

Original: Frobenius norm terms:
Revised: Frobenius norm terms:

Original: [Equation]
Revised: [Equation]

Original: 203 Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: Note that Eq.
Revised: Note that Eq.

Original: 9 encourages the linear kernel matrix f Θ ( X ) f Θ ( X ) ⊤ to be close to ˜ W w while Eq. 10encourage the linear kernel matrix to be far from the ˜ W w at the same time.
Revised: 9 encourages the linear kernel matrix f Θ ( X ) f Θ ( X ) ⊤ to be close to ˜ W w while Eq.encourage the linear kernel matrix to be far from the ˜ W w at the same time.

Original: Our loss follows the non-Euclidean geometry.
Revised: Our loss follows the non-Euclidean geometry.

Original: In the following part, we demonstrate the relationamong Eq.
Revised: In the following part, we demonstrate the relation among Eq.

Original: 4, the Affine-invariant Riemannian metric (AIRM), and Jessen-Bregman LogDet Divergence (JBLD).
Revised: 4, the Affine-invariant Riemannian metric (AIRM), and Jessen-Bregman LogDet Divergence (JBLD).

Original: Indeed, our loss function is bounded from both sides by AIRM and AIRM scaled by √ m respectively.
Revised: Indeed, our loss function is bounded from both sides by AIRM and AIRM scaled by √ m respectively.

Original: Proposition 5.
Revised: Proposition 5.

Original: Let σ ( X ) be the vector of eigenvalues of X ,for any semi-SPD matrix S t , S w ∈ P ∗ m(asterisk * denotes semi-SPD set), we have:
Revised: Let σ ( X ) be the vector of eigenvalues of X ,

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Given A = I + S t and B = I + S w , we have: 
Revised: Given A = I + S t and B = I + S w , we have: log

Original: [Equation]
Revised: [Equation]

Revised: A

Revised: [Equation]

Original: We have Tr( A ) =
Revised: We have Tr( A ) =

Original: ∥ σ ( A ) ∥ 1 , ∥
Revised: ∥ σ ( A ) ∥ 1 , ∥

Original: A ∥ F = ∥ σ ( A ) ∥ 2 and ∥ x ∥ 2 ≤ ∥ x ∥ 1 ≤ √ m ∥ x
Revised: A ∥ F = ∥ σ ( A ) ∥ 2 and ∥ x ∥ 2 ≤ ∥ x ∥ 1 ≤ √ m ∥ x

Original: ∥ 2 .
Revised: ∥ 2 .

Original: Thus, Eq. 4 is trying to find a mapping function maximizing an approximation of AIRM distancebetween the total scatter matrix and the within-class matrix.
Revised: Thus, Eq. 4 is trying to find a mapping function maximizing an approximation of AIRM distance between the total scatter matrix and the within-class matrix.

Original: Definition 1.
Revised: Definition 1.

Original: Given any SPD matrix A , B ∈ P m , we define the regularized Jensen Bregman Logdet
Revised: Given any SPD matrix A , B ∈ P m , we define the regularized Jensen Bregman LogDet

Original: Divergence as follows:
Revised: Divergence as follows:

Original: [Equation]
Revised: [Equation]

Original: where I can be interpreted as a regularization to prevent det( A )
Revised: where I can be interpreted as a regularization to prevent det( A )

Original: = 0 and log(det( A ))
Revised: = 0 and log(det( A ))

Original: =
Revised: =

Original: −∞ .
Revised: −∞ .

Original: Proposition 6.
Revised: Proposition 6.

Original: For any S w , S t ∈ P m and α ≥ 0 , we have:
Revised: For any S w , S t ∈ P m and α ≥ 0 , we have:

Original: [Equation]
Revised: [Equation]

Revised: where the equality holds only if α = 0 and then det( I + α S b )

Revised: = 1 .

Original: Proof.
Revised: Proof.

Original: Based on Eq.
Revised: Based on Eq.

Original: 14, we notice JBLD is a lower bound of our loss: 
Revised: 14, we notice JBLD is a lower bound of our loss: J

Original: [Equation]
Revised: [Equation]

Original: Thus, Eq. 4 can be also interpreted as find a mapping function maximizing the regularized JBLDbetween the within-class matrix and the between-class matrix.
Revised: Thus, Eq. 4 can be also interpreted as find a mapping function maximizing the regularized JBLD between the within-class matrix and the between-class matrix.

Original:  Experiments
Revised:  Experiments

Original: We evaluate GLEN (its relaxation) on transductive and inductive node classification tasks andnode clustering.
Revised: We evaluate GLEN (its relaxation) on transductive and inductive node classification tasks and node clustering.

Original: GLEN is compared to popular unsupervised, contrastive, and (semi-)supervisedapproaches.
Revised: GLEN is compared to popular unsupervised, contrastive, and (semi-)supervised approaches.

Original: Except for the classifier, unsupervised models do not use labels.
Revised: Except for the classifier, unsupervised models do not use labels.

Original: To learn similarity/dissimilarity, contrastive models employ the contrastive setting.
Revised: To learn similarity/dissimilarity, contrastive models employ the contrastive setting.

Original: Labels are used to train the projectionlayer and classifier in semi-supervised models.
Revised: Labels are used to train the projection layer and classifier in semi-supervised models.

Original: A fraction of nodes ( i.e ., 5 or 20 per class) used fortraining are labeled for semi-supervised setting.
Revised: A fraction of nodes ( i.e ., 5 or 20 per class) used for training are labeled for semi-supervised setting.

Original: A SoftMax classifier is used for (semi-)supervisedmodels, while a logistic regression classifier is used for unsupervised and contrastive approaches.
Revised: A SoftMax classifier is used for (semi-)supervised models, while a logistic regression classifier is used for unsupervised and contrastive approaches.

Original: Datasets.
Revised: Datasets.

Original: GLEN is evaluated on four citation networks: Cora, Citeseer, Pubmed, Cora Full [17, 5]for transductive setting.
Revised: GLEN is evaluated on four citation networks: Cora, Citeseer, Pubmed, Cora Full [16, 4] for transductive setting.

Original: We also employ the large scale Ogbn-arxiv from OGB [14].
Revised: We also employ the large scale Ogbn-arxiv from OGB [13].

Original: Finally, the
Revised: Finally, the

Original: Reddit [40] dataset is used in inductive setting.
Revised: Reddit [42] dataset is used in inductive setting.

Original: See supplementary material for details of datasets.
Revised: See supplementary material for details of datasets.

Original: Metrics.
Revised: Metrics.

Original: As fixed data splits [38] often on transductive models benefit models that overfit, we averageresults over 50 random splits for each dataset.
Revised: As fixed data splits [40] often on transductive models benefit models that overfit, we average results over 50 random splits for each dataset.

Original: We evaluate performance for 5 and 20 samples perclass.
Revised: We evaluate performance for 5 and 20 samples per class.

Original: The inductive task is typically tested on graphs that are significantly larger.
Revised: The inductive task is typically tested on graphs that are significantly larger.

Original: As in prior works,we use fixed data splits and publish Micro-F1 scores averaged across 10 runs.
Revised: As in prior works, we use fixed data splits and publish Micro-F1 scores averaged across 10 runs.

Original: Baseline models.
Revised: Baseline models.

Original: We group baseline models into unsupervised, contrastive and (semi-)supervisedmethods, and implement them in the same framework/testbed.
Revised: We group baseline models into unsupervised, contrastive and (semi-)supervised methods, and implement them in the same framework/testbed.

Original: Contrastive methods include DeepWalk [27], GCN+SampledNCE developed as an alternative to GraphSAGE+SampledNCE [10],
Revised: Contrastive methods include DeepWalk [28], GCN+SampledNCE developed as an alternative to GraphSAGE+SampledNCE [9],

Original: Graph2Gauss [5], SCE [41], DGI [31], GRACE [46], GCA [47], GraphCL [39] and COLES [45],which are our main competitors.
Revised: Graph2Gauss [4], SCE [43], DGI [33], GRACE [52], GCA [53], GraphCL [41] and COLES [51], which are our main competitors.

Original: Note that GRACE, GCA and GraphCL are based on multi-view anddata augmentation, and GraphCL is mainly intended for graph classification.
Revised: Note that GRACE, GCA and GraphCL are based on multi-view and data augmentation [Table] and GraphCL is mainly intended for graph classification.

Original: We do not study graphclassification as it requires advanced node pooling with mixed- or high-order statistics [34, 18].
Revised: We do not study graph classification as it requires advanced node pooling with mixed- or high-order statistics [36, 18, 19]

Original: We
Revised: [Table]

Original: [Table]
Revised: [Table]

Original:  compare results with representative (semi-)supervised GCN [17], GAT [31] and MixHop [1] models.
Revised: We compare results with representative (semi-)supervised GCN [16], GAT [33] and MixHop [1] models.

Original: SGC and S 2 GC are unsupervised spectral filter networks.
Revised: SGC and S 2 GC are unsupervised spectral filter networks.

Original: They do not have any learnable parameters.
Revised: They do not have any learnable parameters.

Original: COLES and GLEN could be regarded as dimension reduction techniques for SGC and S 2 GC, thuswe compare them to PCA-S 2 GC and RP-S 2 GC, which use PCA and random projections to obtain theprojection layer.
Revised: COLES and GLEN could be regarded as dimension reduction techniques for SGC and S 2 GC, thus we compare them to PCA-S 2 GC and RP-S 2 GC, which use PCA and random projections to obtain the projection layer.

Original: We set hyperparameters based on the settings described in prior papers.
Revised: We set hyperparameters based on the settings described in prior papers.

Original: 6.1 Transductive Learning
Revised: 6.1 Transductive Learning

Original: In this section, we consider transductive learning where all nodes are available in the training process.
Revised: In this section, we consider transductive learning where all nodes are available in the training process.

Original: COLES vs .
Revised: COLES vs .

Original: GLEN.
Revised: GLEN.

Original: Table 1 shows the performance of GLEN vs .
Revised: Table 9 shows the performance of GLEN vs .

Original: COLES on two different backbones,i.e ., GCN and S 2 GC.
Revised: COLES on two different backbones, i.e ., GCN and S 2 GC.

Original: On both backbones, GLEN shows non-trivial improvements on all four datasets.
Revised: On both backbones, GLEN shows non-trivial improvements on all four datasets.

Original: GLEN-S 2 GC outperforms the COLES by up to 4.6%.
Revised: GLEN-S 2 GC outperforms the COLES by up to 4.6%.

Original: Contrastive Embedding Baselines vs .
Revised: Contrastive Embedding Baselines vs .

Original: GLEN.
Revised: GLEN.

Original: Table 1 shows that GLEN-GCN and GLEN-S 2 GCoutperform unsupervised models.
Revised: Table 9 shows that GLEN-GCN and GLEN-S 2 GC outperform unsupervised models.

Original: In particular, GLEN-GCN outperforms GCN+SampledNCE onall four datasets, which shows that GLEN has an advantage over the SampledNCE framework.
Revised: In particular, GLEN-GCN outperforms GCN+SampledNCE on all four datasets, which shows that GLEN has an advantage over the SampledNCE framework.

Original: Inaddition, GLEN-S [-
-] GC outperforms the best contrastive baseline DGI by up to 3.4%.
Revised: In addition, GLEN-S [-
-]2 GC outperforms the best contrastive baseline DGI by up to 3.4%.

Original: On Cora with 5training samples, GLEN-S 2 GC outperforms S 2 GC by 6.8%.
Revised: On Cora withtraining samples, GLEN-S 2 GC outperforms S 2 GC by 6.8%.

Original: Finally, Table 3 shows that GLEN-S 2 GC(small number of trainable parameters) outperforms other methods on the challenging Ogbn-arxiv.
Revised: Finally, Table 3 shows that GLEN-S 2 GC (small number of trainable parameters) outperforms other methods on the challenging Ogbn-arxiv.

Original: Semi-supervised GNNs vs .
Revised: Semi-supervised GNNs vs .

Original: GLEN.
Revised: GLEN.

Original: Table 1 shows that the contrastive GCN baselines performworse than semi-supervised variants, especially when 20 labeled samples per class are available.
Revised: Table 9 shows that the contrastive GCN baselines perform worse than semi-supervised variants, especially when 20 labeled samples per class are available.

Original: Incontrast, GLEN-GCN outperformed the semi-supervised GCN on Cora by 10% and 3.4% given 5and 20 labeled samples per class.
Revised: In contrast, GLEN-GCN outperformed the semi-supervised GCN on Cora by 10% and 3.4% givenand 20 labeled samples per class.

Original: GLEN-GCN also outperforms GCN on Citeseer and Pubmed by9.9% and 5.2% given 5 labeled samples per class.
Revised: GLEN-GCN also outperforms GCN on Citeseer and Pubmed by 9.9% and 5.2% given 5 labeled samples per class.

Original: These results show the superiority of GLEN onfour datasets when the number of samples per class is 5.
Revised: These results show the superiority of GLEN on four datasets when the number of samples per class is 5.

Original: Even for 20 labeled samples per class,
Revised: Even for 20 labeled samples per class,

Original: GLEN-S 2 GC outperforms the best semi-supervised baselines on all four datasets e.g ., by 3.3% on
Revised: GLEN-S 2 GC outperforms the best semi-supervised baselines on all four datasets e.g ., by 3.3% on

Original: Cora.
Revised: Cora.

Original: Semi-supervised models ( e.g ., GAT and MixHop) are affected by the low number of labeledsamples, which is consistent with [22].
Revised: Semi-supervised models ( e.g ., GAT and MixHop) are affected by the low number of labeled samples, which is consistent with [24].

Original: The accuracy of GLEN-GCN and GLEN-S 2 GC is unaffected.
Revised: The accuracy of GLEN-GCN and GLEN-S 2 GC is unaffected.

Original: Unsupervised GNNs vs .
Revised: Unsupervised GNNs vs .

Original: GLEN.
Revised: GLEN.

Original: SGC and S 2 GC are unsupervised linear networks based on spectralfilters which do not use labels (except for the classifier).
Revised: SGC and S 2 GC are unsupervised linear networks based on spectral filters which do not use labels (except for the classifier).

Original: As a dimension reduction method, GLENhelps both methods reduce the dimension and achieve discriminative features.
Revised: As a dimension reduction method, GLEN helps both methods reduce the dimension and achieve discriminative features.

Original: Table 1 shows that
Revised: Table 9 shows that

Original: GLEN-S 2 GC outperforms RP-S 2 GC and PCA-S 2 GC under the same projection size.
Revised: GLEN-S 2 GC outperforms RP-S 2 GC and PCA-S 2 GC under the same projection size.

Original: GLEN-S [-
 GCalso-] outperforms the unsupervised S 2 GC baseline (high-dimensional representation).
Revised: GLEN-S [-
 GCalso-]2 GC also outperforms the unsupervised S 2 GC baseline (high-dimensional representation).

Original: 6.2 Inductive Learning
Revised: 6.2 Inductive Learning

Original: In inductive learning, models have no access to the test set, thus they need to generalize well tounseen samples.
Revised: In inductive learning, models have no access to the test set, thus they need to generalize well to unseen samples.

Original: Table 2 shows that GLEN and COLES enjoy a significant performance gain (1% -5% in Micro-F1 scores).
Revised: Table 2 shows that GLEN and COLES enjoy a significant performance gain (1% - 5% in Micro-F1 scores).

Original: GLEN (low memory footprint) outperforms SAGE-LSTM.
Revised: GLEN (low memory footprint) outperforms SAGE-LSTM.

Original: In contrast, DGIon Reddit triggered out-of-memory error on Nvidia GTX 1080 GPU (94.0 Micro-F1 is from [31]).
Revised: In contrast, DGI on Reddit triggered out-of-memory error on Nvidia GTX 1080 GPU (94.0 Micro-F1 is from [33]).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: 6.3 Node Clustering
Revised: 6.3 Node Clustering

Original: We compare GLEN-GCN and GLEN-S 2 GC with three types of clustering methods:
Revised: We compare GLEN-GCN and GLEN-S 2 GC with three types of clustering methods:

Original: i. Methods that use only node features e
Revised: i. Methods that use only node features e

Original: g ., k-means and spectral clustering (spectral-f) construct asimilarity matrix with the node features by a linear kernel.
Revised: g ., k-means and spectral clustering (spectral-f) construct a similarity matrix with the node features by a linear kernel.

Original: ii.
Revised: ii.

Original: Structural clustering methods that only use the graph structure: spectral clustering (spectral-g) that takes the graph adjacency matrix as the similarity matrix, and DeepWalk [27].iii.
Revised: Structural clustering methods that only use the graph structure: spectral clustering (spectral-g) that takes the graph adjacency matrix as the similarity matrix, and DeepWalk [28]. iii.

Original: Attributed graph clustering methods that use node features and the graph: Graph Autoencoder (GAE), Graph Variational Autoencoder (VGAE)
Revised: Attributed graph clustering methods that use node features and the graph: Graph Autoencoder (GAE), Graph Variational Autoencoder (VGAE)

Original: [17], Adversarially Regularized Graph Autoen- coder (ARGE), Var.
Revised: [16], Adversarially Regularized Graph Autoencoder (ARGE), Var.

Original: Graph Autoencoder (ARVGE , SGC 
Revised: Graph Autoencoder (ARVGE) [27], SGC [37] , S 2 GC [49], COLES [51].

Original: , S GC [44] COLES [45]. We measure the performance by the clustering Accuracy (Acc), Normalized Mutual Information(NMI) and macro F1-score (F1).
Revised: We measure the performance by the clustering Accuracy (Acc), Normalized Mutual Information (NMI) and macro F1-score (F1).

Original: We run each method 10 times on Cora, CiteSeer and PubMed.
Revised: We run each method 10 times on Cora, CiteSeer and PubMed.

Original: Weset the number of propagation steps to 8 for SGC, S 2 GC, COLES-S 2 GC and COLES-S 2 GC following[42].
Revised: We set the number of propagation steps to 8 for SGC, S 2 GC, COLES-S 2 GC and COLES-S 2 GC following [44].

Original: Table 4 shows that GLEN-S 2 GC outperforms other methods in all cases, whereas 
Revised: Table 4 shows that GLEN-S 2 GC outperforms other methods in all cases, whereas GLEN-GCN outperforms COLES-

Original: Scalability.
Revised: Scalability.

Original: GraphSAGE and DGI require neighbor sampling with redundant forward/backward steps (long runtime).
Revised: GraphSAGE and DGI require neighbor sampling with redundant forward/backward steps (long runtime).

Original: In contrast, GLEN-S 2 GC enjoys a simple implementation with low memoryusage/low runtime.
Revised: In contrast, GLEN-S 2 GC enjoys a simple implementation with low memory usage/low runtime.

Original: For graphs with over 100 thousands nodes and 10 millions edges (Reddit),
Revised: For graphs with over 100 thousands nodes and 10 millions edges (Reddit),

Original: GLEN runs fast on NVIDIA 1080 GPU.
Revised: GLEN runs fast on NVIDIA 1080 GPU.

Original: Even on larger graph benchmarks, GLEN is fast as itoptimizes the total scatter and the within-class matrices whose size depends on embedding sizerather than the node number.
Revised: Even on larger graph benchmarks, GLEN is fast as it optimizes the total scatter and the within-class matrices whose size depends on embedding size rather than the node number.

Original: The runtime of GLEN-S 2 GC is also favourable in comparison tomulti-view augmentation-based GraphCL.
Revised: The runtime of GLEN-S 2 GC is also favourable in comparison to multi-view augmentation-based GraphCL.

Original: Specifically, GLEN-S 2 GC took 0.54s, 0.3s, 5.3s and 15.4son Cora, Citeseer, Pubmed and Cora Full, respectively.
Revised: Specifically, GLEN-S 2 GC took 0.54s, 0.3s, 5.3s and 15.4s on Cora, Citeseer, Pubmed and Cora Full, respectively.

Original: GraphCL took 110.19s, 101.0s, ≥ 8 h and≥ 8 h respectively.
Revised: GraphCL took 110.19s, 101.0s, ≥ 8 h and ≥ 8 h respectively.

Original: Although the logdet difference is somewhat slower than the trace difference inforward/backward propagation, it converges faster, thus enjoying a similar low runtime.
Revised: Although the LogDet difference is somewhat slower than the trace difference in forward/backward propagation, it converges faster, thus enjoying a similar low runtime.

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we model contrastvie learning as a rank difference problem to approximate the conditionthat the rank of total scatter matrix should equal the sum of ranks of within-scatter and between-scattermatrices.
Revised: In this paper, we model contrastvie learning as a rank difference problem to approximate the condition that the rank of total scatter matrix should equal the sum of ranks of within-scatter and between-scatter matrices.

Original: We relax this NP-hard assumption with a differentiable difference of logdet terms.
Revised: We relax this NP-hard assumption with a differentiable difference of LogDet terms.

Original: We alsoshow two perspectives on GLEN and the existing methods based on the low-rank optimization anddistance between SPD matrices.
Revised: We also show two perspectives on GLEN and the existing methods based on the low-rank optimization and distance between SPD matrices.

Original: In low-rank optimization, we explain why the logdet difference isa better surrogate function to optimize rank difference compared to the trace difference.
Revised: In low-rank optimization, we explain why the LogDet difference is a better surrogate function to optimize rank difference compared to the trace difference.

Original: We alsoshow that our solution encourages linear kernel of embeddings become the geometric mean betweenthe total scatter matrix and the within-class matrix.
Revised: We also show that our solution encourages linear kernel of embeddings become the geometric mean between the total scatter matrix and the within-class matrix.

Original: Our GLEN works well with many backbonesoutperforming many unsupervised, contrastive and (semi-)supervised methods.
Revised: Our GLEN works well with many backbones outperforming many unsupervised, contrastive and (semi-)supervised methods.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Original: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .•
Revised: [Yes] See Section ?.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data are proprietary.

Original:  Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Original: i. For allauthors...
Revised: i. For all

Original:  (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] ii.
Revised: [Yes] ii.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] iii.
Revised: [Yes] iii.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] iv.
Revised: [Yes] iv.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: v. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: v. If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]