Original: Q UANTUM -I NSPIRED T ENSORIZED E MBEDDING WITH A PPLICATION TO N ODE R EPRESENTATION L EARNING Anonymous authors
Revised: Q UANTUM -I NSPIRED T ENSORIZED E MBEDDING WITH A PPLICATION TO N ODE R EPRESENTATION L EARNING Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Node representation learning a.k.a.
Revised: Node representation learning a.k.a.

Original: network embedding (NE) is an essential technique for network analysis by representing nodes as vectors.
Revised: network embedding (NE) is an essential technique for network analysis by representing nodes as vectors.

Original: Most existing NE algorithms require a O ( p ) space complexity for each node embedding ( p is the ﬁnal embedding vector size).
Revised: Most existing NE algorithms require a O ( p ) space complexity for each node embedding ( p is the ﬁnal embedding vector size).

Original: Impressed and inspired by the large Hilbert space of quantum systems
Revised: Impressed and inspired by the large Hilbert space of quantum systems

Original: , we propose a brand new NE algorithm node2ket and its variant node2kek+ by imitating behaviors of quantum systems.
Revised: , we propose a brand new NE algorithm node2ket and its variant node2kek+ by imitating behaviors of quantum systems.

Original: Theoretically
Revised: Theoretically

Original: , we give analysis on how it uniﬁes existing embedding methods including both conventional ones and tensorized ones, and prove that our methods achieve to use
Revised: , we give analysis on how it uniﬁes existing embedding methods including both conventional ones and tensorized ones, and prove that our methods achieve to use

Original: O ( p 1 / ln p ln p ) or even smaller space for each node embedding
Revised: O ( p 1 / ln p ln p ) or even smaller space for each node embedding

Original: We are the ﬁrst to successfully conduct node embedding in the super large Hilbert space
Revised: We are the ﬁrst to successfully conduct node embedding in the super large Hilbert space

Original: Experiments are conducted on ﬁve public real-world networks where methods are evaluated through the tasks of network reconstruction link prediction, and node classiﬁcation
Revised: Experiments are conducted on ﬁve public real-world networks where methods are evaluated through the tasks of network reconstruction link prediction, and node classiﬁcation

Original: On
Revised: On

Original: BlogCatalog our method achieves to outperform all baselines with 1/32 training parameters and 1/16 running time on the same machine
Revised: BlogCatalog our method achieves to outperform all baselines with 1/32 training parameters and 1/16 running time on the same machine

Original: On
Revised: On

Original: DBLP the reconstruction precision of node2ket achieves to be 3 times higher than the best baseline
Revised: DBLP the reconstruction precision of node2ket achieves to be 3 times higher than the best baseline

Original: i
Revised: i

Original: .e
Revised: .e

Original: LouvainNE
Revised: LouvainNE

Original: The source code will be publicly available
Revised: The source code will be publicly available

Original: Hilbert space is a big place!
Revised: Hilbert space is a big place!

Original: – Carlton Morris Caves.
Revised: – Carlton Morris Caves.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Node representation learning, also known as network embedding (NE), has been well-studied in decades.
Revised: Node representation learning, also known as network embedding (NE), has been well-studied in decades.

Original: Borrowing idea from word embedding in NLP, deepwalk (Perozzi et al., 2014)
Revised: Borrowing idea from word embedding in NLP, deepwalk (Perozzi et al., 2014)

Original: ﬁrst proposes to learn node embeddings for downstream tasks by feeding random walk sequences to word2vec (Mikolov et al., 2013).
Revised: ﬁrst proposes to learn node embeddings for downstream tasks by feeding random walk sequences to word2vec (Mikolov et al., 2013).

Original: Afterward, numerous NE methods for different types of networks (e.g. heterogeneous/multiplex networks) and for different purposes (e.g. preserving structural similarity) are proposed (see Sec.
Revised: Afterward, numerous NE methods for different types of networks (e.g. heterogeneous/multiplex networks) and for different purposes (e.g. preserving structural similarity) are proposed (see Sec.

Original: for more).
Revised: for more).

Original: Despite the great success, the existing NE methods still suffer from the high space complexity for storing/training the embeddings whose size grows linearly to the multiplication of the number of nodes and the embedding dimension (Xiong et al., 2022) (see
Revised: Despite the great success, the existing NE methods still suffer from the high space complexity for storing/training the embeddings whose size grows linearly to the multiplication of the number of nodes and the embedding dimension (Xiong et al., 2022) (see

Original: Appendix A for details).
Revised: Appendix A for details).

Original: As a result, there is difﬁculty in deploying these methods on memorylimited devices for privacy-sensitive computing e.g. recommender systems in social networks on mobile phones.
Revised: As a result, there is difﬁculty in deploying these methods on memorylimited devices for privacy-sensitive computing e.g. recommender systems in social networks on mobile phones.

Original: Meanwhile, the expressiveness of the embedding vectors can be restricted.
Revised: Meanwhile, the expressiveness of the embedding vectors can be restricted.

Original: Recently, inspired by quantum mechanics, word2ket (Panahi et al., 2020) was proposed for compressive word embedding by imitating the behaviors of quantum bits in quantum systems.
Revised: Recently, inspired by quantum mechanics, word2ket (Panahi et al., 2020) was proposed for compressive word embedding by imitating the behaviors of quantum bits in quantum systems.

Original: By tensor entanglement, word2ket achieves both a high compressive ratio and experimental results comparable to conventional embeddings.
Revised: By tensor entanglement, word2ket achieves both a high compressive ratio and experimental results comparable to conventional embeddings.

Original: Though providing many insights in generating embeddings, word2ket still faces difﬁculties transferring to embedding problems in other ﬁelds such as node representation learning (Hamilton et al., 2017b).
Revised: Though providing many insights in generating embeddings, word2ket still faces difﬁculties transferring to embedding problems in other ﬁelds such as node representation learning (Hamilton et al., 2017b).

Original: These difﬁculties include: i) Word2ket generates embeddings from a ﬁxed complete binary tree, which neglects the latent structure of the input data and hence brings challenges to learning embeddings for data with explicit structures (e.g. networks).
Revised: These difﬁculties include: i) Word2ket generates embeddings from a ﬁxed complete binary tree, which neglects the latent structure of the input data and hence brings challenges to learning embeddings for data with explicit structures (e.g. networks).

Original: ii)
Revised: ii)

Original: Word2ket adopts deep neural networks to decode the embedding and is trained in a supervised way, which is rather different from many embedding methods (Tang et al., 2015; Grover & Leskovec, 2016) that decode embeddings via simply inner product operation and train embeddings by optimizing the objective based on Noise Contrastive Estimation (NCE) (Gutmann & Hyv¨arinen, 2010) in a self-supervised (contrastive learning) style.
Revised: Word2ket adopts deep neural networks to decode the embedding and is trained in a supervised way, which is rather different from many embedding methods (Tang et al., 2015; Grover & Leskovec, 2016) that decode embeddings via simply inner product operation and train embeddings by optimizing the objective based on Noise Contrastive Estimation (NCE) (Gutmann & Hyv¨arinen, 2010) in a self-supervised (contrastive learning) style.

Original: In this paper, we propose a novel network embedding paradigm with special treatments concerning the above two problems.
Revised: In this paper, we propose a novel network embedding paradigm with special treatments concerning the above two problems.

Original: For the embedding generation, we propose an embedding module named as Tensorized Embedding Blocks (TEBs) composed of columns of Tensor Units (TUs), each representing an embedding tensor.
Revised: For the embedding generation, we propose an embedding module named as Tensorized Embedding Blocks (TEBs) composed of columns of Tensor Units (TUs), each representing an embedding tensor.

Original: TEBs can unify the embedding layers in word2ket/word2ketXS by making changes in the way of indexing TUs (see Sec.
Revised: TEBs can unify the embedding layers in word2ket/word2ketXS by making changes in the way of indexing TUs (see Sec.

Original: 4.1 and Sec. 4.5).
Revised: 4.1 and Sec. 4.5).

Original: A good strategy of assigning TU indices to the embedded nodes will result in a model with both high expressiveness and space efﬁciency, which is proved by the experiments.
Revised: A good strategy of assigning TU indices to the embedded nodes will result in a model with both high expressiveness and space efﬁciency, which is proved by the experiments.

Original: For the embedding learning, we optimize the widely adopted NCE-based objective and succeed in learning node embeddings in a 2 32 -dimensional
Revised: For the embedding learning, we optimize the widely adopted NCE-based objective and succeed in learning node embeddings in a 2 32 -dimensional

Original: Hilbert space stably and robustly on a single machine (details in Sec. 4.2).
Revised: Hilbert space stably and robustly on a single machine (details in Sec. 4.2).

Original: Speciﬁcally, we propose two variants on different compressive levels, named as node2ket and node2ket+ .
Revised: Speciﬁcally, we propose two variants on different compressive levels, named as node2ket and node2ket+ .

Original: The former is a preliminary baseline and the latter is more compressive by utilizing graph partition techniques.
Revised: The former is a preliminary baseline and the latter is more compressive by utilizing graph partition techniques.

Original: Furthermore, we give a complexity analysis of the embedding modules in TEBs, and theoretically prove an upper bound of the minimum of the required space to store/train embeddings for node2ket and node2ket+ in Sec.
Revised: Furthermore, we give a complexity analysis of the embedding modules in TEBs, and theoretically prove an upper bound of the minimum of the required space to store/train embeddings for node2ket and node2ket+ in Sec.

Original: 4.3 and Sec.
Revised: 4.3 and Sec.

Original: 4.4 respectively.
Revised: 4.4 respectively.

Original: Our contributions are listed as follows
Revised: Our contributions are listed as follows

Original: : 1) We are the ﬁrst to formalize a general embedding architecture, Tensorized Embedding Blocks (TEBs), which generate embeddings in Hilbert space by tensor product and entanglement, imitating behaviors of quantum systems.
Revised: : 1) We are the ﬁrst to formalize a general embedding architecture, Tensorized Embedding Blocks (TEBs), which generate embeddings in Hilbert space by tensor product and entanglement, imitating behaviors of quantum systems.

Original: In this way, high-dimensional embeddings of high expressiveness in the Hilbert space can be obtained via a small number of parameters.
Revised: In this way, high-dimensional embeddings of high expressiveness in the Hilbert space can be obtained via a small number of parameters.

Original: Our tensorized embedding architecture is a generalization of the recent method word2ket/word2ketXS
Revised: Our tensorized embedding architecture is a generalization of the recent method word2ket/word2ketXS

Original: and also uniﬁes conventional embedding methods.
Revised: and also uniﬁes conventional embedding methods.

Original: We take an initiative (the ﬁrst time to our best knowledge) to adopt tensorized node embedding
Revised: We take an initiative (the ﬁrst time to our best knowledge) to adopt tensorized node embedding

Original: in representation learning based on Noise Contrastive Estimation.
Revised: in representation learning based on Noise Contrastive Estimation.

Original: Speciﬁcally, we propose two variants at different compressive levels, node2ket and node2ket +, with the bound of compressive power proved respectively.
Revised: Speciﬁcally, we propose two variants at different compressive levels, node2ket and node2ket +, with the bound of compressive power proved respectively.

Original: Compared with the preliminary baseline node2ket, node2ket+ further utilizes graph partition techniques in the design of TEBs, in which way features on different levels of granularity are preserved and the embedding model becomes more compressive.
Revised: Compared with the preliminary baseline node2ket, node2ket+ further utilizes graph partition techniques in the design of TEBs, in which way features on different levels of granularity are preserved and the embedding model becomes more compressive.

Original: 3) Experiments on network reconstruction and link prediction show the overwhelming performance of node2ket.
Revised: 3) Experiments on network reconstruction and link prediction show the overwhelming performance of node2ket.

Original: Notably, in the network reconstruction experiments, it becomes the top method on
Revised: Notably, in the network reconstruction experiments, it becomes the top method on

Original: BlogCatalog with 1/32 training parameters and 1/16 running time, and the precision on DBLP is more than 3 times higher than the best one of baselines i.e. LouvainNE (Bhowmick et al., 2020).
Revised: BlogCatalog with 1/32 training parameters and 1/16 running time, and the precision on DBLP is more than 3 times higher than the best one of baselines i.e. LouvainNE (Bhowmick et al., 2020).

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Network Embedding.
Revised: Network Embedding.

Original: Here we mainly discuss the NE methods based on the language model word2vec
Revised: Here we mainly discuss the NE methods based on the language model word2vec

Original: (Mikolov et al., 2013), which are often designed for different types of networks, e.g. homogeneous networks (Perozzi et al., 2014; Grover & Leskovec, 2016; Tang et al., 2015), heterogeneous networks (Dong et al., 2017) and multiplex networks (Liu et al., 2017; Qu et al., 2017;
Revised: (Mikolov et al., 2013), which are often designed for different types of networks, e.g. homogeneous networks (Perozzi et al., 2014; Grover & Leskovec, 2016; Tang et al., 2015), heterogeneous networks (Dong et al., 2017) and multiplex networks (Liu et al., 2017; Qu et al., 2017;

Original: Xiong et al., 2021).
Revised: Xiong et al., 2021).

Original: It also often aims to preserve speciﬁc types of information, e.g. low-order proximity (Tang et al., 2015), structure similarity (Ribeiro et al., 2017), versatile similarity measures (Tsitsulin et al., 2018), and cross-network alignment relations (Du et al., 2022).
Revised: It also often aims to preserve speciﬁc types of information, e.g. low-order proximity (Tang et al., 2015), structure similarity (Ribeiro et al., 2017), versatile similarity measures (Tsitsulin et al., 2018), and cross-network alignment relations (Du et al., 2022).

Original: Apart from
Revised: Apart from

Original: NE methods derived from word2vec, methods based on deep neural networks (Wang et al., 2016) especially graph neural networks (Hamilton et al., 2017a) are also inﬂuential in literature, which, however, are technically divergent to this paper.
Revised: NE methods derived from word2vec, methods based on deep neural networks (Wang et al., 2016) especially graph neural networks (Hamilton et al., 2017a) are also inﬂuential in literature, which, however, are technically divergent to this paper.

Original: Tensorized Embedding Model.
Revised: Tensorized Embedding Model.

Original: Tensors are a generalization of vectors and matrices.
Revised: Tensors are a generalization of vectors and matrices.

Original: Thanks to the capability of representing and manipulating high-dimensional subjects, tensor has been successfully applied to exploit complex physical systems (Nielsen & Chuang, 2002).
Revised: Thanks to the capability of representing and manipulating high-dimensional subjects, tensor has been successfully applied to exploit complex physical systems (Nielsen & Chuang, 2002).

Original: The most crucial application is to understand the quantum many-body systems and to make an efﬁcient description of quantum states residing in the exponential growth of the Hilbert space (Bridgeman & Chubb, 2017).
Revised: The most crucial application is to understand the quantum many-body systems and to make an efﬁcient description of quantum states residing in the exponential growth of the Hilbert space (Bridgeman & Chubb, 2017).

Original: In addition, tensor networks composed of multiple tensors can also be used to understand many of the foundational results in quantum information (Or ´ us, 2019).
Revised: In addition, tensor networks composed of multiple tensors can also be used to understand many of the foundational results in quantum information (Or ´ us, 2019).

Original: Utilizing the tensor network structure makes it reasonably possible to simplify the concept like quantum teleportation, puriﬁcation, and the church of the larger Hilbert space (Bridgeman & Chubb, 2017).
Revised: Utilizing the tensor network structure makes it reasonably possible to simplify the concept like quantum teleportation, puriﬁcation, and the church of the larger Hilbert space (Bridgeman & Chubb, 2017).

Original: Inspired by this, many tensorized embedding models try to take advantage of tensor operations like contraction and decomposition to handle large volumes of data and compress parameters, ranging from image classiﬁcation (Stoudenmire & Schwab, 2016;
Revised: Inspired by this, many tensorized embedding models try to take advantage of tensor operations like contraction and decomposition to handle large volumes of data and compress parameters, ranging from image classiﬁcation (Stoudenmire & Schwab, 2016;

Original: Martyn et al., 2020; Selvan & Dam, 2020; Cheng et al., 2021) to natural language processing (Panahi et al., 2020; Ma et al., 2019; Liu et al., 2020; Qiu & Huang, 2015).
Revised: Martyn et al., 2020; Selvan & Dam, 2020; Cheng et al., 2021) to natural language processing (Panahi et al., 2020; Ma et al., 2019; Liu et al., 2020; Qiu & Huang, 2015).

Original: It is anticipated that the tensor concept introduced by quantum physics will give us new perspectives on designing the structure of deep learning models and comprehending the learning capability of artiﬁcial neural networks (Levine et al., 2017).
Revised: It is anticipated that the tensor concept introduced by quantum physics will give us new perspectives on designing the structure of deep learning models and comprehending the learning capability of artiﬁcial neural networks (Levine et al., 2017).

Original:  P RELIMINARIES AND B ACKGROUND 3. N ETWORK E MBEDDING B ASED ON N OISE C ONTRASTIVE E STIMATION
Revised:  P RELIMINARIES AND B ACKGROUND 3. N ETWORK E MBEDDING B ASED ON N OISE C ONTRASTIVE E STIMATION

Original: Given a network described by the node set V and the edge set E , node representation learning or network embedding aims to represent each node i ∈ V as a ﬁxed-length vector x i ∈ R p .
Revised: Given a network described by the node set V and the edge set E , node representation learning or network embedding aims to represent each node i ∈ V as a ﬁxed-length vector x i ∈ R p .

Original: To fully explore network structures for different purposes, a corpus D is usually created as the input data of embedding training.
Revised: To fully explore network structures for different purposes, a corpus D is usually created as the input data of embedding training.

Original: Typically, D can be the edge set (Tang et al., 2015), random walk sequences (Perozzi et al., 2014; Ribeiro et al., 2017; Du et al., 2022), etc.
Revised: Typically, D can be the edge set (Tang et al., 2015), random walk sequences (Perozzi et al., 2014; Ribeiro et al., 2017; Du et al., 2022), etc.

Original: Objective Function.
Revised: Objective Function.

Original: Initially, the objective is to maximize the following log softmax function:
Revised: Initially, the objective is to maximize the following log softmax function:

Original: [Equation]
Revised: [Equation]

Original: where the positive pairs ( i, j ) are sampled from D .
Revised: where the positive pairs ( i, j ) are sampled from D .

Original: However, Eq. 1 is computationally intractable for a large V , and thus surrogate objectives are proposed, including Hierarchical Softmax and Negative
Revised: However, Eq. 1 is computationally intractable for a large V , and thus surrogate objectives are proposed, including Hierarchical Softmax and Negative

Original: Sampling (Mikolov et al., 2013).
Revised: Sampling (Mikolov et al., 2013).

Original: In this paper, we only discuss a typical objective based on the widely adopted Negative Sampling:
Revised: In this paper, we only discuss a typical objective based on the widely adopted Negative Sampling:

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: where σ ( x ) = 1 / (1 + exp( − x )) is sigmoid function, K is the number of negative samples for each positive sampled pair, P n is the distribution of negative samples j (cid:48) which is usually set proportional to node degrees.
Revised: where σ ( x ) = 1 / (1 + exp( − x )) is sigmoid function, K is the number of negative samples for each positive sampled pair, P n is the distribution of negative samples j (cid:48) which is usually set proportional to node degrees.

Original: 3. Q UANTUM M ECHANICS AND T ENSORIZED E MBEDDING
Revised: 3. Q UANTUM M ECHANICS AND T ENSORIZED E MBEDDING

Original: Quantum Mechanics.
Revised: Quantum Mechanics.

Original: Quantum states are usually written in a ‘bra-ket’ notation.
Revised: Quantum states are usually written in a ‘bra-ket’ notation.

Original: In matrix representations, kets are column vectors denoted as | ψ (cid:105) ( where the name word/node2ket comes ) and bras are the conjugate transpose of the kets denoted as (cid:104) ψ | .
Revised: In matrix representations, kets are column vectors denoted as | ψ (cid:105) ( where the name word/node2ket comes ) and bras are the conjugate transpose of the kets denoted as (cid:104) ψ | .

Original: The state of a single quantum bit (qubit) | ψ (cid:105) is deﬁned in a 2-dimensional Hilbert space C 2 , described by a superposition of two orthonormal basis states | 0 (cid:105) =
Revised: The state of a single quantum bit (qubit) | ψ (cid:105) is deﬁned in a 2-dimensional Hilbert space C 2 , described by a superposition of two orthonormal basis states | 0 (cid:105) =

Original: (1 , 0) (cid:62) and | 1 (cid:105)
Revised: (1 , 0) (cid:62) and | 1 (cid:105)

Original: = (0 , 1) (cid:62) , | ψ (cid:105) =
Revised: = (0 , 1) (cid:62) , | ψ (cid:105) =

Original: α | 0 (cid:105) + β | 1 (cid:105) , where α, β ∈ C are probability amplitudes satisfying | α | 2 + | β | 2 = 1 .
Revised: α | 0 (cid:105) + β | 1 (cid:105) , where α, β ∈ C are probability amplitudes satisfying | α | 2 + | β | 2 = 1 .

Original: The state of an N -qubit quantum system is in the Hilbert space ⊗ Ni =1 C 2 , where ⊗ denotes tensor product.
Revised: The state of an N -qubit quantum system is in the Hilbert space ⊗ Ni =1 C 2 , where ⊗ denotes tensor product.

Original: Considering two quantum systems A and B with corresponding Hilbert spaces H A and H B , the composite space of A and B is H
Revised: Considering two quantum systems A and B with corresponding Hilbert spaces H A and H B , the composite space of A and B is H

Original: = H A ⊗ H B .
Revised: = H A ⊗ H B .

Original: If we use | φ (cid:105) A and | φ (cid:105) B to denote states of system A and B respectively, then a state | φ (cid:105) ∈ H that can be written as | φ (cid:105) =
Revised: If we use | φ (cid:105) A and | φ (cid:105) B to denote states of system A and B respectively, then a state | φ (cid:105) ∈ H that can be written as | φ (cid:105) =

Original: | φ (cid:105)
Revised: | φ (cid:105)

Original: A ⊗ | φ (cid:105) B is called a pure state , otherwise it is an entangled state .
Revised: A ⊗ | φ (cid:105) B is called a pure state , otherwise it is an entangled state .

Original: Tensorized Embedding by Word2ket.
Revised: Tensorized Embedding by Word2ket.

Original: Word2ket (Panahi et al., 2020) conducts word embedding by imitating behaviors of quantum systems.
Revised: Word2ket (Panahi et al., 2020) conducts word embedding by imitating behaviors of quantum systems.

Original: Embedding of each word is represented as an entangled quantum state in the Hilbert space, which is the summation of several pure state embeddings.
Revised: Embedding of each word is represented as an entangled quantum state in the Hilbert space, which is the summation of several pure state embeddings.

Original: A pure state embedding is formed by tensor product between some lower-dimensional tensors.
Revised: A pure state embedding is formed by tensor product between some lower-dimensional tensors.

Original: During the embedding procedure, some simpliﬁcations are adopted: i) real vectors are used as embeddings instead of complex ones in the quantum states; ii) the tensor product between tensors is implemented as the Kronecker product between vectors (it does not affect the results mathematically, since a vector can be deemed as a special tensor (Bridgeman & Chubb, 2016), and Kronecker product can be viewed as a form of vectorization of the tensor product); and iii) when imitating quantum entanglements, it uses learned tensors but not orthonormal basis states for implementation convenience, which means that a generated entangled state embedding may not be a strictly entangled one.
Revised: During the embedding procedure, some simpliﬁcations are adopted: i) real vectors are used as embeddings instead of complex ones in the quantum states; ii) the tensor product between tensors is implemented as the Kronecker product between vectors (it does not affect the results mathematically, since a vector can be deemed as a special tensor (Bridgeman & Chubb, 2016), and Kronecker product can be viewed as a form of vectorization of the tensor product); and iii) when imitating quantum entanglements, it uses learned tensors but not orthonormal basis states for implementation convenience, which means that a generated entangled state embedding may not be a strictly entangled one.

Original: These simpliﬁcations are also adopted by this paper.
Revised: These simpliﬁcations are also adopted by this paper.

Original: [Figure]
Revised: [Figure]

Original:  T ENSORIZED N ETWORK E MBEDDING
Revised:  T ENSORIZED N ETWORK E MBEDDING

Original: Sec. 4.1 ﬁrst introduces how we obtain embeddings from the proposed architecture Tensorized Embedding Blocks (TEBs).
Revised: Sec. 4.1 ﬁrst introduces how we obtain embeddings from the proposed architecture Tensorized Embedding Blocks (TEBs).

Original: In Sec.
Revised: In Sec.

Original: 4.2 we introduce how to train embeddings in TEBs efﬁciently.
Revised: 4.2 we introduce how to train embeddings in TEBs efﬁciently.

Original: Then in Sec. 4.3 and Sec.
Revised: Then in Sec. 4.3 and Sec.

Original: 4.4 we introduce the speciﬁc network embedding methods node2ket and node2ket+ respectively.
Revised: 4.4 we introduce the speciﬁc network embedding methods node2ket and node2ket+ respectively.

Original: Finally in Sec.
Revised: Finally in Sec.

Original: 4.5 we discuss the relation of the framework to other embedding methods.
Revised: 4.5 we discuss the relation of the framework to other embedding methods.

Original: 4. E MBEDDING VIA T ENSORIZED E MBEDDING B LOCKS
Revised: 4. E MBEDDING VIA T ENSORIZED E MBEDDING B LOCKS

Original: Tensorized Embedding Blocks (TEBs).
Revised: Tensorized Embedding Blocks (TEBs).

Original: Before introducing how node2ket works, we ﬁrst propose a novel embedding module named as Tensorized Embedding Blocks.
Revised: Before introducing how node2ket works, we ﬁrst propose a novel embedding module named as Tensorized Embedding Blocks.

Original: A TEB is composed of a certain number of Tensor Units (TUs), and each TU represents an embedding vector in R d .
Revised: A TEB is composed of a certain number of Tensor Units (TUs), and each TU represents an embedding vector in R d .

Original: A TEB has C columns where the c -th column has R c TUs.
Revised: A TEB has C columns where the c -th column has R c TUs.

Original: We denote the embedding vector of the TU located at the r -th row and the c -th column of the b -th block by u brc .
Revised: We denote the embedding vector of the TU located at the r -th row and the c -th column of the b -th block by u brc .

Original: Embeddings Generation.
Revised: Embeddings Generation.

Original: We use B TEBs for embedding generation.
Revised: We use B TEBs for embedding generation.

Original: Though more freedom can be added to the number of rows { R c } Cc =1 for different TEBs, we assume that they are the same without losing generality.
Revised: Though more freedom can be added to the number of rows { R c } Cc =1 for different TEBs, we assume that they are the same without losing generality.

Original: For node i , we obtain the embedding x
Revised: For node i , we obtain the embedding x

Original: i by:
Revised: i by:

Original: [Equation]
Revised: [Equation]

Original: where x b i is the pure state embedding in a R d C -dimensional Hilbert space given by the b -th TEB for node i , and w ib is the corresponding block weight of node i on the b -th TEB.
Revised: where x b i is the pure state embedding in a R d C -dimensional Hilbert space given by the b -th TEB for node i , and w ib is the corresponding block weight of node i on the b -th TEB.

Original: The r ( i, c ) in the subscription of TU embeddings u br ( i,c ) c indicates that the row index r depends on the node i and the column index c .
Revised: The r ( i, c ) in the subscription of TU embeddings u br ( i,c ) c indicates that the row index r depends on the node i and the column index c .

Original: For better understanding, we give a toy example of TEBs and how to generate x i according to the given r ( i, c ) look-up table as shown in Fig.
Revised: For better understanding, we give a toy example of TEBs and how to generate x i according to the given r ( i, c ) look-up table as shown in Fig.

Original: Complexity Analysis.
Revised: Complexity Analysis.

Original: There are three parts of parameters: i) the r ( i, c ) table has O ( C |V| ) parameters, ii) the embeddings of TUs have O ( Bd (cid:80) Cc =1 R c ) parameters, and iii) the block weights have
Revised: There are three parts of parameters: i) the r ( i, c ) table has O ( C |V| ) parameters, ii) the embeddings of TUs have O ( Bd (cid:80) Cc =1 R c ) parameters, and iii) the block weights have

Original: O ( B |V| ) parameters.
Revised: O ( B |V| ) parameters.

Original: The used parameters is totally O
Revised: The used parameters is totally O

Original: ( Bd (cid:80) Cc =1 R c + C |V| + B |V| ) .
Revised: ( Bd (cid:80) Cc =1 R c + C |V| + B |V| ) .

Original: For more detailed complexity analysis please refer to Sec.
Revised: For more detailed complexity analysis please refer to Sec.

Original: 4.3 and Sec. 4.4.
Revised: 4.3 and Sec. 4.4.

Original: 4. M ODEL L EARNING
Revised: 4. M ODEL L EARNING

Original: In this section, we show how to efﬁciently train the generated tensorized embedding by TEBs via updating embeddings directly according to computed gradients, without automatically forward/backward propagation as most DL models did.
Revised: In this section, we show how to efﬁciently train the generated tensorized embedding by TEBs via updating embeddings directly according to computed gradients, without automatically forward/backward propagation as most DL models did.

Original: In this way, we save the computation time/space from O ( d C ) to O ( dC ) , ﬁrstly making learning embeddings in the high-dimensional Hilbert space feasible.
Revised: In this way, we save the computation time/space from O ( d C ) to O ( dC ) , ﬁrstly making learning embeddings in the high-dimensional Hilbert space feasible.

Original: Details are put in Appendix C.2, including gradient derivation and the complete algorithm.
Revised: Details are put in Appendix C.2, including gradient derivation and the complete algorithm.

Original: Gradient.
Revised: Gradient.

Original: During the model optimization, we do not need to calculate the full node embeddings.
Revised: During the model optimization, we do not need to calculate the full node embeddings.

Original: Instead, we compute the gradient and update TU embeddings directly without a complete procedure of forward propagation.
Revised: Instead, we compute the gradient and update TU embeddings directly without a complete procedure of forward propagation.

Original: Given two embeddings x i and x j :
Revised: Given two embeddings x i and x j :

Original: [Equation]
Revised: [Equation]

Original: and the label l ( l = 1 for positive samples and l = 0 otherwise), we have the gradients for TU embeddings according to the objective Eq. 2:
Revised: and the label l ( l = 1 for positive samples and l = 0 otherwise), we have the gradients for TU embeddings according to the objective Eq. 2:

Original: [Equation]
Revised: [Equation]

Original: where 1 l is an indicator returning 1 if l = 1 and 0 otherwise.
Revised: where 1 l is an indicator returning 1 if l = 1 and 0 otherwise.

Original: The gradients for block weights are:
Revised: The gradients for block weights are:

Original: [Equation]
Revised: [Equation]

Original: = (cid:104)
Revised: = (cid:104)

Original: u br ( i,c ) c , u b (cid:48) r
Revised: u br ( i,c ) c , u b (cid:48) r

Original: ( j,c ) c (cid:105) .
Revised: ( j,c ) c (cid:105) .

Original: For detailed derivation of the gradients, please see Appendix C.1.
Revised: For detailed derivation of the gradients, please see Appendix C.1.

Original: Model Training.
Revised: Model Training.

Original: We optimize the objective via Asynchronous Stochastic Gradient Descent (Recht et al., 2011) on CPUs as most works in NCE-based embedding (Mikolov et al., 2013; Tang et al., 2015) and the famous open-source library gensim 1 do.
Revised: We optimize the objective via Asynchronous Stochastic Gradient Descent (Recht et al., 2011) on CPUs as most works in NCE-based embedding (Mikolov et al., 2013; Tang et al., 2015) and the famous open-source library gensim 1 do.

Original: Technical details are given in Appendix C.2.
Revised: Technical details are given in Appendix C.2.

Original:  N ODE 2 KET : A P RELIMINARY B ASELINE
Revised:  N ODE 2 KET : A P RELIMINARY B ASELINE

Original: For node2ket (or namely n2k), we set r ( i, c ) =
Revised: For node2ket (or namely n2k), we set r ( i, c ) =

Original: i as constants, and R 1 = R 2 = · · · = R C = |V| .
Revised: i as constants, and R 1 = R 2 = · · · = R C = |V| .

Original: It means we don’t need space to store the r ( i, c ) table.
Revised: It means we don’t need space to store the r ( i, c ) table.

Original: We give a constraint over the embedding dimension d in Assumption 1, according to which we can reach Theorem 1 where the ultimate compressive power is analyzed.
Revised: We give a constraint over the embedding dimension d in Assumption 1, according to which we can reach Theorem 1 where the ultimate compressive power is analyzed.

Original: The proof and further comments are given in Appendix D.
Revised: The proof and further comments are given in Appendix D.

Original: Assumption 1.
Revised: Assumption 1.

Original: [ Dimensional Assumption ]
Revised: [ Dimensional Assumption ]

Original: If a p -dimensional ﬁnal embedding is required, the dimension of generated embedding should not be smaller than p , i.e. d C ≥ p ;
Revised: If a p -dimensional ﬁnal embedding is required, the dimension of generated embedding should not be smaller than p , i.e. d C ≥ p ;

Original: Theorem 1.
Revised: Theorem 1.

Original: [ Compressive Power of Node2ket ] Given node set V and the ﬁnal embedding dimension p , when r ( i, c ) = i are constants (indicating R c = |V| , ∀ 1 ≤ c ≤ C ) and use block weights as trainable parameters, the total number of parameters is # params =
Revised: [ Compressive Power of Node2ket ] Given node set V and the ﬁnal embedding dimension p , when r ( i, c ) = i are constants (indicating R c = |V| , ∀ 1 ≤ c ≤ C ) and use block weights as trainable parameters, the total number of parameters is # params =

Original: B ( C |V| d + |V| ) .
Revised: B ( C |V| d + |V| ) .

Original: Omitting the integer constraints of d and C , we obtain the minimum number of used parameters of the whole embedding model as:
Revised: Omitting the integer constraints of d and C , we obtain the minimum number of used parameters of the whole embedding model as:

Original: [Equation]
Revised: [Equation]

Original: When the block weights are set as constants, # params =
Revised: When the block weights are set as constants, # params =

Original: BC |V| d , and we have:
Revised: BC |V| d , and we have:

Original: [Equation]
Revised: [Equation]

Original: ln p.
Revised: ln p.

Original: 4. N ODE 2 KET +: C OMPRESSIVE E MBEDDING U TILIZING G RAPH P ARTITION
Revised: 4. N ODE 2 KET +: C OMPRESSIVE E MBEDDING U TILIZING G RAPH P ARTITION

Original: To further compress node2ket, we propose node2ket+ (n2k+) utilizing graph partition techniques.
Revised: To further compress node2ket, we propose node2ket+ (n2k+) utilizing graph partition techniques.

Original: Speciﬁcally, we set r ( i, c ) =
Revised: Speciﬁcally, we set r ( i, c ) =

Original: Par c ( i ) where Par c ( · ) is the c -th partition algorithm and Par c ( i ) returns the index of the partition that node i belongs to.
Revised: Par c ( i ) where Par c ( · ) is the c -th partition algorithm and Par c ( i ) returns the index of the partition that node i belongs to.

Original: In this way, R c is equal to the number of partitions given by the function Par c .
Revised: In this way, R c is equal to the number of partitions given by the function Par c .

Original: In the implementation, we use Louvain partition (Blondel et al., 2008) with different settings of resolution as the graph partition algorithm, which is a scalable algorithm aiming to maximize the modularity of the partitions.
Revised: In the implementation, we use Louvain partition (Blondel et al., 2008) with different settings of resolution as the graph partition algorithm, which is a scalable algorithm aiming to maximize the modularity of the partitions.

Original: We control that R 1 ≤ R 2 ≤ · · · R C , hence the model can extract structure information on different levels of granularity.
Revised: We control that R 1 ≤ R 2 ≤ · · · R C , hence the model can extract structure information on different levels of granularity.

Original: Theoretically, we further give Assumption 2 over the size of TEBs, and then give a bound of the ultimate compressive power of node2ket+ in Theorem 2 based on Assumption 1 and 2.
Revised: Theoretically, we further give Assumption 2 over the size of TEBs, and then give a bound of the ultimate compressive power of node2ket+ in Theorem 2 based on Assumption 1 and 2.

Original: The proof for the following Theoremis presented in Appendix D by borrowing the tool of asymptotic analysis (Bender & Orszag, 2013), which is technically nontrivial and new to the network embedding community.
Revised: The proof for the following Theoremis presented in Appendix D by borrowing the tool of asymptotic analysis (Bender & Orszag, 2013), which is technically nontrivial and new to the network embedding community.

Original: Assumption 2.
Revised: Assumption 2.

Original: [ Combinatorial Assumption ] A TEB should be capable to generate at least |V| different pure state embeddings, which indicates (cid:81)
Revised: [ Combinatorial Assumption ] A TEB should be capable to generate at least |V| different pure state embeddings, which indicates (cid:81)

Original: Cc =1 R c ≥ |V| .
Revised: Cc =1 R c ≥ |V| .

Original: [Table]
Revised: [Table]

Original: Theorem 2.
Revised: Theorem 2.

Original: [ Compressive Power of Node2ket+ ]
Revised: [ Compressive Power of Node2ket+ ]

Original: Given the node set V and the ﬁnal embedding dimension p , when node2ket utilizes both the r ( i, c ) table and block weights, the total number of parameters is # params =
Revised: Given the node set V and the ﬁnal embedding dimension p , when node2ket utilizes both the r ( i, c ) table and block weights, the total number of parameters is # params =

Original: Bd (cid:80) Cc =1 R c + B |V| + C |V| .
Revised: Bd (cid:80) Cc =1 R c + B |V| + C |V| .

Original: We obtain the minimum number of used parameters of the whole embedding model:
Revised: We obtain the minimum number of used parameters of the whole embedding model:

Original: [Equation]
Revised: [Equation]

Original: where C ∗ =
Revised: where C ∗ =

Original: (cid:114) (cid:104) (ln p |V| ) 2 −
Revised: (cid:114) (cid:104) (ln p |V| ) 2 −

Original: [Equation]
Revised: [Equation]

Original: ln
Revised: ln

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: / (cid:0) |V|
Revised: / (cid:0) |V|

Original: [Equation]
Revised: [Equation]

Original: B + 1 (cid:1) .
Revised: B + 1 (cid:1) .

Original: When the block weights are set as constants, we have:
Revised: When the block weights are set as constants, we have:

Original: [Equation]
Revised: [Equation]

Original: 4. R ELATION TO O THER E MBEDDING M ETHODS
Revised: 4. R ELATION TO O THER E MBEDDING M ETHODS

Original: Relation to Conventional Embedding.
Revised: Relation to Conventional Embedding.

Original: When B = 1 , C = 1 , the embeddings of generated by node2ket become x i =
Revised: When B = 1 , C = 1 , the embeddings of generated by node2ket become x i =

Original: u 1 i 1 where no tensor product or entanglement is utilized and the embedding degrades to the conventional embedding.
Revised: u 1 i 1 where no tensor product or entanglement is utilized and the embedding degrades to the conventional embedding.

Original: The embedding look-up step in traditional NE methods becomes a special case in the corresponding step of node2ket though the training is different.
Revised: The embedding look-up step in traditional NE methods becomes a special case in the corresponding step of node2ket though the training is different.

Original: Advantages over Word2ket.
Revised: Advantages over Word2ket.

Original: A complete network embedding procedure basically has three steps: i) prepare the input data, ii) generate embedding for the input data, and iii) learn the embeddings.
Revised: A complete network embedding procedure basically has three steps: i) prepare the input data, ii) generate embedding for the input data, and iii) learn the embeddings.

Original: We analyze the advantages of node2ket over word2ket from the three aspects (details in Appendix
Revised: We analyze the advantages of node2ket over word2ket from the three aspects (details in Appendix

Original: B.1 and see summarization in Table 1).:
Revised: B.1 and see summarization in Table 1).:

Original: i) Node2ket can embed various explicitly structured data beyond sequence.
Revised: i) Node2ket can embed various explicitly structured data beyond sequence.

Original: It is achieved by deﬁning r ( i, c ) to ﬁt the structures of input data utilizing graph partition techniques to encode structures of data into TEBs (e.g. by setting r ( i, c )
Revised: It is achieved by deﬁning r ( i, c ) to ﬁt the structures of input data utilizing graph partition techniques to encode structures of data into TEBs (e.g. by setting r ( i, c )

Original: =
Revised: =

Original: r ( j, c ) if node i and j belong to the same community in node2ket+), while word2ket can not utilize inherent structures of input data.
Revised: r ( j, c ) if node i and j belong to the same community in node2ket+), while word2ket can not utilize inherent structures of input data.

Original: ii) Node2ket has better space efﬁciency due to the ﬂexible deﬁnition.
Revised: ii) Node2ket has better space efﬁciency due to the ﬂexible deﬁnition.

Original: Embedding generation modules of word2ket/word2ketXS are special cases of our proposed TEBs by setting w ib
Revised: Embedding generation modules of word2ket/word2ketXS are special cases of our proposed TEBs by setting w ib

Original: = constant and C = 2 n .
Revised: = constant and C = 2 n .

Original: The simpliﬁcation of word2ket reduces the representational ability of the embeddings and also limits the space efﬁciency of word2ket.
Revised: The simpliﬁcation of word2ket reduces the representational ability of the embeddings and also limits the space efﬁciency of word2ket.

Original: iii)
Revised: iii)

Original: Scalable learning by NCE without explicitly maintaining embedding x i .
Revised: Scalable learning by NCE without explicitly maintaining embedding x i .

Original: Node2ket is distributed trained based on the NCE objective Eq. 2.
Revised: Node2ket is distributed trained based on the NCE objective Eq. 2.

Original: During training, by iteratively updating the embedding u br ( i,c ) c , the process avoids computing the embedding x i and achieves to conduct embedding in the super high-dimensional Hilbert space.
Revised: During training, by iteratively updating the embedding u br ( i,c ) c , the process avoids computing the embedding x i and achieves to conduct embedding in the super high-dimensional Hilbert space.

Original: In comparison, Word2ket/word2ketXS train embeddings in a supervised way with an NN decoder, which requires computing the full embedding of x i and hence fails to do embedding in high-dimensional space.
Revised: In comparison, Word2ket/word2ketXS train embeddings in a supervised way with an NN decoder, which requires computing the full embedding of x i and hence fails to do embedding in high-dimensional space.

Original: Experimentally node2ket achieves
Revised: Experimentally node2ket achieves

Original: an embedding dimension of 2 32 (can be indeed more than 2 128 on our machine) while word2ket only reports an 8000 embedding dimension.
Revised: an embedding dimension of 2 32 (can be indeed more than 2 128 on our machine) while word2ket only reports an 8000 embedding dimension.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In experiments, we investigate downstream tasks including network reconstruction (Sec. 5.2), link prediction (Sec. 5.3), and node classiﬁcation (Appendix E.6).
Revised: In experiments, we investigate downstream tasks including network reconstruction (Sec. 5.2), link prediction (Sec. 5.3), and node classiﬁcation (Appendix E.6).

Original: In Sec. 5.4, we conduct ablation studies to investigate the roles of different parts in the model.
Revised: In Sec. 5.4, we conduct ablation studies to investigate the roles of different parts in the model.

Original: In Sec.5.5, good parallelizability of the methods is shown.
Revised: In Sec.5.5, good parallelizability of the methods is shown.

Original: And in Appendix E.5, we further display the overwhelming performance of our methods on network reconstruction by visualization of the reconstructed results.
Revised: And in Appendix E.5, we further display the overwhelming performance of our methods on network reconstruction by visualization of the reconstructed results.

Original: [Figure]
Revised: [Figure]

Original: 5. E XPERIMENTAL S ETTINGS
Revised: 5. E XPERIMENTAL S ETTINGS

Original: All of the experiments are run on a single machine with Intel(R) Core(TM) i9-7920X CPU @
Revised: All of the experiments are run on a single machine with Intel(R) Core(TM) i9-7920X CPU @

Original: 2.90GHz with 24 logical cores and 128GB memory.
Revised: 2.90GHz with 24 logical cores and 128GB memory.

Original: Datasets.
Revised: Datasets.

Original: We investigate ﬁve publicly available real-world datasets of different scales, different densities, and containing different types of information.
Revised: We investigate ﬁve publicly available real-world datasets of different scales, different densities, and containing different types of information.

Original: All of the networks are processed as undirected ones.
Revised: All of the networks are processed as undirected ones.

Original: The detailed statistics of the networks are given in Table 7 in Appendix E.2.
Revised: The detailed statistics of the networks are given in Table 7 in Appendix E.2.

Original: We give a brief introduction of the datasets as follows:
Revised: We give a brief introduction of the datasets as follows:

Original: YouTube (Tang & Liu, 2009b) is a social network with millions of users on youtube.com; BlogCatalog (Tang & Liu, 2009a) (BC) is a social network of bloggers who have social connections with each other; PPI (Breitkreutz et al., 2007) is a subgraph of the PPI network for Homo Sapiens; Arxiv GR-QC (Leskovec et al., 2007) is a collaboration network from arXiv and covers scientiﬁc collaborations between authors with papers submitted to
Revised: YouTube (Tang & Liu, 2009b) is a social network with millions of users on youtube.com; BlogCatalog (Tang & Liu, 2009a) (BC) is a social network of bloggers who have social connections with each other; PPI (Breitkreutz et al., 2007) is a subgraph of the PPI network for Homo Sapiens; Arxiv GR-QC (Leskovec et al., 2007) is a collaboration network from arXiv and covers scientiﬁc collaborations between authors with papers submitted to

Original: General Relativity and Quantum Cosmology category; DBLP (Ley, 2002) is a citation network of
Revised: General Relativity and Quantum Cosmology category; DBLP (Ley, 2002) is a citation network of

Original: DBLP, where each node denotes a publication, and each edge represents a citation.
Revised: DBLP, where each node denotes a publication, and each edge represents a citation.

Original: Baselines.
Revised: Baselines.

Original: The investigated NE methods include three based on NCE, two based on matrix factorization, and one based on community detection.
Revised: The investigated NE methods include three based on NCE, two based on matrix factorization, and one based on community detection.

Original: Here we introduce them brieﬂy: LINE (Tang et al., 2015) models ﬁrst-order proximity (connection strengths) and second-order proximity (similarity of neighborhoods) of given networks, and technically feed the adjacency matrix to two types of NCEbased objectives to learn node embeddings; node2vec (Grover & Leskovec, 2016) designs a biased random walk algorithm to simultaneously preserve Bread-First Search information and Depth-First
Revised: Here we introduce them brieﬂy: LINE (Tang et al., 2015) models ﬁrst-order proximity (connection strengths) and second-order proximity (similarity of neighborhoods) of given networks, and technically feed the adjacency matrix to two types of NCEbased objectives to learn node embeddings; node2vec (Grover & Leskovec, 2016) designs a biased random walk algorithm to simultaneously preserve Bread-First Search information and Depth-First

Original: Search information, and feed the random walk sequences to word2vec; VERSE (Tsitsulin et al., 2018) feeds node similarities (Personalized PageRank, Adajcency Similarity, and SimRank) to the
Revised: Search information, and feed the random walk sequences to word2vec; VERSE (Tsitsulin et al., 2018) feeds node similarities (Personalized PageRank, Adajcency Similarity, and SimRank) to the

Original: NCE-based objective; NetSMF (Qiu et al., 2019) obtains node embeddings by sparse matrix factorization on a sparsiﬁed dense random walk matrix; ProNE (Zhang et al., 2019) initializes node embeddings by sparse matrix factorization, and then enhances the embeddings via spectral propagation; LouvainNE (Bhowmick et al., 2020)
Revised: NCE-based objective; NetSMF (Qiu et al., 2019) obtains node embeddings by sparse matrix factorization on a sparsiﬁed dense random walk matrix; ProNE (Zhang et al., 2019) initializes node embeddings by sparse matrix factorization, and then enhances the embeddings via spectral propagation; LouvainNE (Bhowmick et al., 2020)

Original: ﬁrst adopts Louvain graph partition algorithm
Revised: ﬁrst adopts Louvain graph partition algorithm

Original: (Blondel et al., 2008) and then learn node embeddings from the partitions.
Revised: (Blondel et al., 2008) and then learn node embeddings from the partitions.

Original: HHNE (Zhang et al., 2021) trains hyperbolic node embeddings for heterogenous networks because hyperbolic geometry can reﬂect some properties of complex networks.
Revised: HHNE (Zhang et al., 2021) trains hyperbolic node embeddings for heterogenous networks because hyperbolic geometry can reﬂect some properties of complex networks.

Original: Parameter Settings.
Revised: Parameter Settings.

Original: For all methods that can set the number of threads (node2ket and baselines except ProNE and LouvainNE), we use 8 threads as the default.
Revised: For all methods that can set the number of threads (node2ket and baselines except ProNE and LouvainNE), we use 8 threads as the default.

Original: Embedding dimension is set to 128 as the default.
Revised: Embedding dimension is set to 128 as the default.

Original: For node2ket, we set the number of blocks B = 1 , number of negative samples
Revised: For node2ket, we set the number of blocks B = 1 , number of negative samples

Original: K = 1 by default.
Revised: K = 1 by default.

Original: For more details of parameter settings, please refer to Appendix E.3.
Revised: For more details of parameter settings, please refer to Appendix E.3.

Original: 5. E XPERIMENTS ON N ETWORK R ECONSTRUCTION
Revised: 5. E XPERIMENTS ON N ETWORK R ECONSTRUCTION

Original: Network reconstruction is used to measure an embedding model’s ability to preserve nodes’ origin local structure.
Revised: Network reconstruction is used to measure an embedding model’s ability to preserve nodes’ origin local structure.

Original: In line with (Wang et al., 2016), for baseline methods, we measure the score of a pair of node embeddings by negative Euclidean distance.
Revised: In line with (Wang et al., 2016), for baseline methods, we measure the score of a pair of node embeddings by negative Euclidean distance.

Original: While computing the Euclidean distance between two embeddings in the d C -dimensional Hilbert space is computationally intractable, we use the inner product instead.
Revised: While computing the Euclidean distance between two embeddings in the d C -dimensional Hilbert space is computationally intractable, we use the inner product instead.

Original: During the reconstruction process, we calculate the scores between
Revised: During the reconstruction process, we calculate the scores between

Original: C = C =
Revised: C = C =

Original: C =
Revised: C =

Original: P r e c
Revised: P r e c

Original: i s i o n ( % ) a) Performance w.r.t.
Revised: i s i o n ( % ) a) Performance w.r.t.

Original: B and C
Revised: B and C

Original: B=1 B=
Revised: B=1 B=

Original: B=
Revised: B=

Original: C = C =
Revised: C = C =

Original: C =

Revised: C =

Original: I m p r
Revised: I m p r

Original: o v e d
Revised: o v e d

Original: P r e c
Revised: P r e c

Original: i s i o n ( % ) ( p e r c e n t a g e p
Revised: i s i o n ( % ) ( p e r c e n t a g e p

Original: o i n t s ) b) Utilize block weights
Revised: o i n t s ) b) Utilize block weights

Original: B= B=
Revised: B= B=

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: n ( % ) c) Trade-off between C and d
Revised: n ( % ) c) Trade-off between C and d

Original: [Figure]
Revised: [Figure]

Original: QC
Revised: QC

Original: [Figure]
Revised: [Figure]

Original: BC DBLP
Revised: BC DBLP

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: all possible pairs of the embeddings, namely |V| ( |V|− 1) / 2 undirected node pairs.
Revised: all possible pairs of the embeddings, namely |V| ( |V|− 1) / 2 undirected node pairs.

Original: Then we sort the scores and select the top- |E| node pairs, denoted as E (cid:48) .
Revised: Then we sort the scores and select the top- |E| node pairs, denoted as E (cid:48) .

Original: Then the precision is deﬁned as Precision = |E
Revised: Then the precision is deﬁned as Precision = |E

Original: ∩ E (cid:48) | / |E| .
Revised: ∩ E (cid:48) | / |E| .

Original: For large-scale networks, we use Prec @ N to approximately measure the precision (for detailed explanation please see Appendix E.1).
Revised: For large-scale networks, we use Prec @ N to approximately measure the precision (for detailed explanation please see Appendix E.1).

Original: Evaluation on Medium-scale Networks:
Revised: Evaluation on Medium-scale Networks:

Original: We set the embedding dimension as 128 for all the baselines.
Revised: We set the embedding dimension as 128 for all the baselines.

Original: For node2ket, we set B = 1 , C = 8 , and d = 16 , which means that to store the embeddings, the number of used parameters for each node is BCd = 128 , the same as the baselines.
Revised: For node2ket, we set B = 1 , C = 8 , and d = 16 , which means that to store the embeddings, the number of used parameters for each node is BCd = 128 , the same as the baselines.

Original: For node2ket+, we also set B = 1 , C = 8 , and d = 16 .
Revised: For node2ket+, we also set B = 1 , C = 8 , and d = 16 .

Original: Note that node2ket+ decreases the number of
Revised: Note that node2ket+ decreases the number of

Original: TUs but brings space consumption of the r ( i, c ) table, so the average consumed space to store embedding for each node may not be equal to 128.
Revised: TUs but brings space consumption of the r ( i, c ) table, so the average consumed space to store embedding for each node may not be equal to 128.

Original: The experimental results are given in Table 2, which shows that i) both node2ket and node2ket+ achieve the precision that conventional embedding methods can hardly reach by efforts in feature engineering as most works do; ii) even after decreasing the number of TUs, node2ket+’s performance does not decrease signiﬁcantly compared with node2ket, indicating that further compression without sacriﬁcing performance is still possible.
Revised: The experimental results are given in Table 2, which shows that i) both node2ket and node2ket+ achieve the precision that conventional embedding methods can hardly reach by efforts in feature engineering as most works do; ii) even after decreasing the number of TUs, node2ket+’s performance does not decrease signiﬁcantly compared with node2ket, indicating that further compression without sacriﬁcing performance is still possible.

Original: Evaluation on Large-scale Network (YouTube): We only evaluate baselines that are scalable and could be run on our machine.
Revised: Evaluation on Large-scale Network (YouTube): We only evaluate baselines that are scalable and could be run on our machine.

Original: We set the embedding dimension as 4 for baselines.
Revised: We set the embedding dimension as 4 for baselines.

Original: Since node2ket fails to generate embeddings because it requires TU dimensions d ≥ 4 and C ≥ 2 , we only evaluate node2ket+, whose used space to store embeddings is a little less than baselines (compressive ratio=0.99).
Revised: Since node2ket fails to generate embeddings because it requires TU dimensions d ≥ 4 and C ≥ 2 , we only evaluate node2ket+, whose used space to store embeddings is a little less than baselines (compressive ratio=0.99).

Original: Results are given in Table 3, showing that node2ket+ outperforms baselines mostly and also converges faster compared with other NCE-based embedding methods.
Revised: Results are given in Table 3, showing that node2ket+ outperforms baselines mostly and also converges faster compared with other NCE-based embedding methods.

Original: Efﬁciency Evaluation and Scalability.
Revised: Efﬁciency Evaluation and Scalability.

Original: We conduct the experiments on BlogCatalog.
Revised: We conduct the experiments on BlogCatalog.

Original: Two questions are raised: to outperform baselines, i) how many training parameters does node2ket need? and ii) how much time does node2ket take?
Revised: Two questions are raised: to outperform baselines, i) how many training parameters does node2ket need? and ii) how much time does node2ket take?

Original: Answers are in Fig.
Revised: Answers are in Fig.

Original: 2 – node2ket achieves to outperforms the best baselines with 1/64 training parameters in the evaluation of required training parameters, and with 1/32 training parameters and 1/16 running time in the evaluation of running time.
Revised: 2 – node2ket achieves to outperforms the best baselines with 1/64 training parameters in the evaluation of required training parameters, and with 1/32 training parameters and 1/16 running time in the evaluation of running time.

Original: The results show both the efﬁciency on space saving and time saving.
Revised: The results show both the efﬁciency on space saving and time saving.

Original: The scalability has been proved in the experiments on YouTube (Table 3), where it takes about two minutes for the running of node2ket+ on a network of 1 million nodes with 8 CPU threads.
Revised: The scalability has been proved in the experiments on YouTube (Table 3), where it takes about two minutes for the running of node2ket+ on a network of 1 million nodes with 8 CPU threads.

Original: 5. E XPERIMENTS ON L INK P REDICTION
Revised: 5. E XPERIMENTS ON L INK P REDICTION

Original: We randomly mask 1% edges of the given network, and create the same amount of edges as noise edges by randomly connecting nodes in the network.
Revised: We randomly mask 1% edges of the given network, and create the same amount of edges as noise edges by randomly connecting nodes in the network.

Original: Then, we conduct network embedding on the network where masked edges are removed, and predict the masked edges from the mixture of masked edges and noise edges.
Revised: Then, we conduct network embedding on the network where masked edges are removed, and predict the masked edges from the mixture of masked edges and noise edges.

Original: The used metric is the same as network reconstruction (Sec. 5.2).
Revised: The used metric is the same as network reconstruction (Sec. 5.2).

Original: Experimental results in Table 4 show that node2ket outperforms all the baselines consistently.
Revised: Experimental results in Table 4 show that node2ket outperforms all the baselines consistently.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: A BLATION S TUDY
Revised: A BLATION S TUDY

Original: We conduct ablation studies on node2ket, while the conclusions are the same for node2ket+.
Revised: We conduct ablation studies on node2ket, while the conclusions are the same for node2ket+.

Original: Number of TEBs B and Number of Columns C .
Revised: Number of TEBs B and Number of Columns C .

Original: Experiments of network reconstruction are conducted on GR-QC.
Revised: Experiments of network reconstruction are conducted on GR-QC.

Original: We set w ib = 1 / √ B as constants, d =
Revised: We set w ib = 1 / √ B as constants, d =

Original: [Table]
Revised: [Table]

Original: and run node2ket with C = 2 , 4 ,and B = 1 , 2 , 4 .
Revised: and run node2ket with C = 2 , 4 ,and B = 1 , 2 , 4 .

Original: Results are shown in Fig.
Revised: Results are shown in Fig.

Original: 3 a), which shows that model performance improved when C or B increases when d is small.
Revised: 3 a), which shows that model performance improved when C or B increases when d is small.

Original: However, we also observe that when d is larger, increasing
Revised: However, we also observe that when d is larger, increasing

Original: B will have a negative inﬂuence on model performance.
Revised: B will have a negative inﬂuence on model performance.

Original: Use Block Weights w ib as Trainable Parameters.
Revised: Use Block Weights w ib as Trainable Parameters.

Original: Experiments of network reconstruction are conducted on GR-QC.
Revised: Experiments of network reconstruction are conducted on GR-QC.

Original: We set d = 4 , and run node2ket with C = 2 , 4 , 8 and B = 2 , 4 .
Revised: We set d = 4 , and run node2ket with C = 2 , 4 , 8 and B = 2 , 4 .

Original: In Fig.
Revised: In Fig.

Original: 3 b) we show the improved precision compared with setting w ib as constants, which proves the positive roles of w ib in improving model’s expressiveness when d is small.
Revised: 3 b) we show the improved precision compared with setting w ib as constants, which proves the positive roles of w ib in improving model’s expressiveness when d is small.

Original: However, when d is larger, trainable w ib is likely to make a negative inﬂuence on model performance.
Revised: However, when d is larger, trainable w ib is likely to make a negative inﬂuence on model performance.

Original: The experimental results for the above two studies when d = 8 are in Table 8 in Appendix E.4.
Revised: The experimental results for the above two studies when d = 8 are in Table 8 in Appendix E.4.

Original: Trade-off between Number of Columns C and Embedding Dimension d .
Revised: Trade-off between Number of Columns C and Embedding Dimension d .

Original: Experiments of network reconstruction are conducted on all the medium-scale datasets.
Revised: Experiments of network reconstruction are conducted on all the medium-scale datasets.

Original: We ﬁx Cd = 64 and run node2ket with C = 1 , 2 , 4 , 8 , 16 , 32 , and plot the results in Fig.
Revised: We ﬁx Cd = 64 and run node2ket with C = 1 , 2 , 4 , 8 , 16 , 32 , and plot the results in Fig.

Original: 3 c).
Revised: 3 c).

Original: We see that i) when
Revised: We see that i) when

Original: C = 1 , i.e. the tensorized embedding model is degraded to the conventional embedding model, the model performance decreases, and ii) when d ≤ 4 , the performance is also negatively inﬂuenced.
Revised: C = 1 , i.e. the tensorized embedding model is degraded to the conventional embedding model, the model performance decreases, and ii) when d ≤ 4 , the performance is also negatively inﬂuenced.

Original: 5. P ARALLELIZABILITY
Revised: 5. P ARALLELIZABILITY

Original: [Figure]
Revised: [Figure]

Original: We evaluate node2ket on DBLP.
Revised: We evaluate node2ket on DBLP.

Original: We set C = 8 , d = 8 , and run the same number of SGD iterations with the thread number varying from 1 to 16.
Revised: We set C = 8 , d = 8 , and run the same number of SGD iterations with the thread number varying from 1 to 16.

Original: We record the running time and precision as shown in Fig.
Revised: We record the running time and precision as shown in Fig.

Original: The results show that i) the model performance is stable under the setting of different numbers of threads, and ii) the running time t almost decrease linearly w.r.t.
Revised: The results show that i) the model performance is stable under the setting of different numbers of threads, and ii) the running time t almost decrease linearly w.r.t.

Original: the number of threads when the number of threads is not larger than 16.
Revised: the number of threads when the number of threads is not larger than 16.

Original:  C ONCLUSIONS AND F UTURE W ORK
Revised:  C ONCLUSIONS AND F UTURE W ORK

Original: We have proposed the network embedding methods node2ket and node2ket +, by training the embedding module
Revised: We have proposed the network embedding methods node2ket and node2ket +, by training the embedding module

Original: Tensorized Embedding Blocks with an objective based on Noise Contrastive Estimation.
Revised: Tensorized Embedding Blocks with an objective based on Noise Contrastive Estimation.

Original: Limitation & future works.
Revised: Limitation & future works.

Original: With the (possible) arrival of Noisy Intermediate Scale Quantum (NISQ) era (Preskill, 2018), the deployment of learning on quantum devices is becoming an everubiquitous research topic (Cong et al., 2019).
Revised: With the (possible) arrival of Noisy Intermediate Scale Quantum (NISQ) era (Preskill, 2018), the deployment of learning on quantum devices is becoming an everubiquitous research topic (Cong et al., 2019).

Original: Our method can be explored for its implementation in a more quantum-friendly manner beyond the current quantum-inspired one, e.g. by variational quantum circuits (Cerezo et al., 2021), so that it will be more feasible on quantum devices.
Revised: Our method can be explored for its implementation in a more quantum-friendly manner beyond the current quantum-inspired one, e.g. by variational quantum circuits (Cerezo et al., 2021), so that it will be more feasible on quantum devices.

Original: A A NALYSIS OF S PACE TO S TORE /T RAIN E MBEDDINGS
Revised: A A NALYSIS OF S PACE TO S TORE /T RAIN E MBEDDINGS

Original: NCE-based NE methods.
Revised: NCE-based NE methods.

Original: In general, there are two most widely adopted objectives used in network embedding (Armandpour et al., 2019; Xiong et al., 2022):
Revised: In general, there are two most widely adopted objectives used in network embedding (Armandpour et al., 2019; Xiong et al., 2022):

Original: NCE-x: (cid:88)
Revised: NCE-x: (cid:88)

Original: [Equation]
Revised: [Equation]

Original: log
Revised: log

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: K (cid:88) k
Revised: K (cid:88) k

Original: =
Revised: =

Original: E j (cid:48) ∼ P n log σ ( − x (cid:62) i x j (cid:48) )
Revised: E j (cid:48) ∼ P n log σ ( − x (cid:62) i x j (cid:48) )

Original: (cid:105) , (8) and
Revised: (cid:105) , (8) and

Original: NCE-h: (cid:88) ( i,j ) ∈D (cid:104) log σ ( x (cid:62) i h j )
Revised: NCE-h: (cid:88) ( i,j ) ∈D (cid:104) log σ ( x (cid:62) i h j )

Original: +
Revised: +

Original: K (cid:88) k
Revised: K (cid:88) k

Original: =
Revised: =

Original: E j (cid:48) ∼ P n log σ ( − x (cid:62) i
Revised: E j (cid:48) ∼ P n log σ ( − x (cid:62) i

Original: h j (cid:48) )
Revised: h j (cid:48) )

Original: (cid:105) .
Revised: (cid:105) .

Original: The NCE-x objective tends to make positively sampled nodes close in the embedding space (when
Revised: The NCE-x objective tends to make positively sampled nodes close in the embedding space (when

Original: D is the edge set, it means ﬁrst-order similarity (Tang et al., 2015)), while the NCE-h objective tends to make nodes with common positive samples close in the embedding space (when D is the edge set, it means second-order similarity (Tang et al., 2015)).
Revised: D is the edge set, it means ﬁrst-order similarity (Tang et al., 2015)), while the NCE-h objective tends to make nodes with common positive samples close in the embedding space (when D is the edge set, it means second-order similarity (Tang et al., 2015)).

Original: One can see that mathematically NCE-h further utilizes hidden embedding h compared with NCE-x, and hence NCE-h double the number of training parameters compared with NCE-x. We list the number of parameters of the baseline methods in Table 5.
Revised: One can see that mathematically NCE-h further utilizes hidden embedding h compared with NCE-x, and hence NCE-h double the number of training parameters compared with NCE-x. We list the number of parameters of the baseline methods in Table 5.

Original: [Table]
Revised: [Table]

Original: NCE
Revised: NCE

Original: [Equation]
Revised: [Equation]

Original: Node2ket.
Revised: Node2ket.

Original: In this paper, our proposed node2ket adopts NCE-x as the objective, which, however, can be easily substituted with NCE-h by simply doubling the number of TEBs.
Revised: In this paper, our proposed node2ket adopts NCE-x as the objective, which, however, can be easily substituted with NCE-h by simply doubling the number of TEBs.

Original: We list the used parameters of different parts of node2ket in Table 6.
Revised: We list the used parameters of different parts of node2ket in Table 6.

Original: Because of its NCE-x core, the numbers of parameters used to train/store embeddings are the same.
Revised: Because of its NCE-x core, the numbers of parameters used to train/store embeddings are the same.

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: B R ELATIONS TO W ORD 2 KET AND W ORD 2 KET XS B. U NIFYING W ORD 2 KET AND W ORD 2 KET XS BY TEB S
Revised: B R ELATIONS TO W ORD 2 KET AND W ORD 2 KET XS B. U NIFYING W ORD 2 KET AND W ORD 2 KET XS BY TEB S

Original: Here we provide detailed explanation on how to use the language of Tensorized Embedding Blocks to describe word2ket and word2ketXS.
Revised: Here we provide detailed explanation on how to use the language of Tensorized Embedding Blocks to describe word2ket and word2ketXS.

Original: Word2ket generates embedding x i by:
Revised: Word2ket generates embedding x i by:

Original: [Equation]
Revised: [Equation]

Original: where r is the rank and n is the order.
Revised: where r is the rank and n is the order.

Original: Comparing Eq.
Revised: Comparing Eq.

Original: 10 with the embedding generated by Eq.
Revised: 10 with the embedding generated by Eq.

Original: 3, one can see that in the language of Tensorized Embedding Blocks, the rank is equal to the block number B , the order is equal to the column number C , with block weights w ib = 1 set as constants.
Revised: 3, one can see that in the language of Tensorized Embedding Blocks, the rank is equal to the block number B , the order is equal to the column number C , with block weights w ib = 1 set as constants.

Original: Though very similar to the way node2ket generates embeddings, it is worth noting that word2ket requires that n = 2 y because the embeddings are generated from leaf nodes of a complete binary tree thus computation of ﬁnal embeddings can be speeded up, while in comparison, the column number C of node2ket can be any integer no less than 2 .
Revised: Though very similar to the way node2ket generates embeddings, it is worth noting that word2ket requires that n = 2 y because the embeddings are generated from leaf nodes of a complete binary tree thus computation of ﬁnal embeddings can be speeded up, while in comparison, the column number C of node2ket can be any integer no less than 2 .

Original: Word2ketXS generates embedding from linear operators.
Revised: Word2ketXS generates embedding from linear operators.

Original: The embeddings are written as:
Revised: The embeddings are written as:

Original: [Equation]
Revised: [Equation]

Original: where n , r has the same meaning as they are in the word2ket, X is the word embedding linear operator which can be understood as the embedding matrix of shape p × |V| .
Revised: where n , r has the same meaning as they are in the word2ket, X is the word embedding linear operator which can be understood as the embedding matrix of shape p × |V| .

Original: We write X =
Revised: We write X =

Original: { x i } |V|− 1 i = where x i is i -th column of X and also the embedding of node i .
Revised: { x i } |V|− 1 i = where x i is i -th column of X and also the embedding of node i .

Original: U kj is also a linear operator which can be understood as a matrix of shape q × t ( q n = p and t n = |V| ), and we write it as a set of column vectors U kj =
Revised: U kj is also a linear operator which can be understood as a matrix of shape q × t ( q n = p and t n = |V| ), and we write it as a set of column vectors U kj =

Original: { u kvj } t − 1 v =0 .
Revised: { u kvj } t − 1 v =0 .

Original: We deﬁne X k
Revised: We deﬁne X k

Original: = { x ki } |V|− 1 i
Revised: = { x ki } |V|− 1 i

Original: =
Revised: =

Original: =
Revised: =

Original: (cid:78) nj =1 U kj .
Revised: (cid:78) nj =1 U kj .

Original: By the property of Kronecker product in Lemma 1 whose proof is obvious and omitted, we can ﬁnd that:
Revised: By the property of Kronecker product in Lemma 1 whose proof is obvious and omitted, we can ﬁnd that:

Original: [Equation]
Revised: [Equation]

Original: and by X = (cid:80) rk =1 X b , we have:
Revised: and by X = (cid:80) rk =1 X b , we have:

Original: [Equation]
Revised: [Equation]

Original: So far we have achieved writing the embedding of word2ketXS in the language of TEBs.
Revised: So far we have achieved writing the embedding of word2ketXS in the language of TEBs.

Original: It is also worth noting that word2ketXS requires that n = 2 y .
Revised: It is also worth noting that word2ketXS requires that n = 2 y .

Original: The reason is the same as word2ket.
Revised: The reason is the same as word2ket.

Original: Lemma 1 ( Property of Kronecker Product ) .
Revised: Lemma 1 ( Property of Kronecker Product ) .

Original: Given two matrix A p × q = { a 0 , a 1 , · · · , a q − 1 } , and
Revised: Given two matrix A p × q = { a 0 , a 1 , · · · , a q − 1 } , and

Original: B m × n = { b 0 , b 1 , · · · , b n − 1 } , suppose C is the Kronecker product of A and B ,
Revised: B m × n = { b 0 , b 1 , · · · , b n − 1 } , suppose C is the Kronecker product of A and B ,

Original: i.e. C =
Revised: i.e. C =

Original: A ⊗ B , then C pm × qn = { c 0 , c 1 , · · · , c qn − 1 } , where c sm + t = a s ⊗ b t .
Revised: A ⊗ B , then C pm × qn = { c 0 , c 1 , · · · , c qn − 1 } , where c sm + t = a s ⊗ b t .

Original: B. T ECHNICAL D IFFERENCES BETWEEN N ODE 2 KET /N ODE 2 KET + AND
Revised: B. T ECHNICAL D IFFERENCES BETWEEN N ODE 2 KET /N ODE 2 KET + AND

Original: [Figure]
Revised: [Figure]

Original: ORD
Revised: ORD

Original: [Figure]
Revised: [Figure]

Original: KET
Revised: KET

Original: [Figure]
Revised: [Figure]

Original: W
Revised: W

Original: ORD
Revised: ORD

Original: [Figure]
Revised: [Figure]

Original: The differences are summarized in Table 1 and Fig.
Revised: The differences are summarized in Table 1 and Fig.

Original: 5, which are mainly from the following three aspects: i)
Revised: 5, which are mainly from the following three aspects: i)

Original: The embedding generation.
Revised: The embedding generation.

Original: We have shown that both the embedding generation modules of word2ket/word2ketXS are special cases of TEBs in Appendix B.1, and know that the differences of word2ket and word2ketXS lie in the shape of TEBs ( B , C , and { R c } Cc =1 ) and the deﬁnition of r ( i, c ) indices.
Revised: We have shown that both the embedding generation modules of word2ket/word2ketXS are special cases of TEBs in Appendix B.1, and know that the differences of word2ket and word2ketXS lie in the shape of TEBs ( B , C , and { R c } Cc =1 ) and the deﬁnition of r ( i, c ) indices.

Original: Then we can ﬁnd that the differences between the embedding generation models of word2ket/word2ketXS and node2ket/node2ket+ are mainly whether block weights w ib are adopted and whether the shape of TEBs is ﬂexible to be re-designed by re-deﬁning r ( i, c ) .
Revised: Then we can ﬁnd that the differences between the embedding generation models of word2ket/word2ketXS and node2ket/node2ket+ are mainly whether block weights w ib are adopted and whether the shape of TEBs is ﬂexible to be re-designed by re-deﬁning r ( i, c ) .

Original: The adoption of block weights is proved to improve model expressiveness when d is small (Fig. 3 b)), and a ﬂexible shape of TEBs is promised to make the embedding model more compressive thus improving space efﬁciency.
Revised: The adoption of block weights is proved to improve model expressiveness when d is small (Fig. 3 b)), and a ﬂexible shape of TEBs is promised to make the embedding model more compressive thus improving space efﬁciency.

Original: ii)
Revised: ii)

Original: The embedding learning phase.
Revised: The embedding learning phase.

Original: Word2ket/word2ketXS train embeddings in a supervised way with an NN decoder, while node2ket/node2ket+ train embeddings by NCE (contrastive learning) which mainly cares about the inner product between the embeddings.
Revised: Word2ket/word2ketXS train embeddings in a supervised way with an NN decoder, while node2ket/node2ket+ train embeddings by NCE (contrastive learning) which mainly cares about the inner product between the embeddings.

Original: In this way, full embeddings are required for word2ket/word2ketXS to feed into the decoder but are not a must for node2ket/node2ket+.
Revised: In this way, full embeddings are required for word2ket/word2ketXS to feed into the decoder but are not a must for node2ket/node2ket+.

Original: Therefore, node2ket/node2ket+ do not need further memory for the full embedding and manage to perform embedding in the Hilbert space of a very large dimension ( 2 32 in experiments, which can be indeed more than 2 128 on our machine).
Revised: Therefore, node2ket/node2ket+ do not need further memory for the full embedding and manage to perform embedding in the Hilbert space of a very large dimension ( 2 32 in experiments, which can be indeed more than 2 128 on our machine).

Original: iii)
Revised: iii)

Original: The input data.
Revised: The input data.

Original: Word2ket accepts sentences as the input data, which is a kind of unstructured data.
Revised: Word2ket accepts sentences as the input data, which is a kind of unstructured data.

Original: In comparison, node2ket is designed to process highly structured networks, with an effort on reshaping TEBs to improve both model expressiveness and space efﬁciency.
Revised: In comparison, node2ket is designed to process highly structured networks, with an effort on reshaping TEBs to improve both model expressiveness and space efﬁciency.

Original: [Equation]
Revised: [Equation]

Original: C. G RADIENT D ERIVATION
Revised: C. G RADIENT D ERIVATION

Original: We provide detailed gradient derivation of Eq.
Revised: We provide detailed gradient derivation of Eq.

Original: 5 and Eq.
Revised: 5 and Eq.

Original: 6 here.
Revised: 6 here.

Original: Given two embeddings:
Revised: Given two embeddings:

Original: [Equation]
Revised: [Equation]

Original: we have the inner product:
Revised: we have the inner product:

Original: [Equation]
Revised: [Equation]

Original: 15) from which we have the partial derivative:
Revised: 15) from which we have the partial derivative:

Original: [Equation]
Revised: [Equation]

Original: When ( i, j ) is a positive sample with label l = 1 , the objective is log σ (cid:104) x
Revised: When ( i, j ) is a positive sample with label l = 1 , the objective is log σ (cid:104) x

Original: i , x j (cid:105) , and when it is a negative sample with label l = 0 , the objective becomes log σ ( −(cid:104) x
Revised: i , x j (cid:105) , and when it is a negative sample with label l = 0 , the objective becomes log σ ( −(cid:104) x

Original: i , x j (cid:105) ) .
Revised: i , x j (cid:105) ) .

Original: We deﬁne an indicator function 1 l which takes value 1 if l = 1 and 0 if l = 0 , then the objective can be written as log σ (2( 1 l − 0 .
Revised: We deﬁne an indicator function 1 l which takes value 1 if l = 1 and 0 if l = 0 , then the objective can be written as log σ (2( 1 l − 0 .

Original: 5) (cid:104) x i , x j (cid:105) ) .
Revised: 5) (cid:104) x i , x j (cid:105) ) .

Original: Then we have the following formula with Eq. 16:
Revised: Then we have the following formula with Eq. 16:

Original: [Equation]
Revised: [Equation]

Original: which is the result of Eq.
Revised: which is the result of Eq.

Original: For the gradient derivation of block weights w ib , Then we have the following formula with Eq. 17:
Revised: For the gradient derivation of block weights w ib , Then we have the following formula with Eq. 17:

Original: [Equation]
Revised: [Equation]

Original: , x j (cid:105) )
Revised: , x j (cid:105) )

Original: ∂w ib = (cid:2)
Revised: ∂w ib = (cid:2)

Original: [Equation]
Revised: [Equation]

Original: l − σ ( (cid:104) x
Revised: l − σ ( (cid:104) x

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: x j (cid:105) )
Revised: x j (cid:105) )

Original: (cid:3) ∂ (cid:104) x i , x j (cid:105)
Revised: (cid:3) ∂ (cid:104) x i , x j (cid:105)

Original: ∂w ib = (cid:2) 1 l − σ ( (cid:104) x i , x j (cid:105) )
Revised: ∂w ib = (cid:2) 1 l − σ ( (cid:104) x i , x j (cid:105) )

Original: (cid:3)
Revised: (cid:3)

Original: B (cid:88) b (cid:48)
Revised: B (cid:88) b (cid:48)

Original: =
Revised: =

Original: w jb (cid:48)
Revised: w jb (cid:48)

Original: C (cid:89) c
Revised: C (cid:89) c

Original: = (cid:104) u br ( i,c ) c , u b (cid:48) r
Revised: = (cid:104) u br ( i,c ) c , u b (cid:48) r

Original: ( j,c ) c (cid:105) , (19) which is the result of Eq.
Revised: ( j,c ) c (cid:105) , (19) which is the result of Eq.

Original: C. M ODEL L EARNING
Revised: C. M ODEL L EARNING

Original: We give the pseudo-code of a thread of ASGD in Algorithm 1 and give and illustration of how distributed learning by ASGD is implemented in Fig.
Revised: We give the pseudo-code of a thread of ASGD in Algorithm 1 and give and illustration of how distributed learning by ASGD is implemented in Fig.

Original: The details of embedding regularization and normalization are in Appendix C.2.1.
Revised: The details of embedding regularization and normalization are in Appendix C.2.1.

Original: [Figure]
Revised: [Figure]

Original: C.2. E MBEDDING R EGULARIZATION AND N ORMALIZATION FOR T ENSOR U NITS
Revised: C.2. E MBEDDING R EGULARIZATION AND N ORMALIZATION FOR T ENSOR U NITS

Original: The multi-column architecture of the Tensorized Embedding Blocks brings difﬁculties to embedding training.
Revised: The multi-column architecture of the Tensorized Embedding Blocks brings difﬁculties to embedding training.

Original: Following unexpected results are likely to happen without proper normalization: i) embeddings converge to zero vectors or vectors of inﬁnite length, ii) embeddings converge to the same
Revised: Following unexpected results are likely to happen without proper normalization: i) embeddings converge to zero vectors or vectors of inﬁnite length, ii) embeddings converge to the same

Original: ; vector.
Revised: ; vector.

Original: We adopts embedding regularization and normalization in the implementation to improve the robustness of the algorithm, as shown in Fig.
Revised: We adopts embedding regularization and normalization in the implementation to improve the robustness of the algorithm, as shown in Fig.

Original: Regularization.
Revised: Regularization.

Original: (Fig.
Revised: (Fig.

Original: 7 a))
Revised: 7 a))

Original: Distance penalty terms can be added to positive node pairs to accelerate model convergence and improve the ability to preserve node proximities (Xiong et al., 2022).
Revised: Distance penalty terms can be added to positive node pairs to accelerate model convergence and improve the ability to preserve node proximities (Xiong et al., 2022).

Original: In the implementation, we adopt Wasserstein-3 distance regularizaton term over TU embeddings of positive node pairs.
Revised: In the implementation, we adopt Wasserstein-3 distance regularizaton term over TU embeddings of positive node pairs.

Original: Normalization-1.
Revised: Normalization-1.

Original: (Fig.
Revised: (Fig.

Original: 7 b))
Revised: 7 b))

Original: To increase the stability of embedding training, we make TU embeddings constrained by (cid:107) u rc (cid:107) = 1 .
Revised: To increase the stability of embedding training, we make TU embeddings constrained by (cid:107) u rc (cid:107) = 1 .

Original: In this way, TU embeddings won’t converge to vectors of zero or inﬁnite length.
Revised: In this way, TU embeddings won’t converge to vectors of zero or inﬁnite length.

Original: Normalization-2.
Revised: Normalization-2.

Original: (Fig.
Revised: (Fig.

Original: 7 c))
Revised: 7 c))

Original: During training, we move the axis towards − 1 √ d 1 for a step.
Revised: During training, we move the axis towards − 1 √ d 1 for a step.

Original: In this way, most of the pairs of embeddings on the unit ball yield a positive inner product.
Revised: In this way, most of the pairs of embeddings on the unit ball yield a positive inner product.

Original: We have known that the inner product of two embeddings x i and x j is determined by the inner product of the TUs, i.e. (cid:104) x i , x j (cid:105) =
Revised: We have known that the inner product of two embeddings x i and x j is determined by the inner product of the TUs, i.e. (cid:104) x i , x j (cid:105) =

Original: (cid:80) Bb (cid:80) Bb (cid:48)
Revised: (cid:80) Bb (cid:80) Bb (cid:48)

Original: =1 w ib w jb (cid:48) (cid:81)
Revised: =1 w ib w jb (cid:48) (cid:81)

Original: Cc =1 (cid:104) u br ( i,c ) c , u b (cid:48) r
Revised: Cc =1 (cid:104) u br ( i,c ) c , u b (cid:48) r

Original: ( j,c ) c (cid:105) in Eq.
Revised: ( j,c ) c (cid:105) in Eq.

Original: By making most of inner products between embedding pairs positive, we can mostly avoid the cases where two negative inner products of TUs ﬁnally yield a positive inner product of the node embeddings.
Revised: By making most of inner products between embedding pairs positive, we can mostly avoid the cases where two negative inner products of TUs ﬁnally yield a positive inner product of the node embeddings.

Original: [Figure]
Revised: [Figure]

Original: D P ROOF TO T HEOREMS
Revised: D P ROOF TO T HEOREMS

Original: Theorem 1.
Revised: Theorem 1.

Original: Given node set V and the ﬁnal embedding dimension p , when r ( i, c ) =
Revised: Given node set V and the ﬁnal embedding dimension p , when r ( i, c ) =

Original: i are constants (indicating R c = |V| , ∀ 1 ≤ c ≤ C ) and use block weights as trainable parameters, the total number of parameters is # params =
Revised: i are constants (indicating R c = |V| , ∀ 1 ≤ c ≤ C ) and use block weights as trainable parameters, the total number of parameters is # params =

Original: B ( C |V| d + |V| ) .
Revised: B ( C |V| d + |V| ) .

Original: Omitting the integer constraints of d , we have:
Revised: Omitting the integer constraints of d , we have:

Original: [Equation]
Revised: [Equation]

Original: When the block weights are set as constants, # params =
Revised: When the block weights are set as constants, # params =

Original: BC |V| d , and we have:
Revised: BC |V| d , and we have:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The number of parameters for embeddings is Bd (cid:80)
Revised: The number of parameters for embeddings is Bd (cid:80)

Original: c R c = BC |V| d , for block weights it is
Revised: c R c = BC |V| d , for block weights it is

Original: B |V| .
Revised: B |V| .

Original: Then the total number of training parameters is:
Revised: Then the total number of training parameters is:

Original: [Equation]
Revised: [Equation]

Original: where # params =
Revised: where # params =

Original: f ( C )
Revised: f ( C )

Original: iff d = p 1 /C .
Revised: iff d = p 1 /C .

Original: Then we have the derivative:
Revised: Then we have the derivative:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: When d f ( C ) d C = 0 , we can reach the solution C ∗ = ln p .
Revised: When d f ( C ) d C = 0 , we can reach the solution C ∗ = ln p .

Original: And we have:
Revised: And we have:

Original: [Equation]
Revised: [Equation]

Original: Now we can reach the conclusion, when d = p 1 /C and C
Revised: Now we can reach the conclusion, when d = p 1 /C and C

Original: = ln p :
Revised: = ln p :

Original: [Equation]
Revised: [Equation]

Original: Following the above proof, it is easy to reach the conclusion min # params =
Revised: Following the above proof, it is easy to reach the conclusion min # params =

Original: B |V| p 1 / ln p ln p when the block weights are set as constants.
Revised: B |V| p 1 / ln p ln p when the block weights are set as constants.

Original: Comments: By Theorem 1, we prove that we can use O ( p 1 / ln p ln p ) space for each node embedding in p -dimensional space with B = 1 .
Revised: Comments: By Theorem 1, we prove that we can use O ( p 1 / ln p ln p ) space for each node embedding in p -dimensional space with B = 1 .

Original: For example, when p = (cid:98) e 1000 (cid:99) which is too large for classical algorithm to deal with, by node2ket, we may use C
Revised: For example, when p = (cid:98) e 1000 (cid:99) which is too large for classical algorithm to deal with, by node2ket, we may use C

Original: = ln p = 1000 and d = (cid:100) p 1 /C (cid:101) =
Revised: = ln p = 1000 and d = (cid:100) p 1 /C (cid:101) =

Original: (cid:100) e (cid:101) , totally about 1000 (cid:100) e (cid:101) or so parameters to train the embedding of dimension (cid:98) e 1000 (cid:99) in the Hilbert space.
Revised: (cid:100) e (cid:101) , totally about 1000 (cid:100) e (cid:101) or so parameters to train the embedding of dimension (cid:98) e 1000 (cid:99) in the Hilbert space.

Original: Before proving Theorem 2, we introduce the following lemma which is obvious by Cauchy inequality:
Revised: Before proving Theorem 2, we introduce the following lemma which is obvious by Cauchy inequality:

Original: Lemma 2.
Revised: Lemma 2.

Original: [Equation]
Revised: [Equation]

Original: , when the equality holds, R
Revised: , when the equality holds, R

Original: [Equation]
Revised: [Equation]

Original: Theorem 2.
Revised: Theorem 2.

Original: Given the node set V and the ﬁnal embedding dimension p , when node2ket utilizes both the r ( i, c ) table and block weights, the total number of parameters is # params =
Revised: Given the node set V and the ﬁnal embedding dimension p , when node2ket utilizes both the r ( i, c ) table and block weights, the total number of parameters is # params =

Original: Bd (cid:80) Cc =1 R c +
Revised: Bd (cid:80) Cc =1 R c +

Original: B |V| + C |V| .
Revised: B |V| + C |V| .

Original: We have:
Revised: We have:

Original: [Equation]
Revised: [Equation]

Original: where C ∗ = (cid:114)(cid:104) (ln p |V| ) 2 − 12 (cid:16) ln p |V| + 1 p |V| (cid:17)(cid:105) / (cid:0) |V| /B
Revised: where C ∗ = (cid:114)(cid:104) (ln p |V| ) 2 − 12 (cid:16) ln p |V| + 1 p |V| (cid:17)(cid:105) / (cid:0) |V| /B

Original: + 1 (cid:1) .
Revised: + 1 (cid:1) .

Original: When the block weights are set as constants, we have:
Revised: When the block weights are set as constants, we have:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Lemma 2, we have:
Revised: By Lemma 2, we have:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where the inequality on the ﬁrst line comes from Lemma 2, the second line comes from d ≥ p 1 /C , and the third line comes form (cid:81)
Revised: where the inequality on the ﬁrst line comes from Lemma 2, the second line comes from d ≥ p 1 /C , and the third line comes form (cid:81)

Original: Cc =1 R c ≥ |V| .
Revised: Cc =1 R c ≥ |V| .

Original: Then we have the derivative:
Revised: Then we have the derivative:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: for which it is very hard to solve d f ( C ) d C = 0 .
Revised: for which it is very hard to solve d f ( C ) d C = 0 .

Original: To give an approximate solution, we ﬁrst do Taylor
Revised: To give an approximate solution, we ﬁrst do Taylor

Original: Expansion for ( p |V| ) 1 /C as:
Revised: Expansion for ( p |V| ) 1 /C as:

Original: [Equation]
Revised: [Equation]

Original: 27) according to which we have:
Revised: 27) according to which we have:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: We let d f ( C )
Revised: We let d f ( C )

Original: d C ≈ 0 omitting the O (1 /C 3 ) term, then we get a solution:
Revised: d C ≈ 0 omitting the O (1 /C 3 ) term, then we get a solution:

Original: [Equation]
Revised: [Equation]

Original: (29) by which d f ( C ) d C (cid:12)(cid:12)(cid:12) C
Revised: (29) by which d f ( C ) d C (cid:12)(cid:12)(cid:12) C

Original: = C ∗ ≈ 0 .
Revised: = C ∗ ≈ 0 .

Original: Since C ∗ is an approximate solution, we have f ( C ∗ ) > min f ( C ) , and hence: min # params <
Revised: Since C ∗ is an approximate solution, we have f ( C ∗ ) > min f ( C ) , and hence: min # params <

Original: f ( C ∗ ) , (30) which is the ﬁrst conclusion of the theorem.
Revised: f ( C ∗ ) , (30) which is the ﬁrst conclusion of the theorem.

Original: For the second one, we just need to remove the second
Revised: For the second one, we just need to remove the second

Original: B |V| term in f ( C ) , similar to the proof of Theorem 1.
Revised: B |V| term in f ( C ) , similar to the proof of Theorem 1.

Original: Comments: It is impossible to ﬁnd analytic solution of d f ( C ) d C = 0 , which gives the C that makes # params minimum.
Revised: Comments: It is impossible to ﬁnd analytic solution of d f ( C ) d C = 0 , which gives the C that makes # params minimum.

Original: We can only give an upper bound of # params .
Revised: We can only give an upper bound of # params .

Original: The procedure to reach
Revised: The procedure to reach

Original: Theorem 2 is similar to Theorem 1.
Revised: Theorem 2 is similar to Theorem 1.

Original: E E XPERIMENTS E. D ETAILED E XPLANATION ON THE M ETRIC P REC @
Revised: E E XPERIMENTS E. D ETAILED E XPLANATION ON THE M ETRIC P REC @

Original: N
Revised: N

Original: IN N ETWORK R ECONSTRUCTION
Revised: IN N ETWORK R ECONSTRUCTION

Original: For large-scale networks, it is infeasible to compute the scores of all |V| ( |V|− 1) node pairs and rank them to reconstruct the raw networks.
Revised: For large-scale networks, it is infeasible to compute the scores of all |V| ( |V|− 1) node pairs and rank them to reconstruct the raw networks.

Original: So, we can only approximately estimate the reconstruction precision by sampling.
Revised: So, we can only approximately estimate the reconstruction precision by sampling.

Original: The description in the paper is actually straight forward.
Revised: The description in the paper is actually straight forward.

Original: First we sample |E| randomly matched node pairs as the noise node pairs, denoted as E n .
Revised: First we sample |E| randomly matched node pairs as the noise node pairs, denoted as E n .

Original: Speciﬁcally, if we have 10 edges in the given network (i.e. |E| = 10 ), we randomly sample 10( N − 1) node pairs as fake (noise) edges E n ( |E n | = 10( N − 1) ).
Revised: Speciﬁcally, if we have 10 edges in the given network (i.e. |E| = 10 ), we randomly sample 10( N − 1) node pairs as fake (noise) edges E n ( |E n | = 10( N − 1) ).

Original: Then we calculate the scores of edges in E ∪ E n and select the top- |E| pairs as the reconstructed edges.
Revised: Then we calculate the scores of edges in E ∪ E n and select the top- |E| pairs as the reconstructed edges.

Original: It means now we have totally 10 |E| edges (containing true edges and fake edges), we hope to select 10 edges from the 10 N edges.
Revised: It means now we have totally 10 |E| edges (containing true edges and fake edges), we hope to select 10 edges from the 10 N edges.

Original: So far, the computation of precision Prec @
Revised: So far, the computation of precision Prec @

Original: N becomes obvious.
Revised: N becomes obvious.

Original: Since the process contains random sampling, we need to ﬁx the random seed to ensure the fairness of experiments.
Revised: Since the process contains random sampling, we need to ﬁx the random seed to ensure the fairness of experiments.

Original: Previous works e.g. (Wang et al., 2016) also conduct NR for large-scale networks in this way, which is indeed a common approach in the research.
Revised: Previous works e.g. (Wang et al., 2016) also conduct NR for large-scale networks in this way, which is indeed a common approach in the research.

Original: E. S TATISTICS OF D ATASETS
Revised: E. S TATISTICS OF D ATASETS

Original: The statistics of datasets are given in Table 7.
Revised: The statistics of datasets are given in Table 7.

Original: All of them are public online.
Revised: All of them are public online.

Original: [Table]
Revised: [Table]

Original: E. D ETAILED E XPERIMENTAL S ETTINGS E.3. B ASELINES
Revised: E. D ETAILED E XPERIMENTAL S ETTINGS E.3. B ASELINES

Original: LINE 2 (Tang et al., 2015) models ﬁrst-order and second-order proximity on the adjacency matrix, trains them separately with edge sampling, and concatenates the representations after normalization on node embedding.
Revised: LINE 2 (Tang et al., 2015) models ﬁrst-order and second-order proximity on the adjacency matrix, trains them separately with edge sampling, and concatenates the representations after normalization on node embedding.

Original: The number of samples is set as 1 × 10 8 .
Revised: The number of samples is set as 1 × 10 8 .

Original: node2vec 3 (Grover & Leskovec, 2016) uses biased random walk to explore both breadth-ﬁrst and depth-ﬁrst structure, and train node embedding and hidden embedding by negative sampling.
Revised: node2vec 3 (Grover & Leskovec, 2016) uses biased random walk to explore both breadth-ﬁrst and depth-ﬁrst structure, and train node embedding and hidden embedding by negative sampling.

Original: We set p = 1 and q = 1 in the experiments, and window size 10, walk length 80 and number of walks for each node 80.
Revised: We set p = 1 and q = 1 in the experiments, and window size 10, walk length 80 and number of walks for each node 80.

Original: VERSE 4 (Tsitsulin et al., 2018) uses node similarity explicitly by Personalized PageRank (PPR),
Revised: VERSE 4 (Tsitsulin et al., 2018) uses node similarity explicitly by Personalized PageRank (PPR),

Original: Adajcency Similarity, and SimRank, and learns node embedding by NCE-x. We use the default version with PPR similarity, damping factor α = 0 . 85 .
Revised: Adajcency Similarity, and SimRank, and learns node embedding by NCE-x. We use the default version with PPR similarity, damping factor α = 0 . 85 .

Original: We run 10 5 epochs for each node.
Revised: We run 10 5 epochs for each node.

Original: NetSMF 5 (Qiu et al., 2019) obtains node embeddings by sparse matrix factorization on a sparsiﬁed dense random walk matrix.
Revised: NetSMF 5 (Qiu et al., 2019) obtains node embeddings by sparse matrix factorization on a sparsiﬁed dense random walk matrix.

Original: We set rank as 512, window size as 10, rounds as 1000, which is the recommended setting by the ofﬁcial code for BlogCatalog.
Revised: We set rank as 512, window size as 10, rounds as 1000, which is the recommended setting by the ofﬁcial code for BlogCatalog.

Original: ProNE 6 (Zhang et al., 2019) initializes embedding by sparse matrix factorization, and then enhances the embedding via spectral propagation.
Revised: ProNE 6 (Zhang et al., 2019) initializes embedding by sparse matrix factorization, and then enhances the embedding via spectral propagation.

Original: We run Python version in the repository with the default setting, i.e. the term number of the Chebyshev expansion k = 10 , θ = 0 .
Revised: We run Python version in the repository with the default setting, i.e. the term number of the Chebyshev expansion k = 10 , θ = 0 .

Original: 5 , and µ = 0 .
Revised: 5 , and µ = 0 .

Original: We use the enhanced embedding as the node embedding result.
Revised: We use the enhanced embedding as the node embedding result.

Original: LouvainNE 7 (Bhowmick et al., 2020)
Revised: LouvainNE 7 (Bhowmick et al., 2020)

Original: ﬁrst adopts Louvain graph partition al-gorithm (Blondel et al., 2008) and then learn node embeddings from the partitions.
Revised: ﬁrst adopts Louvain graph partition al-gorithm (Blondel et al., 2008) and then learn node embeddings from the partitions.

Original: We use Louvain partition as the graph partition algorithm, and set the damping parameter a=0.01 which is the default value in the repository.
Revised: We use Louvain partition as the graph partition algorithm, and set the damping parameter a=0.01 which is the default value in the repository.

Original: HHNE 8 (Zhang et al., 2021) adopts hyperbolic embedding for heterogeneous networks.
Revised: HHNE 8 (Zhang et al., 2021) adopts hyperbolic embedding for heterogeneous networks.

Original: We feed random walk sequences into the hyperbolic embedding core.
Revised: We feed random walk sequences into the hyperbolic embedding core.

Original: We set window size as 5, the number of negative samples as 10.
Revised: We set window size as 5, the number of negative samples as 10.

Original: The other parameters follow the default setting of the ofﬁcial repository.
Revised: The other parameters follow the default setting of the ofﬁcial repository.

Original: E.3. N ODE 2 KET AND N ODE 2 KET +
Revised: E.3. N ODE 2 KET AND N ODE 2 KET +

Original: Experimental parameter setting of node2ket in Table 2 (network reconstruction).
Revised: Experimental parameter setting of node2ket in Table 2 (network reconstruction).

Original: On GR-QC : we do 10M sampling iterations.
Revised: On GR-QC : we do 10M sampling iterations.

Original: On BlogCatalog : we do 100M sampling iterations.
Revised: On BlogCatalog : we do 100M sampling iterations.

Original: On DBLP: we do 20M sampling iterations.
Revised: On DBLP: we do 20M sampling iterations.

Original: On PPI : we do 20M sampling iterations.
Revised: On PPI : we do 20M sampling iterations.

Original: Experimental parameter setting of node2ket+ in Table 2 (network reconstruction).
Revised: Experimental parameter setting of node2ket+ in Table 2 (network reconstruction).

Original: On GR-QC : we do 10M sampling iterations, Louvain graph partition adopts resolution [800 , 1000 , 1200 , 1400 , 1600 , 1800 , − 1 , − 1] (-1 means no parition) for the 8 columns in the
Revised: On GR-QC : we do 10M sampling iterations, Louvain graph partition adopts resolution [800 , 1000 , 1200 , 1400 , 1600 , 1800 , − 1 , − 1] (-1 means no parition) for the 8 columns in the

Original: TEBs, which results in the row number [2229 , 2495 , 2656 , 2861 , 2978 , 3113 , 5242 , 5242] .
Revised: TEBs, which results in the row number [2229 , 2495 , 2656 , 2861 , 2978 , 3113 , 5242 , 5242] .

Original: On
Revised: On

Original: BlogCatalog : we do 100M sampling iterations, Louvain graph partition adopts resolution [800 , 1000 , 1200 , 1400 , 1600 , 1800 , − 1 , − 1] (-1 means no parition) for the 8 columns in the
Revised: BlogCatalog : we do 100M sampling iterations, Louvain graph partition adopts resolution [800 , 1000 , 1200 , 1400 , 1600 , 1800 , − 1 , − 1] (-1 means no parition) for the 8 columns in the

Original: TEBs, which results in the row number [7122 , 7606 , 7989 , 8296 , 8559 , 8754 , 10312 , 10312] .
Revised: TEBs, which results in the row number [7122 , 7606 , 7989 , 8296 , 8559 , 8754 , 10312 , 10312] .

Original: On DBLP: we do 20M sampling iterations, Louvain graph partition adopts resolution [1000 , 1500 , 2000 , 2500 , 3000 , − 1 , − 1 , − 1] (-1 means no parition) for the 8 columns in the
Revised: On DBLP: we do 20M sampling iterations, Louvain graph partition adopts resolution [1000 , 1500 , 2000 , 2500 , 3000 , − 1 , − 1 , − 1] (-1 means no parition) for the 8 columns in the

Original: TEBs, which results in the row number [4874 , 5995 , 6944 , 7775 , 8372 , 12591 , 12591 , 12591] .
Revised: TEBs, which results in the row number [4874 , 5995 , 6944 , 7775 , 8372 , 12591 , 12591 , 12591] .

Original: On PPI : we do 20M sampling iterations, Louvain graph partition adopts resolution [2000 , 2200 , 2400 , 2600 ,
Revised: On PPI : we do 20M sampling iterations, Louvain graph partition adopts resolution [2000 , 2200 , 2400 , 2600 ,

Original: − 1 , − 1 , − 1 , −
Revised: − 1 , − 1 , − 1 , −

Original: means
Revised: means

Original: no
Revised: no

Original: parition) for the 8 columns in the TEBs,
Revised: parition) for the 8 columns in the TEBs,

Original: which
Revised: which

Original: results in the row number [3372 , 3429 , 3458 , 3505 , 3890 , 3890 , 3890 , 3890]
Revised: results in the row number [3372 , 3429 , 3458 , 3505 , 3890 , 3890 , 3890 , 3890]

Original: Experimental parameter setting of node2ket+ in Table 3 (network reconstruction).
Revised: Experimental parameter setting of node2ket+ in Table 3 (network reconstruction).

Original: We do 1500M sampling iterations, Louvain graph partition adopts resolution [1500 , 3000] for the 2 columns in the TEBs, which results in the row number [81790 , 142163] .
Revised: We do 1500M sampling iterations, Louvain graph partition adopts resolution [1500 , 3000] for the 2 columns in the TEBs, which results in the row number [81790 , 142163] .

Original: Experimental parameter setting of node2ket and in Table 4 (link prediction).
Revised: Experimental parameter setting of node2ket and in Table 4 (link prediction).

Original: On GR-QC : we do 100M sampling iterations, we further sample from 2-hop neighbors for positive samples.
Revised: On GR-QC : we do 100M sampling iterations, we further sample from 2-hop neighbors for positive samples.

Original: On BlogCatalog : we do 100M sampling iterations.
Revised: On BlogCatalog : we do 100M sampling iterations.

Original: On DBLP: we do 50M sampling iterations, we further sample from 3-hop neighbors for positive samples.
Revised: On DBLP: we do 50M sampling iterations, we further sample from 3-hop neighbors for positive samples.

Original: On PPI : we do 100M sampling iterations, we sample from 3-hop neighbors for positive samples.
Revised: On PPI : we do 100M sampling iterations, we sample from 3-hop neighbors for positive samples.

Original: Experimental parameter setting of node2ket and in Table 9 and Table 10 (node classiﬁcation).
Revised: Experimental parameter setting of node2ket and in Table 9 and Table 10 (node classiﬁcation).

Original: On PPI, we do sampling iterations.
Revised: On PPI, we do sampling iterations.

Original: On YouTube, we do sampling iterations.
Revised: On YouTube, we do sampling iterations.

Original: For baselines we set embedding dimension as p = 8 .
Revised: For baselines we set embedding dimension as p = 8 .

Original: For node2ket, we set B = 1 , C = 2 , and d = 4 , and do 100M sampling iterations.
Revised: For node2ket, we set B = 1 , C = 2 , and d = 4 , and do 100M sampling iterations.

Original: E. F ULL R ESULTS OF A BLATION S TUDY
Revised: E. F ULL R ESULTS OF A BLATION S TUDY

Original: The results are give in Table 8.
Revised: The results are give in Table 8.

Original: When B = 1 , w ib becomes meaningless whether w ib are constants or trainable parameters, so the reported results are the same.
Revised: When B = 1 , w ib becomes meaningless whether w ib are constants or trainable parameters, so the reported results are the same.

Original: From the results we see that trainable w ib and the increasing block number B may make a negative inﬂuence on the model.
Revised: From the results we see that trainable w ib and the increasing block number B may make a negative inﬂuence on the model.

Original: It is possibly because the increased number of parameters brings difﬁculties to model training.
Revised: It is possibly because the increased number of parameters brings difﬁculties to model training.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: E. V ISUALIZATION OF R ECONSTRUCTION R ESULTS E.5. I NTERPRETABLE V ISUALIZATION ON L ADDER G RAPH
Revised: E. V ISUALIZATION OF R ECONSTRUCTION R ESULTS E.5. I NTERPRETABLE V ISUALIZATION ON L ADDER G RAPH

Original: We conduct node embedding via node2ket on a circular ladder graph of 20 nodes, which is plotted in Fig.
Revised: We conduct node embedding via node2ket on a circular ladder graph of 20 nodes, which is plotted in Fig.

Original: 8 a) and the adjacency matrix is visualized in Fig.
Revised: 8 a) and the adjacency matrix is visualized in Fig.

Original: 8 f).
Revised: 8 f).

Original: For node2ket, we set B = 1 ,
Revised: For node2ket, we set B = 1 ,

Original: C = 2 , and d = 4 .
Revised: C = 2 , and d = 4 .

Original: After obtaining the TUs’ embeddings, we reconstruct the adjacency matrix from Column 1 and Column 2 of the TEB and visualize them in Fig.
Revised: After obtaining the TUs’ embeddings, we reconstruct the adjacency matrix from Column 1 and Column 2 of the TEB and visualize them in Fig.

Original: 8 b) and c).
Revised: 8 b) and c).

Original: Then, from the ﬁnal node embeddings, we compute the reconstructed adjacency matrix and visualize it in realvalued/binarized form as shown in Fig.
Revised: Then, from the ﬁnal node embeddings, we compute the reconstructed adjacency matrix and visualize it in realvalued/binarized form as shown in Fig.

Original: 8 d) and e).
Revised: 8 d) and e).

Original: From the results, we have the ﬁndings: i) by comparing Fig.
Revised: From the results, we have the ﬁndings: i) by comparing Fig.

Original: 8 b), c), and d), we can see that column 1 and column 2 preserves structures in an approximately orthogonal manner, which means the extracted information is different among columns of a TEB, and ii) by comparing Fig.
Revised: 8 b), c), and d), we can see that column 1 and column 2 preserves structures in an approximately orthogonal manner, which means the extracted information is different among columns of a TEB, and ii) by comparing Fig.

Original: 8 e) with the ground truth f), node2ket fully reconstructs the input adjacency matrix.
Revised: 8 e) with the ground truth f), node2ket fully reconstructs the input adjacency matrix.

Original: To compare the reconstruction performance of different methods on different data, we further investigate a Hypercube graph, 3D-Grid graph, and the real-world Karate
Revised: To compare the reconstruction performance of different methods on different data, we further investigate a Hypercube graph, 3D-Grid graph, and the real-world Karate

Original: Club graph in Appendix E.5.
Revised: Club graph in Appendix E.5.

Original: The results directly show the overwhelming performance of node2ket on the network reconstruction task.
Revised: The results directly show the overwhelming performance of node2ket on the network reconstruction task.

Original: E.5. V ISUALIZATION ON O THER G RAPHS
Revised: E.5. V ISUALIZATION ON O THER G RAPHS

Original: We investigate a circular ladder graph, a 5-hypercube graph, a 3D-grid graph, and the Karate club graph, and we evaluate all the baselines.
Revised: We investigate a circular ladder graph, a 5-hypercube graph, a 3D-grid graph, and the Karate club graph, and we evaluate all the baselines.

Original: The results in Fig.
Revised: The results in Fig.

Original: 9 show overwhelming reconstruction ability of node2ket, compared with different methods.
Revised: 9 show overwhelming reconstruction ability of node2ket, compared with different methods.

Original: We also ﬁnd that node2ket does not perfectly reconstruct the adjacency matrix for some complex networks, e.g. the 3D-Grid graph and the realworld Karate Club graph.
Revised: We also ﬁnd that node2ket does not perfectly reconstruct the adjacency matrix for some complex networks, e.g. the 3D-Grid graph and the realworld Karate Club graph.

Original: Settings.
Revised: Settings.

Original: All baseline methods use 8-dim embedding
Revised: All baseline methods use 8-dim embedding

Original: [Figure]
Revised: [Figure]

Original: E. E XPERIMENTS ON N ODE C LASSIFICATION
Revised: E. E XPERIMENTS ON N ODE C LASSIFICATION

Original: We conduct experiments on PPI (totally 50 labels) and YouTube (totally 47 labels).
Revised: We conduct experiments on PPI (totally 50 labels) and YouTube (totally 47 labels).

Original: After embeddings are obtained, LIBLINEAR (Fan et al., 2008) is used to train one-vs-rest Logistic Regression classiﬁers.
Revised: After embeddings are obtained, LIBLINEAR (Fan et al., 2008) is used to train one-vs-rest Logistic Regression classiﬁers.

Original: We range the portion of 0training data from 1% to 9% for YouTube and 10% to 90% for PPI.
Revised: We range the portion of 0training data from 1% to 9% for YouTube and 10% to 90% for PPI.

Original: We use Macro-F1 as the metric.
Revised: We use Macro-F1 as the metric.

Original: In line with (Perozzi et al., 2014; Tang et al., 2015; Wang et al., 2016) for each method we perform 10 times of trials and report the average scores.
Revised: In line with (Perozzi et al., 2014; Tang et al., 2015; Wang et al., 2016) for each method we perform 10 times of trials and report the average scores.

Original: Results are given in Table 9 and which shows that node2ket consistently outperforms baselines under the setting.
Revised: Results are given in Table 9 and which shows that node2ket consistently outperforms baselines under the setting.

Original: Portion
Revised: Portion

Original: Data
Revised: Data

Original: [Table]
Revised: [Table]

Original: node2vec 6.
Revised: node2vec 6.

Original: proNE 7.
Revised: proNE 7.

Original: NetSMF 8.
Revised: NetSMF 8.

Original: LouvainNE 7.
Revised: LouvainNE 7.

Original: HHNE 9.
Revised: HHNE 9.

Original: node2ket 10.
Revised: node2ket 10.

Revised: Portion of Training Data

Original: [Table]
Revised: [Table]

Original:  line 22.
Revised: % 7% 9% line 22.

Original: verse 24.
Revised: verse 24.

Original: node2vec 11.
Revised: node2vec 11.

Original: proNE 21.
Revised: proNE 21.

Original: NetSMF 25.
Revised: NetSMF 25.

Original: LouvainNE 20.
Revised: LouvainNE 20.

Original: HHNE 23.
Revised: HHNE 23.

Original: node2ket 29.
Revised: node2ket 29.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: E
Revised: E

Original: R S E f )
Revised: R S E f )

Original: [Figure]
Revised: [Figure]

Original: t
Revised: t

Original: S
Revised: S

Original: [Figure]
Revised: [Figure]

Original: F g ) P r o
Revised: F g ) P r o

Original: N E h )
Revised: N E h )

Original: L o u v a i n N
Revised: L o u v a i n N

Original: E
Revised: E