Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper employs the tensorized embedding proposed in word2ket to the network embedding. It is similar with the second order part in network embedding method LINE, which is motivated from word2vec. Its superiority is verified on the tasks of topology reconstruction and link prediction.
strength_and_weaknesses: Strength 
- It is interesting to employ tensorized embedding to network embedding field.

Weakness
- The novelty is very limited. Most of the parts are similar to word2ket. Thus, the contribution to network embedding field is very limited, since only the objective function is different. 
- There is no contribution on the graph topology exploitation. The topology is only used in the the objective function, which is the same as pervious network embedding ones.
- The experimental evaluations are not sufficient. The performance on node representation is only verified on topology-related tasks. However, that on node classification task is not given. Furthermore, the performance on topology-based tasks are only compared with the classic methods, instead of the SOTA such as hyperbolic space embedding methods.
- The writing is redundant. Most content is similar to word2ket.

clarity,_quality,_novelty_and_reproducibility: Both novelty and clarity need to be improved.

summary_of_the_review: My main concern are the limited novelty and contribution and insufficient evaluations show in weakness.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper studied a novel network embedding method inspired by quantum fields and modeled with tensors and contractions (i.e., outer product). More specially, the embedding vector for each node is represented by a linear combination of rank-one tensors, equivalent to the CP model studied in numerical algebra. In the paper, the authors also discussed the compressive power of the proposed method in a theoretical way. In the experiment part, this paper showed superior performance on network analysis tasks and evaluated the parallelizability and interpretability, which is crucial in practice.
strength_and_weaknesses: ++Strength

1. Very interesting model and the combination with quantum is also very smooth and clearly discussed.
2. The improvement in the empirical performance is remarkable.

— Weakness

1. The theoretical results are not well explained.
clarity,_quality,_novelty_and_reproducibility: **Clarity and Quality:**

The full paper is quite well-written and clearly introduces the main contributions of the work. The clarity and quality of the paper can be improved if authors pay some attention to the following several points:

1. 
2. The clarity of the “objective function” part can be further improved. As one of the readers (or reviewers) of this paper, I’m not familiar with network embedding well. So when I went through the “objective function” part, I got confused with some vocabulary, such as what “negative sample” and the distribution $\mathbb{P}_n$ stand for in this scenario? and why in the second term of Eq. (2) there is no index $k$ in the summation over $k\in{}[K]$? Maybe this knowledge is well-known by network analyzers, but it would be more friendly for general ML researchers if this part could be introduced more clearly.
3. In Sec 4.3 the paper discussed the theoretically optimal to compressive power (although I fully understand what the meaning of “compressive power” is). It would be better if more explanations (such as insights) could be given following each theoretical result. For example, why does the logarithm appear in the equations? If the assumptions are plausible in practice? and so on.

**Novelty:**

Although this might not be the first paper to apply tensors to network embedding, the proposed methods and the used tricks are very interesting and inspireful for tensor researchers such as the graph partition used in NODE2KET+. 

**Reproducibility**

The experimental settings are clearly introduced.
summary_of_the_review: This is excellent work at least in the tensor community. The problem focused in this paper would inspire lots for tensor researchers to explore more valuable applications in machine learning. It also gives a bridge to connect machine learning to quantum computing.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes an unsupervised network (graph) embedding technique. Rather than using unstructured vectors to represent nodes, the embedding framework uses sums of Kronecker products of learnable vectors to represent the nodes. This is inspired by entanglement in quantum physics where similar representations are used to represent quantum states.
strength_and_weaknesses: Strengths:

S1. The idea itself is interesting. Using compressed tensor formats in ML has been used for a variety of things like analyzing deep NN expressiveness and compression of NN weight layers. I haven't seen it in the context of node embedding before (although my knowledge of the network embedding literature is limited).

S2. Intuitively, it seems like the proposed technique could lead to a substantial reduction in the number of parameters required for a given level of performance in a down-stream task.

Weaknesses:

W1. The main weakness is the experiments. The network reconstruction experiments are not convincing. It is not clear why the ability to reconstruct the original network is important for any interesting down-stream tasks. For example, simply storing the whole network itself only requires 2*(#edges) numbers, and will give perfect "network reconstruction" without any computational cost (so this is a "trivial method" that any other proposal ought to beat). By contrast, the proposed methods node2ket and node2ket+ require many more parameters than are required to store the original network, but give much worse reconstruction accuracy. Based on the provided parameters in the paper and appendix, I calculated the following parameter requirements for the datasets in Table 1.

GR-QC:
- Exact representation: 2*(#edges) = 28,992
- node2ket: 8 * 5242 * 16 = 670,976 (23x more than exact representation)
- node2ket+: (2229 + 2495 + 2656 + 2861 + 2978 + 3113 + 5242 + 5242) * 16 = 429,056 (15x more than exact representation)

BC: 
- Exact representation: 667,966
- node2ket: 8 * 10312 * 16 = 1,319,936 (2x more than exact representation)
- node2ket+: (7122 + 7606 + 7989 + 8296 + 8559 + 8754 + 10312 + 10312) * 16 = 1,103,200 (2x more than exact representation)

DBLP: 
- Exact representation: 99,254
- node2ket: 8 * 12591 * 16 = 1,611,648 (16x more than exact representation)
- node2ket+: (4874 + 5995 + 6944 + 7775 + 8372 + 12591 + 12591 + 12591) * 16 = 1,147,728 (12x more than exact representation)

PPI: 
- Exact representation: 153,168
- node2ket: 8 * 3890 * 16 = 497,920 (3x more than exact representation)
- node2ket+: (3372 + 3429 + 3458 + 3505 + 3890 + 3890 + 3890 + 3890) * 16 = 469,184 (3x more than exact representation)

Similarly, the circular ladder only requires 54 numbers for exact representation, while node2ket uses 64 parameters.

The only case I saw where there was some compression was in Fig 2 (b) when node2vec uses C=2 and d=4. In this case, node2vec uses 0.12 the number of parameters required to store the original network.

The Link prediction experiment in Sec 5.3 is poorly explained. How is the prediction done? What do you mean by "mixture of masked edges and noise edges"? I couldn't find a clarification in the appendix.
clarity,_quality,_novelty_and_reproducibility: The text is sometimes a bit hard to follow due to lacking language, but this is not a huge issue. There are some typos, for example:

- Page 6: Fisrt -> First
- Page 7: abilitty -> ability
- Page 7: in in -> in
- Page 9: reconstruction reconstruction -> reconstruction

There are also multiple parts of the paper that are poorly explained:
- First paragraph in Sec 4.1 is hard to follow.
- In Sec 4.2: "We compute the gradient and update TU embeddings directly without a complete procedure of forward propagation". Do you mean that you are able to avoid using automatic differentiation since you can easily compute the gradients yourself? If so, you should clarify this.
- In Eqs. (5) and (6): You should clarify that this is the gradient of _one_ term in the sum in Eq. (2).
- Theorems 1 and 2: Writing "min # params" is not very clear. It would be better if you wrote in words what the bound is (e.g., "It is sufficient to use x parameters to achieve a generated embedding which is at least p".)
- Fig. 2 caption: "n2k" is an abbreviation which is never defined.
- Toward the end of Sec 4, the part "ii) TEBs are designed to have a flexible ... iii) achieves to fully ... full embedding." These two points are vague. For (ii), it would be much better if you could precisely say which parameters are fixed to what in word2ket/word2ketXS. For (iii), it's not clear what you mean.
- How is Prec@N defined?
- The last 4 sentences in the 1st paragraph of Sec 5.2 ("First we sample ... for different methods.") don't make sense.
- Is the y-axis in Fig. 3 (b) percent or percentage points?

summary_of_the_review: The method proposed in this paper is an interesting idea, but the experiments are not convincing at all. In particular, it's not clear why network reconstruction matters for down-stream tasks. The paper is also unclear in multiple places; the writing needs to be improved.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: The paper proposes an unsupervised network (graph) embedding technique, which is a novel network embedding method inspired by quantum fields and modeled with tensors and contractions (i.e., outer product). In the experiment part, this paper showed superior performance on network analysis tasks and evaluated the parallelizability and interpretability. 

Strengths: The model is interesting model and the combination with quantum is also very smooth and clearly discussed. This work is the first to enable adopting the tensorized embedding for the network embedding domain. 

Weakness:  The contributions to network embedding are very limited. The embedding space is similar to word2ket to some extent, while the employed objective function is similar to LINE. Essentially, this paper is the combination of these two previous works.
justification_for_why_not_higher_score: After authors' rebuttal, the reviewers still have major concerns about limited novelty and less convincing experiments. It seems to be combination of existing techniques, however its advantages are not clearly demonstrated by experiments.
Decision: Reject