Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents a new tensor network decomposition model for high-order tensor analysis. The proposed model factorizes a high-order tensor into a set of latent factors connected to a specific wheel topology. This can be viewed as a combination of the classic Tucker decomposition and the tensor ring decomposition. The numerical experiments of tensor completion application on both synthetic data and real-world examples demonstrate the superiority of the proposed method in terms of reconstruction accuracy. 
strengths_and_weaknesses: [Strengths]  
- Innovative construction of the tensor network model inheriting the merits of the classic Tucker decomposition model
- State-of-the-art performance on real-world tensor completion applications
- Linear scaling for the number of hyperparameters with increased tensor dimension

[Weeknesses]
- Increased computational burden compared with tensor-train and tensor-ring models
- Expensive storage burden scaled exponentially with the dimension of tensor data -- sacrificing one of the significant benefits of the tensor-train model 
questions: - Considering the numerical results in Table 1, why are the reported PSNR values for competing methods TRLRF [A] and  FCTN-TC [B] different from those reported in Table 1 of [B] on the same data (container, news, and HSV)? 
- Why does the most complicated model FCTN-TC run much faster than others (e.g., 26.27s v.s. 154.67s) in Toy data

[A] Tensor ring decomposition with rank minimization on latent space: An efficient approach for tensor completion, AAAI 2019  
[B] Fully-connected tensor network decomposition and its application to higher-order tensor completion, AAAI 2021  

IMO, this paper serves as a nice contribution to the tensor decomposition domain and should be accepted if the above question could be properly answered. 
***
**post-rebuttal: I keep my original score though I feel the hyperparameter sensitivity issue should be extensively studied --- I'm not sure whether the superior results beyond other competing methods are partially owing to the test data selection and hyperparameter tuning. Missing to reveal this problem would definitely make the results weaker**
limitations: N.A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a new tensor decomposition method to improve the downsides of existing ones. The proposed tensor wheel decomposition can be viewed as a hybrid between TR decomposition and Tucker decomposition, i.e., it is similar to TR decomposition but there is an additional core tensor of which the order is the same as the input data and all the factor tensors are connected (tensor-producted) to it. This has some advantages: (1) The Nth order structure of the input data can be preserved. (2) Nevertheless, it is more manageable than FCTN. (3) All the factors are now interconnected by the core tensor, unlike TR decomposition where some factors are far away so it is not easy to find direct relationships between them. Experiments show that, for a few synthetic and real data, the proposed method achieves the best performance.
strengths_and_weaknesses: The paper proposes a clever way to bypass the downsides of the existing tensor network type decompositions. Unlike TT and TR decompositions, nonadjacent factors are connected via the core tensor and the original structure of the data is better preserved. At the same time, the number of connections is minimized (unlike FCTN) so that it is more manageable. The idea is quite intuitive and convincing, and accordingly, the superior performance is also convincing.

The proposed decomposition is found by a PAM-style method, and the convergence proof is provided. One shortcoming of the paper is that there is no deeper analysis of the decomposition. There are some theorems in the paper, however, they are about basic algebraic properties rather than deeper ones such as approximation error bounds.

Another thing that bothers me is the comparison with FCTN. In Figure 3, Why does FCTN-TC take a shorter time than TW-TC? This seems somewhat unintuitive, considering the main claim (TW decomposition being more manageable than FCTN). Is it because the best performing hyperparameter setting of FCTN happens to be very small? Moreover, it seems that FCTN-TC usually performs worse than TT-TC and TR-TC even though it has heavier connections. Why is this?
questions: The TC experiments only use i.i.d missing conditions which are not realistic. How will it go if we use more realistic conditions, e.g., structured missing conditions?

limitations: I believe that this paper does not have any serious potential negative societal impact.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposed a new tensor network decomposition method, namely Tensor Wheel Decomposition. Different from prior tensor train or tensor ring decomposition, the tensor wheel decomposition introduced a new core tensor factors C in the factorization. An ALS algorithm (algorithm 1) and proximal solution (algorithm 2) are provided. Empirical study in completion task on synthetic data and real data shows that the new algorithm is better than the TT and TR based factorization methods. 
strengths_and_weaknesses: The paper extends the existing tensor networks method to a new tensor factorization. The extension is trivial, but might still brings some new knowledge to the area. 

The new tensor wheel factorization seems to be a combination of tensor ring factorization and Tucker factorization, without core tensor taking into considerations. While the comparison to Tucker factorization is not included in the paper. 
questions: [Q1] In the experimental results of Table1, since the rank are learnt different based on different algorithm, could the author list the rank parameter or the model size to understand the model complexity from different algorithm?

[Q2] The tensor wheel seems to be more general as a combination of tucker decomposition and tensor ring decomposition. Could the author highlight the difference between the TW to Tucker decomposition, and include tucker decomposition as a comparable here? 

[Q3] In Figure 3, the data is generated via Tucker decomposition while the comparison is mainly between TT, TR, and TW. Will generating data via TW or TR with Tucker decomposition included, makes more sense for algorithm comparison? Additionally, from the systematic data, it seems TW algorithm is mostly slow and having large performance variance, where TR seems to be faster with smaller performance variance. Will there be any regularization methods to reduce TW performance variance? 
limitations: The paper proposed a new tensor wheel factorization method. Although it is a trivial but interesting extension, comparison to Tucker decomposition method is needed to demonstrate the improvement. From the existing results, model complexity and performance trade-off also needs to provided to demonstrate the efficiency of the new factorization method.  Last but not least, interpretations on the algorithms efficiency and performance variance is needed to show the advantage of TW as compared to TT and TR.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, a new Tensor Network (TN) topology is proposed, the Tensor Wheel (TW), which can be seen as a combination of Tensor Ring (TR) and Tucker network topologies. The paper explores the properties of TW and develop a Proximal Alternating Minimization (PAM) algorithm to make inference of missing values in tensor data. Experimental results of tensor completion on synthetically generated and real-world signals are presented and compared against some state-of-the-art tensor completion methods.
strengths_and_weaknesses: Strengths:
-	Clear definition of the new topology and description of some of its properties
Weaknesses
-	(major) I don’t agree with the limitation (ii) of current TN models: “At least one Nth-order factor is required to physically inherit the complex interactions from an Nth-order tensor”. TT and TR can model complex modes interactions if the ranks are large enough. The fact that there is a lack of direct connections from any pair of nodes is not a limitation because any nodes are fully connected through a TR or TT. However, the price to pay with TT or TR to model complex modes interactions is having bigger core tensor (larger number of parameters). The new proposed topology has also a large price to pay in terms of model size because the core tensor C grows exponentially with the number of dimensions, which makes it intractable in practice. The paper lacks from a comparison of TR/TT and TW for a fixed size of both models (see my criticism to experiments below).
-	The new proposed model can be used only with a small number of dimensions because of the curse of dimensionality imposed by the core tensor C.
-	(major) I think the proposed TW model is equivalent to TR by noting that, if the core tensor C is represented by a TR (this can be done always), then by fusing this TR with the cores G_n we can reach to TR representation equivalent to the former TW model. I would have liked to see this analysis in the paper and a discussion justifying TW over TR.
-	(major) Comparison against other models in the experiments are unclear. The value of the used ranks for all the models are omitted which make not possible a fair comparison. To show the superiority of TW over TT and TR, the authors must compare the tensor completion results for all the models but having the same number of model parameters. The number of model parameters can be computed by adding the number of entries of all core tensors for each model (see my question about experiment settings below).
-	(minor) The title should include the term “tensor completion” because that is the only application of the new model that is presented in the paper.
-	(minor) The absolute value operation in the definition of the Frobenius norm in line 77 is not needed because tensor entries are real numbers. 
-	(minor) I don’t agree with the statement in line 163: “Apparently, the O(NIR^3+R^N) scales exponentially”. The exponential grow is not apparent, it is a fact. 
------------------------------------------------------
I updated my scores after rebuttal. See my comments below
questions: -	Could you please provide the details of the ranks used for each model used in Fig. 1 and Fig. 2? 
-      Can you compute the number of model parameters in each case (sum of number of entries in each core tensor for each model)?
------------------------------------------------------
I updated my scores after rebuttal. See my comments below
limitations: Yes, the authors have stated that the main limitation of their proposed model is its exponentionally grow of model parameters with the number of dimensions.
review_title: null

Meta-review: metareview: Two reviewers consider that the proposed construction is clearly innovative.
and all reviewers consider that the contribution is useful to the tensor learning community. 

The experiments show that the proposed method yields improved performance. 

The three reviewers who participated in the discussion with the authors and/or took into account the rebuttal of the author expressed that they were satisfied with the rebuttal. 

Reviewer Bfwm who is the only reviewer assigning a score of 4 or lower, did not consider the rebuttal and did not respond to any message after the initial review.  
The AC considers that his concerns have been well addressed by the authors and this reviewer states in their initial review that this work "might still brings new knowledge to the area".

The authors are encouraged to take into account in particular the fruitful discussion with reviewer Dt2K to enhance their manuscript with additional discussions and insights, and to further strengthen their experiments if possible (consider core tensors in Tucker with different ranks in different modes), given that the results tend to be sensitive to the choice of hyperparameters (and possibly on the hyperparameter search strategy) and to the choice of dataset.
confidence: Certain
Decision: Accept