Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents a low-rank tensor approximation algorithm for non-negative tensors. The algorithm is based on two observations. First, rank-1 approximation for tensors can be viewed as a mean-field approximation by treating each tensor as a probability distribution. Second, the authors provide a sufficient condition for distribution parameters to reduce Tucker ranks of tensors and, interestingly, this sufficient condition can be achieved by iterative application of the mean-field approximation. Since the mean-field approximation is always given as a closed formula, a fast low-rank approximation algorithm can be formulated without using a gradient method. The authors empirically demonstrate on several datasets that the algorithm is faster than the existing non-negative Tucker rank reduction methods ( in terms of achieving comparable approximations of given tensors).
main_review: Pros
* Strong theoretical and algorithmic contribution with an original connection between tensor factorizations and mean-field approximation theory.
* The proposed algorithm is novel and interesting 
* I'd rate the significance of the work as high as the paper draws on ideas from information geometry and physics. Hence the paper will likely be of interest to diverse sets of researchers. 
* The paper is clearly and well written. 

Cons 
* The paper is a bit sparse on the experimental side but that's not necessarily a con given the algorithmic/theoretical contribution as mentioned above. 

limitations_and_societal_impact: The paper does not include any narrative of any negative societal impact. That said, I don't immediately see an immediate negative impact beyond the hypothetical possibility of tensor decompositions being used for unethical downstream tasks.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper considers the low Tucker-rank approximation problem, uses the mean-field theory to interpret rank-one approximation for tensors by treating tensors as probability distributions, and then develops a two-step Tucker-rank reduction algorithm which does not involve gradient descent. Theoretical analysis of the proposed algorithm is provided in the information geometry framework. In particular, a Bingo rule is introduced to relate Tucker rank and modal expansion of a tensor in another coordinate system, which paves the way to reformulate the Tucker rank reduction problem as an m-projection. 
main_review: The proposed tensor-rank reduction method is interesting, and theoretical discussions from the perspective of mean-field theory has certain novelty. Numerical results on various types of data sets are supportive and convincing. This work is important for low-rank approximations in tensor recovery and could be insightful for other tensor rank reduction methods. Minor issues: In line 19-20, there are more tensor ranks other than CP-rank and Tucker-rank, e.g., tubal rank. In line 84, what does "the total sum of $\mathcal{P}$" mean? In line 94, what are "uninformative indices"? Maybe some examples could be referred. In the caption of Figure 1, "Bingo space" is not defined. In lines 211-212, how can the distribution independence assumption be guaranteed for generic tensors in the original form? Does the performance of the algorithm rely on this assumption? In Figure 3, are there any reasons for choosing the tensors with the same dimension across the modes?
limitations_and_societal_impact: The proposed framework works only for nonnegative tensors due to the exponential involved coordinate transform, which may limit the use in practice. Moreover, if the number of modes or some dimension is very large, the first step of LTR involves a random sampling which may waste some iterations and computation of (1) could be slow. Stopping criteria for LTR could be mentioned as well.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper considers the problem of low-rank approximation for non-negative tensors. The authors propose a Tucker rank reduction algorithm based on information geometry, and provide a probabilistic perspective through mean-field approximation.
main_review: As the paper is mainly focused on the theoretical analysis of low-rank tensor approximation, I think the main drawback is that there is no theoretical guarantee for both the approximation error and the computational complexity of the algorithm. The performance of the algorithm is only shown by numerical experiments. In addition, the proposed sufficient condition and the mean-field connection seem to be of only theoretical interest, and fail to provide insights and guidance for practical applications.

limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper derives a tensor decomposition method using mean-field approximation and information geometry. The main claim of this paper is the speed of computation compared to some basic baseline methods. An interesting direction but there are several directions that the paper can improve further.
main_review: This paper is technically interesting and can be a useful addition to the tensor decomposition method. However, this new method does not show any additional benefits other than the gain in speed.

Experiments are not convincing. First of all, I feel that the dimensions for synthetic data are small. I would like to see the speed for much larger tensors. It would be informative if the Tucker ranks of tensor have different values, rather than considering equal r. I believe that it will be challenging to identify the Tucker rank in cases where there is not equal r. 
From the given plots it is difficult to identify differences in accuracy between the proposed method and the NTD. Further, it is hard to understand what is meant by the size of the core tensor, can the authors explain it (and how it is decided)?
Can authors include simulations with larger tensors that with different Tucker ranks on each mode?   Can the authors provide more details on the prediction accuracy of tensor decomposition? 

It would be useful to know if the proposed methods mean-field approximation can be extended to other tensor decomposition methods such as tensor networks. Tensor networks such as tensor trains have shown to be more computationally robust for higher-order tensors. 

The paper is written well. I am not highly familiar with all the methods used discussed in the paper, however, the paper seems to be technically correct. The plots in the experiments section should be large enough to see lines clearly. Can the authors give core-computational steps in algorithmic format for easy understanding? 
limitations_and_societal_impact: Only the advantage of the speed of decomposition is discussed. It is useful if reconstruction accuracy is also improved.  

Another limitation is the restriction to Tucker decomposition, since recently developed methods such as tensor trains can be more feasible to use with higher-order tensors.
review_title: null

Meta-review: This paper gives a new way to do Tucker rank reduction for non-negative tensors. The new approach puts additional constraints on the form of the decomposition (characterized by "bingo-space" defined in the paper) and allows the algorithm to find the optimal solution under these constraints very efficiently. The approach is quite novel and has some reasonable guarantees. The authors should clarify these guarantees in detail in the revised version.
Decision: Accept (Poster)