Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors' starting point is the "Legendre Decomposition for Tensors" paper. The Legendre decomposition is modified to approximate a data tensor by "dividing" the original data tensor N-ways and approximating the N-blocks which "interact" with each other through a subset of the tensor modes. The authors assume that tensor blocks have dominant interactions while others can be deemed to be irrelevant and set to zero. The authors refer to this modified approximation as a many-body approximation.  The approach was tested on simulation data and 4D light field data etc. The authors report on some of the experiments lower reconstruction errors relative to prior research.



strength_and_weaknesses: Strengths:
1. This modified non-negative tensor approximation subdivides a data tensor into multiple tensor blocks that approximate the tensor whole.  The tensor blocks have modes that are entirely their own, and modes that they share with other tensor blocks.

2. Mode interaction is visualized through a new network diagram.

Weakness:
1. There is no algorithmic description.  From Appendix B, I see that a tensor is reshaped and split into several tensor blocks each approximated by a tensor ring. Presumably, some of the irrelevant mode interactions between the tensor rings (tensor blocks) are set to zero.  Which mode interactions were set to zero? Would any reshaping and tensor block subdivision work equally well?

2. The authors have not clearly articulated 

+ (i) the criteria for splitting a data tensor into $N$ data blocks,
+ (ii) the criteria for determining which mode interactions are deemed to be dominant or irrelevant and set to zero, or 
+ (iii) the choice of tensor ring ranks.
 
4. It is unclear what the experimental results are demonstrating.  What is the AI, ML, CV, or CG problem being solved?   The goal is to find meaningful representations that increase recognition rates or result in high-quality reconstructed or synthesized images, etc.  Reconstruction errors are not always good proxy indicators for the previously mentioned tasks. For CG, the authors may wish to display reconstructed or newly synthesized images versus reconstruction errors versus decomposition speed versus reconstruction speed. 

If the goal is compression, this should be stated.  The authors may wish to compare their work with jpeg or other state-of-the-art compression techniques.
clarity,_quality,_novelty_and_reproducibility: This paper performs a Legendre tensor approximation as a many-block tensor approximation.  It also introduces a tensor network interaction diagrams for visualization.  

The authors' approach has many similarities to causal block multilinear decomposition.  A block tensor decomposition splits a data tensor into $N$ tensor blocks that have their own individual intrinsic mode matrices and share a subset of their extrinsic mode matrices.  The data blocks "interact" with each other to compute the mode matrix representations for the extrinsic modes that they share.   The zeros in a compound/hierarchical core tensor make the interactions self-evident.

See: http://web.cs.ucla.edu/~maov/CausalX_Block_Multilinear_Factor_Analysis_ICPR2020.pdf

See Fig 4b: http://web.cs.ucla.edu/~maov/CausalDeepLearning.pdf


The authors employ abstract notation which would benefit from being grounded in an application.  Without such grounding, it isn't always clear what the variables represent.  

According to Appendix B, the authors seem to map the Dino light field data, for example, into a "many-body" approximation problem by reshaping a 5th-order data tensor (9 x 9 x 512 x 512 x 3)  into a 9th-order tensor (6 x8 x 6 x 8 x 6 x 8 x 6 x 8 x 12) and splitting the new tensor into 7 different 9th order tensors that are decomposed by 7 tensor rings with different ring ranks.  Reporting this information is nice, but one still needs an explanation.   Why is the 7-block subdivision of the Dino LFD a "many-body" problem?

Does it matter what type of reshaping or subdivision is performed?  Would any arbitrary reshaping and subdivision of a data tensor work equally well?  Which modes were considered dominant versus irrelevant in the 4D light field example?  Which modes were dimensionally reduced?  Are the reconstructed images still usable? 

The network diagrams use both square and circle nodes that are labeled with indices, matrices, and tensors. Some are filled in and others are not.  Not entirely clear on the difference.
 

Detailed comments:
-----
The introduction and the statements that motivate the work need some editing:

+ "Tensor decomposition is one of the most popular methods that extract features by approximating tensors by the sum of products of smaller size of tensors, often called factors."

Why are the terms of a summation called factors? "Factorization or factoring consists of writing a number or another mathematical object as a product of several factors, usually smaller or simpler objects of the same kind."   A tensor decomposition, including the CP decomposition, consists of writing a data tensor as the product of a core tensor and a set of mode matrices. Mode matrices are called causal factors when performing causal inference.
 
+ "In most of tensor decomposition approaches, a low-rank structure is typically assumed, where a given tensor is approximated by a linear combination of a small number of bases. Such decomposition requires the following two information. First, it requires the structure, which specifies the type of decomposition such as CP decomposition (Hitchcock, 1927) and Tucker decomposition..."

As written, the authors seem to categorize both the CP and the Tucker decompositions as rank decompositions. The Tucker and Candecomp/Parafac (CP) tensor decomposition embody different properties of the matrix SVD.  The CP decomposition computes the best fitting K rank-1 terms for a user specified $K$.  The Tucker decomposition computes a set of orthonormal mode matrices and may be thought of as performing a multilinear rank -- rank-$(R_1,\dots, R_M)$.  One can write a CP decomposition using a product of terms similar to Tucker. Tucker can be written as a sum of rank-1 terms. However, a CP is not an orthonormal mode matrix computation and Tucker is not a rank decomposition.  FYI, CP is not a rank decomposition either.  

+ "In recent years, tensor networks (Cichocki et al., 2016) have been introduced, which can intuitively and flexibly design the structure including tensor train decomposition (Oseledets, 2011)..."

Hmmm ...  Tensor networks are a particular type of visualization of tensor algebra data analysis.  Tensor Ring is a sequential version of the M-mode SVD which performs a parallel Tucker.  Tensor Train is a sequential one-time iteration of Tensor Ring. 

See sec 5.2: https://arxiv.org/abs/1606.05535 
See fig. 3f: http://web.cs.ucla.edu/~maov/CausalDeepLearning.pdf

+ How is an m-projection computed? What is an e-flat?
+ What do the vertical red dashed lines in Fig 4 and 5 mean?
+ Why use natural gradients? Why not use simulated annealing or stochastic gradient descent?
+ What are natural parameters?
+  However, the indices $i_d$, $i^{'}_d$, $j_d$ and $I_d$  are used inconsistently.  In tensor algebra, upper case italics are used to represent upper bounds. For example, $1\le d\le D$ and $1\le r_d \le {\bar R}_d \le R_d$. 
In eq 1. $i^{'}_d$
 is used as a variable and $i_d$ is used as an upper bound, but in the text above the upper bound is $I_d$ . The Mobius function uses $j_d$ as a variable, $i_d$ as a lower bound and $i^{'}_d$
 as an upper bound.

+ Please define $\mu$  What do the superscript and subscripts mean?

+ "a tensor as an undirected graph, whose nodes correspond to matrices or tensors and edges are the modes of summation in tensor products"

I see both square and circle nodes that are labeled with indices, matrices and tensors. Some are filled in and others are not. What is the difference?  I would like to see every edge and node labeled with the mathematical operators and variable names.

Missing References :

+ Shashua and Hazan wrote the first non-negative tensor decomposition paper.

@inproceedings{shashua2005non,
  title={Non-negative tensor factorization with applications to statistics and computer vision},
  author={Shashua, Amnon and Hazan, Tamir},
  booktitle={Proceedings of the 22nd international conference on Machine learning},
  pages={792--799},
  year={2005}
}

+ Vasilescu and Terzopoulos introduced tensor factor analysis for computer vision, computer graphics and machine learning and decomposed data tensors into their causal factors of data formation.  The TensorFaces paper introduces a parallel Tucker decomposition -- the M-mode SVD (MPCA).

@inproceedings{Vasilescu2002,
  author =	 "M. A. O. Vasilescu and D. Terzopoulos",
  fullauthor =	 "M. Alex O. Vasilescu and Demetri Terzopoulos",
  title =	 "Multilinear Analysis  of Image Ensembles: {T}ensor{F}aces",
  booktitle =	 "Proc. European Conf. on Computer Vision (ECCV 2002)",
  address =	 "Copenhagen, Denmark",
  month =	 "May",
  year =  "2002",
  pages = "447-460"
}

+ Multilinear projection estimates the causal factors of data formation from unlabeled test image(s) given an estimated forward model:

@INPROCEEDINGS{Vasilescu2011, 
author={M. A. O. Vasilescu}, 
booktitle={Proc. IEEE Inter. Conf. on Automatic Face Gesture Recognition (FG 2011)},
title={Multilinear projection for face recognition via canonical decomposition}, 
year={2011}, 
pages={476-483}, 
doi={10.1109/FG.2011.5771445}, 
month={Mar},}

+ Tensor causal factor analysis paper that split a data tensor into data blocks and specifies which data blocks share the same mode matrix and hence contribute to the mode matrix computation.

@inproceedings{Vasilescu20,
    author={Vasilescu, M. Alex O. and Kim, Eric and Zeng, Xiao S.},
    booktitle={2020 25th International Conference of Pattern Recognition (ICPR 2020)},
    title={Causal{X}: Causal e{X}planations and Block Multilinear Factor Analysis},
    year={2021},
    location={Milan, Italy},
    month={Jan},
    pages={10736--10743},
    url={http://web.cs.ucla.edu/~maov/CausalX_Block_Multilinear_Factor_Analysis_ICPR2020.pdf}
    
+ Panagakis etal 2021 are describing papers that are performing tensor regression. The paper contains multiple misconceptions that have been floating around in the ML tensor community.  The authors may wish to address those misconceptions.   Alternatively, the authors may want to reference other cleaner papers by the same authors, such as:

 @article{Kossaifi20,
  author  = {Jean Kossaifi and Zachary C. Lipton and Arinbjorn Kolbeinsson and Aran Khanna and Tommaso Furlanello and Anima Anandkumar},
  title   = {Tensor Regression Networks},
  journal = {Journal of Machine Learning Research},
  year    = {2020},
  volume  = {21},
  number  = {123},
  pages   = {1-21},
}

+ Original light field/lumigraph papers:

@inproceedings{gortler1996lumigraph,
  title={The lumigraph},
  author={Gortler, Steven J and Grzeszczuk, Radek and Szeliski, Richard and Cohen, Michael F},
  booktitle={Proceedings of the 23rd annual conference on Computer graphics and interactive techniques},
  pages={43--54},
  year={1996}
}

@inproceedings{levoy1996light,
  title={Light field rendering},
  author={Levoy, Marc and Hanrahan, Pat},
  booktitle={Proceedings of the 23rd annual conference on Computer graphics and interactive techniques},
  pages={31--42},
  year={1996}
}

TensorTexture models the bidirectional texture function (a type of 6D light field - images of a scene are captured as the viewpoint and illumination were varied).  This is the first tensor paper in computer graphics.

@article{Vasilescu04,
  author =	 "M. A. O. Vasilescu and D. Terzopoulos",
  title =	 "{T}ensor{T}extures: {M}ultilinear Image-Based Rendering",
  journal =	 "ACM Transactions on Graphics",
  volume =	 "23",
  number =	 "3",
  month =	 "Aug",
  year =	 "2004",
  pages=	 "336-342",
  note =     "Proc. ACM SIGGRAPH 2004 Conf., Los Angeles, CA",
}
summary_of_the_review: The paper introduces a modified Legendre decomposition that subdivides a data tensor into multiple tensor blocks to approximate a tensor whole.  It also introduces an interaction tensor network diagram for better visualization.  However, the paper is missing an algorithm, a concrete example, some important discussions, and comparisons to prior work.

I look forward to reading the final version of this paper when all outstanding issues discussed above are fixed.

______
______
______
______
______
---  UPDATED REVIEW  given the authors' response:  ---
---
The paper's title, text, fig 3, and experiment description originally led me to believe that a tensor was approximated by multiple bodies  that interact through a subset of modes, such as the parts that make up a body. I assumed that the division into multiple bodies was achieved by setting some of the natural parameters of a modified Legendre decomposition to zero which was then followed by factorization of each body with a Legendre decomposition. The authors informed me that this was not the case.  

The paper does state that it approximates a data tensor by formulating a "many-body approximation as a special case of Legendre decomposition by setting some of the natural parameters to be zero" (pg.2, paragraph 2).  However, the multi-body language is unrelated to the multi-bodies/particles one encounters in physics and unrelated to the multi-bodies/parts that make up an object.  Apparently, the "bodies" and "interactions" are artifacts of a sequential implementation that do not have a  physical meaning and do not hold true if one employs a parallel implementation.  Despite repeated requests, the authors have refused to define the meaning of a 2-body or N-body.  Why?

The name "interaction diagram" is a misnomer. A more appropriate name is an operation order diagram.   All the mode interactions are active in a Tucker decomposition which is self-evident in an M-mode SVD (parallel Tucker), but it is not immediately apparent if one looks at the operation order of a tensor ring (sequential Tucker). In Fig. 2, "the mode interaction diagram" for a Tucker decomposition depicts the operation order of a sequential implementation, ie a tensor ring. 

The algorithm in the supplement is the natural gradient descent algorithm that does not integrate equations 1,3,5.  The equations are in the main body of the paper and contain undefined variables that would have had to be specified if they were part of the algorithm.  It would have been nice if the authors would have explained the meaning of the superscripts and subscripts which can vary from author to author.

The experiments are inconclusive. The authors have not solved any AI/ML/CV/CG problems. Hence, it is unclear if their approach is useful. The authors have asserted that their approximated images are still usable, but have not displayed any synthesized or reconstructed images.  Why?


Conclusion:
---
The authors' claim that they are performing a multi-body approximation is unsubstantiated.  The terms "multi-bodies" and "interactions"  are unrelated to the multi-bodies/particles in physics and these terms seem to be artifacts of a sequential optimization.  Beyond the inappropriate nomenclature that is misleading, the paper talks in generalities without being grounded in a concrete AI/ML/CV/CG example and without providing any intuition.  The algorithm is missing many details. 

Currently, this paper is not ready for publication at ICLR.


Other comments:
---
1. The title is misleading and a type of flag planting.  The authors are not performing a multibody approximation where the bodies are physics particles or the parts of an object.  Hence, a more appropriate title might be "Legendre Tensor Decomposition envisioned as an M-body Approximation," and even this is a stretch.

2. > Our approach, describing interactions between modes using energy functions, is different from existing methods that focus on interactions between mode matrices (Vasilescu & Terzopoulos 2002; Vasilescu 2011) or block tensors (Vasilescu et al. 2007)

The above is inaccurate.  Object parts may not interact with each other at all, or may interact with each other through a subset of modes.  The following is a more accurate description:

Tensor factor analysis was introduced in computer vision with the TensorFaces papers (Vasilescu and Terzopoulos 2002; Vasilescu 2011) and in computer graphics with the TensorTexture paper (Vasilescu and Terzopoulos 2004). Vasilescu etal employ energy functions and focus on interactions between the many bodies that make up a tensor whole, such as the parts that make up an object (Vasilescu etal 2021; Vasilescu ICPR 2022). 

By comparison, we define multi-body as  ...  (please provide a definition).

___
___

Given the other glowing review ratings, it is unlikely that there will be a reply forthcoming to any of my inquiries. Hence, I have gone ahead and just downgraded the paper. Best of luck and congratulations on your ICLR 2023 paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a new kind of tensor decomposition (TD) framework for non-negative tensors which is inspired by many-body interactions in physics. Unlike most traditional TDs which are hard to fit since they correspond to non-convex optimization problems, the proposed framework corresponds to a *convex* optimization problem. The authors propose a gradient descent-based scheme for solving it.
Another notable feature of the proposed framework is that it *does not* have a notion of rank---a parameter which is difficult to choose in more standard TDs. 

### Update after rebuttal ###

The authors have done a good job of addressing all the issues I brought up in my original review. The optimization scheme makes sense now. What the various n-body constraints imply in terms on constraints on the $\theta$ parameters has also been clarified. The connections to other decompositions is clearer. The explanation of physics terms has also been improved which makes the paper more readable for non-physicists. 

With these improvements, I think the paper is good enough for publishing in ICLR. I have increased my score from 3 to 8.

A few minor things I noticed when reading through the paper again after the rebuttal that the authors can fix in the camera ready version as appropriate:
- When mentioning natural parameters and dual flatness for the first time, it would be great if you provided a reference to the supplement where these things are discussed more in depth.
- In Eq (8), the order of the two summations should be changed, right? The first one should be $\sum_{k=1}^D \sum_{m=1}^{k-1}$ instead of what it is now, and the second one should be $\sum_{k=1}^D \sum_{m=1}^{k-1} \sum_{p=1}^{m-1}$.
- In sentence above Eq (10), there's a word missing (indicated in bold): "If all energies greater than 2nd-order or those *greater* than 3rd-order...". 
- In the various places where you refer to the supplementary material, it would be great if you indicated which section in the supplementary material.
- Section "Synthetic data" at the bottom of page 8, there's a stray period that should be removed. ("...are shown in Figure 4 (a)*.*, and those...")
- Section "Real data" on page 9: You should refer to Fig 5 at some point, since it contains the results for this experiment.
- On page 14, in one of the paragraph headings: "Flattness" -> "Flatness"
strength_and_weaknesses: **--- Strengths ---**

**S1.** The novelty of the proposal is a strong strength. I haven't seen anything like this proposal before. I think Sec 2.5 explains the novelty in a nice way: The framework does rank-1 decompositions, where the notion of "rank-1" is now generalized beyond the traditional tensor rank.

**S2.** The fact that the decomposition corresponds to a *convex* problem is another considerable strength, since most other tensor decompositions correspond to notoriously difficult non-convex optimization problems.

**S3.** The fact that no rank parameters need to be chosen is another important strength. Choosing an appropriate rank is typically also difficult for more traditional tensor approaches. Intuitively, it seems like it might be easier to choose appropriate levels of interaction between modes in the proposed framework than it is to choose rank in more traditional approaches.

**S4.** Although the experiments are not very comprehensive, the speed-up results in the few experiments that are provided are impressive.

**--- Weaknesses ---**
 
The proposal in the paper is very interesting. However, unfortunately there are points that aren't explained properly and that are difficult to follow.

**W1.** The optimization procedure---arguably a key advantage of the method---is difficult to understand. 
- Concepts like "natural parameters", "flats" and "natural gradient" are never properly defined. Since most of the ICLR audience are unlikely to be familiar with these concepts, they should be explained. If there's not enough room in the main paper, this could be put in the appendix.
- The vectors $\theta^B$ and $\eta^B$ are, as far as I can tell, length-$|\mathcal{B}|$ vectors that contain the entries from $\theta$ and $\eta$ corresponding to indices in $B$, i.e., indices for which $\theta$ is zero: $\theta_b = 0$ for all $b \in B$. With this in mind, it seems like the stepping scheme in Eq (6) only updates entries in $\theta$ that are already zero. For a distribution $Q_1$ to lie in the e-flat as shown in Fig 1 (a), doesn't all entries $\theta_b$ for $b \in B$ have to be fixed to zero? In that case, why are those entries updated in Eq (6)? It seems like all this optimization procedure does is update parameters we know should be zero, while leaving the other parameters unchanged...
- The complexity analysis in Sec 2.6 is hard to follow since it doesn't refer back to the particular computational steps discussed earlier.
- What termination criteria is used in the gradient descent method? This is important when interpreting the experiment results.

**W2.** The practical connection between the $\theta$ parameters (e.g., in Eq (7)) and the $H$ parameters (e.g., Eq (9)) isn't explained well. The optimization is done over the parameters in $\theta$, but the conditions (e.g., set $n$-body interactions to zero for $n \geq 3$) are imposed on the $H$ parameters, which are sums of $\theta$ parameters. It is not clear to me how such conditions on the $H$ parameters translate to conditions on the $\theta$ parameters. For example, if $H_{i_k, i_m}^{(k,m)}$ is zero, then what indices should be added to the zero-index set $B$? The parameter count in Eq (15) counts the number of parameters in the $H$ factors---how does this relate to the number of $\theta$ parameters that are actually being optimized?

**W3.** The comparison between specific $n$-body approximations and conventional tensor decompositions is vague. For example, in Sec 3 you say "if we impose that decomposed factors can be represented as products with hyper-diagonal tensors $\Omega$, this decomposition is equivalent to a cyclic two-body approximation." It's not clear to me what this means. How should the symbol $\simeq$ in the figures be interpreted? Since fitting a tensor ring is a non-convex problem, it's clear that they can't be equivalent.

**W4.** Examples of things throughout the text that aren't well-defined/hard to follow:
- Sec 2.1: "dual flatness and orthogonality" of coordinate systems. What does this mean?
- Sec 2.3: "element-wise cyclic product of matrices". 
- While terms like "partition function" may be standard in physics, they will be unfamiliar to most members of the ICLR audience.
clarity,_quality,_novelty_and_reproducibility: **--- Clarity/Quality ---**

The quality of writing is overall good, but clarity is lacking. This is in fact the main issue of the paper; see details under weaknesses.

A few other more minor points relating to clarity/quality:
- The 3rd listed contribution in Sec 1 ("We empirically show that ... reconstruction errors.") is vague. What does "more efficient" mean? Fewer parameters, less computation time?
- In Eq (7), the examples are, respectively, 1-, 2- and 3-body parameters, right? It would be good if this was clarified.
- Above Eq (10), by "substituting 0 for energies greater than 2nd and 3rd order", do you mean setting H^{(l_1, \ldots, l_n)}_{i_{l_1}, \ldots, i_{i_n}} = 0 for  $n \geq 2$ and $n \geq 3$, respectively? Please clarify. 
- Below Eqs (10), (11), should the terms $/6$ and $/4$ in the definition of $Z$ be removed for correct scaling, since you're already taking 6th and 4th order roots?
- On page 5, the statement "$\mathcal{P}$ is approximated by the elementwise product of an $n$-order tensor for $n=2$ and $n=3$" is confusing. This makes it sound like you're doing Hadamard product between 2- and 3- way tensors, which would result in a 2- or 3-way tensor $\mathcal{P}$.
- Below Eq (15), should $|\mathcal{B}|/|\mathcal{R}|$ be $I/R^2$?
- Below Eq (18): reconstracted -> reconstructed

**--- Novelty ---**

The proposed framework seems highly novel to me. This is the main strength of the paper.

**--- Reproducibility ---**

It is hard to fully understand the method from the paper. There is code, but it doesn't appear to be well-documented.
summary_of_the_review: I think the proposal itself is novel and interesting. It has many benefits like a convex optimization formulation and a lack of rank parameters to tune. Additionally, the experiment results (speed-ups in particular) are promising. These considerable strengths are unfortunately brought down by lack of clarity in the text. For example, the gradient descent optimization scheme is hard to follow, the connection between various parameters is not explained properly, and the connection between the presented framework and more traditional decompositions is not explained well. Also, the use of various physics terms that are non-standard in ML and numerical linear algebra add to the confusion. 

I think once these clarity issues are sorted out, this will be a strong and interesting paper. I encourage the authors to work on improving the clarity, and I hope that my comments will be helpful for doing that.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, a new tensor decomposition for nonnegative tensors is proposed interpreting the tensor as a probability measure over a multidimensional discrete space given by its indices commonly known as a random field. 
The authors used tools from Physics in which an Energy function is defined containing terms that corresponds to potentials associated to first order, second order, etc. interactions between variables in the random field. 
In this model, instead of using the concept of rank of the decomposition, the complexity is regulated by the order and map of interactions between variables (indices). 
For the optimization of the model to fit it to some data, they applied Information Theory tools so that the optimization problem is reduced to a convex one and a natural gradient-based algorithm is derived to minimize a Kulback-Liebler distance.
The paper demonstrates that, for example, by considering a cyclic 2-order interaction the obtained model is equivalent to a Tensor Ring (TR) network.
Experimental results using only a cyclic 2-order interaction model are included demonstrating the good properties in terms of accuracy and computation time of the proposed learning algorithm.

strength_and_weaknesses: Strengths:
-	Original model with nice properties such as its rank-free feature and the convexity of the associated optimization problem.
-	Strong theoretical ground based on Information Theory is provided.
-	Very well written and clear presentation of the model and algorithm

Weaknesses:
-	The model only applies to nonnegative tensors, which limits its applicability to many tensor data problems. Current title: “MANY-BODY APPROXIMATION FOR TENSORS” suggests a general model. A more appropriate title would be “MANY-BODY APPROXIMATION FOR NONNEGATIVE TENSORS”
-	The equivalence between the proposed model (cyclic Many Body) and Tensor Ring (TR) should be explored more deeply. For example, given a cyclic Many Body model, it is shown that it is equivalent to a TR network. However, it is not clear what is the rank of the obtained TR model. Also, is the obtained TR-rank the minimal rank? Is there a way to check the actual rank of the proposed model? 
-	The experimental results are rather limited:
1)	They only consider a cyclic 2-order interactions model in the experiments, which is equivalent to a TR network. I think more experiments considering higher order interactions would provide rich insights about the expression power of the new model compared to classical tensor decompositions.
2)	I found the experiments with totally random tensors not very useful because in that case there is no structure on the data tensor itself. The value of this experiment, if any, should be better explained in the paper.
3)	The results on real datasets are not properly discussed. The behavior of the proposed algorithm is not well understood. For example, the error on the 4DLFD dataset is larger than other algorithms without any clear explanation about this result. In general, it seems that HALS always provides the best results in terms of error. Is there any explanation about it?

Minor issues:
-	There is a typo or a problem of visualization in Figure 1. The KL distance should be between P and \bar{P}.
-	In section 2.5, I think the following sentence needs to be fixed: “any tensor P can be represented by …” -> “any tensor P can be approximated by … ”, because in general not any tensor can exactly represented as a rank-1 tensor.

clarity,_quality,_novelty_and_reproducibility: The paper is very clear, and the concept, models and algorithm are original and well supported by theory.
summary_of_the_review: A novel tensor decomposition for nonnegative tensors is proposed and an efficient algorithm supported by information theory ideas is provided. Although the experimental evaluation could be highly improved, I think the community working on tensor decompositions will appreciate this work because it provides new models that are rank-free and with provable convergent algorithms.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper discusses and proposes a tensor decomposition based on the Legendre decomposition of non-negative tensors which represent probability distribution. The decomposition is parametrized in terms of the $\theta$-parameters, which are essentially the logarithm of the probability up to a certain triangular-type linear transformation. 

Strengths: Interesting observation is that the optimization problem (once the set of non-zero indices is fixed) can be written as a convex optimization, which is not true, for example, for other tensor decompositions. 

Weaknesses. The representation is a product representation, without any sum. So, after taking the logarithm, we get a linear model with some non-negativity constraints (but I admire that the model itself is interesting, but seems it has been proposed before).

There are also some comments about connection to tensor ring decomposition. The properties of the tensor-ring decomposition are completely different from the tensor-train: since the tensor network for the tensor ring decomposition has cycles, this format is in general unstable, whereas tensor-train is stable. The tensor ring for particular tensors can give better compression.

Overall, I don't see which representation should be more stable, it probably will depend on the application.  The connection to tensor networks and the expressive power is not given, as well as more illustrative and practical experiments. 


Numerical examples are on very strange and non-standard datasets, which can not be considered characteristics. One can take the standard functions, such as $\sin(x_1 + \ldots + x_d)$ or $\frac{1}{x_1 + \ldots + x_d}$ for testing the accuracy of different decompositions, rather than obscure example with no clear answers.  

justification_for_why_not_higher_score: Although the paper has received some higher scores, based on my own reading I can not recommend acceptance. I think, that the numerical experiments should be stronger, and the novelty is limited (decomposition is known before as well as the algorithms). No non-trivial algorithms for selecting the set for $B$ is given, so when writing this lines I am quite confused by the nice evaluation by two of the reviewers. I think the algorithm has potential, but it has to be tested and evaluated at a different level.
Decision: Reject