Venue: ICLR 2018 conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a tensor train decomposition with a ring structure for function approximation and data compression. Most of the techniques used are well-known in the tensor community (outside of machine learning). The main contribution of the paper is the introduce such techniques to the ML community and presents experimental results for support.

The paper is rather preliminary in its examination. For example, it is claimed that the proposed decomposition provides "enhanced representation ability", but this is not justified rigorously either via more comprehensive experimentation or via a theoretical justification. Furthermore, the paper lacks in novelty aspect, as it is uses mostly well-known techniques. 
review_title: Preliminary examination. Lacks novelty.

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper presents a tensor decomposition method called tensor ring (TR) decomposition. The proposed decomposition approximates each tensor element via a trace operation over the sequential multilinear products of lower order core tensors. This is in contrast with another popular approach based on tensor train (TT) decomposition which requires several constraints on the core tensors (such as the rank of the first and last core tensor to be 1).

To learn TR representations, the paper presents a non-iterative TR-SVD algorithm that is similar to TT-SVD algorithm. To find the optimal lower TR-ranks, a block-wise ALS algorithms is presented, and an SGD algorithm is also presented to make the model scalable.

The proposed method is compared against the TT method on some synthetic high order tensors and on an image completion task, and shown to yield better results.

This is an interesting work. TT decompositions have gained popularity in the tensor factorization literature recently and the paper tries to address some of their key limitations. This seems to be a good direction. The experimental results are somewhat limited but the overall framework looks appealing.
review_title: review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper addresses the problem of tensor decomposition which is relevant and interesting. The paper proposes Tensor Ring (TR) decomposition which improves over and bases on the Tensor Train (TT) decomposition method. TT decomposes a tensor in to a sequences of latent tensors where the first and last tensors are a 2D matrices. 

The proposed TR method generalizes TT in that the first and last tensors are also 3rd-order tensors instead of 2nd-order. I think such generalization is interesting but the innovation seems to be very limited. 

The paper develops three different kinds of solvers for TR decomposition, i.e., SVD, ALS and SGD. All of these are well known methods. 

Finally, the paper provides experimental results on synthetic data (3 oscillated functions) and image data (few sampled images). I think the paper could be greatly improved by providing more experiments and ablations to validate the benefits of the proposed methods.

Please refer to below for more comments and questions.

-- The rating has been updated.

Pros:
1. The topic is interesting.
2. The generalization over TT makes sense.

Cons:
1. The writing of the paper could be improved and more clear: the conclusions on inner product and F-norm can be integrated into "Theorem 5". And those "theorems" in section 4 are just some properties from previous definitions; they are not theorems. 
2. The property of TR decomposition is that the tensors can be shifted (circular invariance). This is an interesting property and it seems to be the major strength of TR over TT. I think the paper could be significantly improved by providing more applications of this property in both theory and experiments.
3. As the number of latent tensors increase, the ALS method becomes much worse approximation of the original optimization. Any insights or results on the optimization performance vs. the number of latent tensors?
4. Also, the paper mentions Eq. 5 (ALS) is optimized by solving d subproblems alternatively. I think this only contains a single round of optimization. Should ALS be applied repeated (each round solves d problems) until convergence?
5. What is the memory consumption for different solvers?
6. SGD also needs to update at least d times for all d latent tensors. Why is the complexity O(r^3) independent of the parameter d?
7. The ALS is so slow (if looking at the results in section 5.1), which becomes not practical. The experimental part could be improved by providing more results and description about a guidance on how to choose from different solvers.
8. What does "iteration" mean in experimental results such as table 2? Different algorithms have different cost for "each iteration" so comparing that seems not fair. The results could make more sense by providing total time consumptions and time cost per iteration. also applies to table 4.
9. Why is the \epsion in table 3 not consistent? Why not choose \epsion = 9e-4 and \epsilon=2e-15 for tensorization?
10. Also, table 3 could be greatly improved by providing more ablations such as results for (n=16, d=8), (n=4, d=4), etc. That could help readers to better understand the effect of TR.
11. Section 5.3 could be improved by providing a curve (compression vs. error) instead of just providing a table of sampled operating points.
12. The paper mentions the application of image representation but only experiment on 32x32 images. How does the proposed method handle large images? Otherwise, it does not seem to be a practical application.
13. Figure 5: Are the RSE measures computed over the whole CIFAR-10 dataset or the displayed images?

Minor:
- Typo: Page 4 Line 7 "Note that this algorithm use the similar strategy": use -> uses
review_title: Interesting but limited contribution and validation

Meta-review: This paper proposes a new way of learning tensors representation with ring-structured decompositions rather than through Tensor Train methods. The paper investigates the mathematical properties of this decomposition and provides synthetic experiments. There was some debate, with the reviewers, about the novelty and impact of this method, where overall the feeling was this work was too preliminary to be accepted. The idea, from my understanding, is interesting and would benefit from discussion at the workshop track, but the authors are investigated to make a stronger case for the novelty of this method in any further work and, in particular, to consider showing empirical improvement on "real" data where TT methods are currently applied.
Decision: Invite to Workshop Track