Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a tensor decomposition leveraging contrastive self-supervised learning based on the data augmentation for positive samples and the unbiased estimation of negative samples, which can align with downstream classification tasks. The proposed method optimizes the objective by using an alternating least squares algorithm. The experimental results using four real-world tensor datasets show that the proposed method achieves better accuracy on the downstream classification tasks using a much smaller number of parameters than several comparison models including tensor-based models, self-supervised models, and auto-encoder models. 
strengths_and_weaknesses: Originality:
The proposition of contrastive learning using data augmentation for positive samples will greatly contribute to the research community of tensor decomposition. Although the proposed method is a fundamentally unsupervised method and able to utilize a large number of unlabeled data, it should be compared with several works of supervised tensor learning such as [Tao et al., 2005].

[Tao et al., 2005] Dacheng Tao, Xuelong Li, Weiming Hu, Stephen J. Maybank, Xindong Wu: Supervised Tensor Learning. ICDM 2005: 450-457

Quality:
The problem setting makes sense. The proposed method is convincing along with theoretical discussion.

Clarity:
This paper is well-organized and clearly written.

Significance:
The classification accuracy shown in Table 2 is not significantly different with or without the empirical self-supervised loss. Moreover, as described in the appendix, the proposed method is not sensitive to beta. There is room to make the proposed method more significant.

questions: Have you compared the proposed method with some supervised tensor learning by using the training/test dataset?
limitations: As mentioned above, the classification accuracy is not significantly different with or without the empirical self-supervised loss in the experimental results. It should be made more clear the situation the proposed method is strongly needed.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors present  "augmented" tensor decomposition, which utilizes an objective that is designed for downstream tasks (by way of "alignment") in addition to data reconstruction ("fitness"). Hence, the quality of the decomposition is determined not only by reconstruction error, but by correct clustering on features. The authors contribute an algorithm for implementing this approach and motivate the significance with a comparison to existing CP decomposition approaches. The authors' algorithm builds on Alternating Least Squares (ALS), a typical method in CP decomposition algorithms.
strengths_and_weaknesses: Strengths:
- The approach is well motivated. The novelty and potential significance of the authors' contributions is made immediately clear in Section 1.
- The contributions are clearly defined and include: a method to generate dissimilar data samples in the absence of labels.
- Presentation is clear and easy to follow.
- The experiments on real data use a convincing variety of datasets: there is sufficient variation in dataset size, balance of dimensions, and downstream tasks.

Weaknesses:
- The discussion of classification appears to be wholly motivated by classification of samples using features (eg. classifying patients based on EEG-related features). The nature of the classification task is heavily dependent on the application -- eg. decomposition of DNA methylation data (where it is desirable to do dimensionality reduction on the # of features, which is very large, in addition to clustering patients) is often motivated by simultaneous clustering/classification of features in addition to patients. Would be helpful to clarify whether more flexible classification is feasible/reasonable under this approach.
- The perennial question in CP/Tucker decomposition is how to choose the dimension of latent features (ie. the decomposition rank). Would like to see more discussion on this topic, eg. the stability of this method's performance to perturbations in this regard (ex. is ATD still superior if R is varied within a reasonable range?) 
- The datasets in Section 4 (Experiments) all have a relatively similar # of classes. The results would be more compelling if there was more variation in this dimension.
questions: - Is there a concern about the validity of using classification accuracy to measure model performance, in cases where classes are highly imbalanced?
- Appendix D.6 addresses that ATD's performance improves with larger R. What about its performance relative to the other models in Table 2?

limitations: - Good discussion of limitations. Please emphasize whether the computational complexity compared to existing methods is a limitation or strength. One obvious limitation (given that medical data is heavily used in the experiments on real data) that may be useful to mention is that this method is developed for real-valued data (as opposed to bounded data).
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper propose a novel self-supervised framework for CP Tensor Decomposition, in which an iterative method is developed to enable  an alternative least squares fashion. The experiment results on various datasets show its efficiency.
strengths_and_weaknesses: 
1. incorporating self-supervised learning into CP Tensor Decomposition seems to new to me.
2. the experiments are extensive and promising
3. this paper is well-motivated


questions:  could the authors provide more explaination about the derivation of Eq.(8)?
limitations: no
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In the paper, the authors investigated a new variant of CP decomposition (CPD) for the classification task. Compared to the classic CPD, the new one imposes the self-supervised loss, so better embedding spaces, modeled by latent factors, can be learned by augmenting the training data. 
strengths_and_weaknesses: **Strength:**

- (clarity) The paper is well written. The background, formulation, and algorithm are clearly introduced.
- (originality) Considering the self-supervision loss is new for tensor decomposition, and the experimental results verified the effectiveness of the method.

**Weakness:**

- (quality, clarity) Part of the technical details are not well discussed
    1. Although Thm. 1 proved the convergence for the sub-problem of learning $\mathbf{X}$, only *one* iteration is employed in the whole algorithm as given in the appendix (page 14). It might work well in practice (mentioned in the appendix), but such modification makes me suspect if the whole algorithm enables to converge to fixed points. Careful discussions lack on this point.
    2. It also lacks the discussion (apology if I miss it) regarding how the data augmentation quality impacts the new method's performance.
    3. In the ablation study, the comparison with other tensor baselines should be added. Otherwise, it is hard to fairly evaluate if the proposed method is non-sensitive to the parameters, e.g., the CP rank.
    4. The proposed method's advantage of the number of learnable parameters is not clearly discussed. For example, the deep learning models may run faster than tensor-based ones on GPU even though more parameters are required. If the aim is to evaluate the expressive power of the model, the number of parameters also seems not a good metric, especially when the model architectures are different.
- (significance) The proposed method might be useful for specific tasks such as those in the experiment section. But there are few new ideas or contributions to the tensor community.
questions: 1. What is the running time for training the all methods implemented in the experiment?
2. What happens if the augmented data get unreliable, e.g., low-quality?
3. How other tensor methods are influenced by varying the rank in the experiment?
limitations: The limitation compared with deep learning methods are briefly mentioned, but there is no discussion compared with other tensor-based methods. For example, does the additional loss reduces the interpretability or the expressive power of tensor models?
review_title: null

Meta-review: metareview: The authors proposes augmented tensor decomposition (ATD) to adapt data reconstruction towards the downstream tasks, e.g., appropriate feature clustering.  It leverages data augmentation and self-supervised learning, with optimization accomplished akin to alternating least square (ALS).  Significantly superior performance is obtained in experiment, compared with existing CP methods and ALS.    

All the reviewers, including myself, find the paper a solid contribution to the methodology and analysis. There were a few concerns and clarification requests, and the rebuttal did a good job addressing them. These additional results and insights can be included in the final version of the paper.
confidence: Certain
Decision: Accept