Title: Non-Convex Tensor Recovery from Tube-Wise Sensing
Venue: NeurIPS 2025
Accepted: yes
Keywords: Tensor compressive sensing

non-convex

sample complexity
Abstract: In this paper, we propose a novel tube-wise local tensor compressed sensing (CS) model, where sensing operators are independently applied to each tube of a third-order tensor. To recover the low-rank ground truth tensor, we minimize a non-convex objective via Burer–Monteiro factorization and solve it using gradient descent with spectral initialization. We prove that this approach achieves exact recovery with a linear convergence rate. Notably, our method attains provably lower sample complexity than existing TCS methods. Our proof leverages the leave-one-out technique to show that gradient descent generates iterates implicitly biased towards solutions with bounded incoherence, which ensures contraction of optimization error in consecutive iterates. Empirical results validate the effectiveness of GD in solving the proposed local TCS model.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn conventional Tensor Compressed Sensing (TCS), the sensing operator $A_k$ acts on the entire tensor $X$, which has several drawbacks: (1) it is not scalable for large-scale data, (2) it is not suitable when the data size is variable, and (3) block-wise $Ak$ is preferable for privacy-preserving applications. To address these issues, slice-wise TCS approaches have been proposed. Inspired by existing slice-wise methods, the authors develop a tube-wise TCS method.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n\n- [**S1**] Despite the non-convex nature of the problem, the authors provide a theoretical guarantee that their proposed method can recover an exact low-rank tensor $X$ at a linear rate under the following conditions:\n  - A sufficient number of measurements are available.\n  - Each frontal slice of X has a very small rank.\n  - X satisfies an incoherence condition.\n  - The algorithm is initialized by spectral initialization.\n\n- [**S2**] The sample complexity is shown to be proportional to 1/n, which is a favorable result.\n\n- [**S3**] The introduction clearly motivates the problem by highlighting limitations in existing methods and proposing a justified new approach. The authors clearly explain the differences from prior work and the novelty of their own method, positioning their research well within the existing literature. The writing is easy to follow.\n\nOverall, the paper is well-written, and these strengths suggest the paper makes a strong contribution to the community.\n\n### Weaknesses:\n\nThe following are minor concerns, and I do not consider them grounds for rejection, provided the authors make slight revisions.\n\n-  [**W1**] Although the method assumes a small tubal rank and the sample complexity is proportional to r^5, real-world datasets in the ML community often do not guarantee such low ranks. The authors assume rank-5 tensors in their experiments. It would be helpful to illustrate practical use cases where such an assumption is realistic and beneficial.\n\n- [**W2**] The paper focuses only on third-order tensors. While I am not sure if this may be standard in signal processing, tensors of arbitrary order are frequently discussed in the ML community, such as tensor networks. I recommend that the authors either justify their focus on third-order tensors or discuss how their approach could be generalized for arbitrary order as future work.\n\n- [**W3**] To avoid misunderstanding, Assumptions 1 and 2 should be more clearly explained in the introduction. The authors claim: “We prove that despite being nonconvex, with a sufficient amount of measurements, applying gradient descent with proper spectral initialization finds the ground truth X at a linear rate.” However, this result holds under the additional assumptions that each frontal slice has a sufficiently small rank (assumption 1) and the tensor satisfies the incoherence condition (assumption 2) . These assumptions are currently not clearly described in the abstract or introduction.\n\n- [**W4**] In the checklist, the authors do not include a justification for their claims. While this is likely acceptable given the theoretical nature of the paper (with clear assumptions, proofs, and also source code), please ensure this section is properly filled in the camera-ready version.\n\nQUESTIONS:\n### Suggestions:\n\nI hope the following suggestions help improve the clarity and readability of the paper:\n\n- [**S1**] Please explicitly state the dimension of the vector y in Equation (6). While I understood that y_ij corresponds to y(i,j,:) as defined in Definition 1, it took time for me to understand it.\n\n- [**S2**] Consider describing the tensor size of R in Equation (11). Where is the set O_R defined?\n\n- [**S3**] The notation for truncated SVD is inconsistent: it appears as “t-SVD” (Line 137) and “T-SVD” (Algorithm 2). Please unify this notation throughout the paper.\n\n- [**S4**]  Please explain the meaning of the check symbol and the variable “r” (tubal-rank) in the caption of Table 1.\n\n- [**S5**] At first glance, Equations (6) and (7) give the impression that the gradient is independent of y, which feels a bit odd. Of course, I understand that X itself depends on y, but I think it would be helpful to explicitly mention this dependency in the paragraph for clarity.\n\nLIMITATIONS:\nyes (but please fill in the justification in the checklist)", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-05"}

comments: SUMMARY:
This manuscript propose a new tensor compressed sensing (TCS) model with local tube-wise strategy. The method works by minimizing a nonconvex objective via Burer-Monteiro factorization. Besides, the authors prove that this method achieves exact recovery with a linear convergence rate.

STRENGTHS AND WEAKNESSES:
This paper introduces a local tube-wise method under the tensor product framework. The corresponding detailed definitions and theory are provided. Additionally, experiments analyze about convergence, initialization, and sample complexity.

However, this paper lacks sufficient experiments to demonstrate the contributions of this work. More comparative methods need to be added. Furthermore, the motivation from slice-wise to tube-wise requires further description.

QUESTIONS:
1. The slice-wise method, i.e., Alt-PGD-Min, and the proposed method are both local TCS models. What are the advantages of the tube-wise method compared with the slice-wise approach?

2. This paper provides tube-wise sample complexity. However, the experiments do not involve a comparison of the sample complexity with other methods. How can the effectiveness of the proposed tube-wise sample complexity be sufficiently verified?

3. There was no comparison with other benchmark methods in the experiments. Moreover, the effectiveness of the proposed method in practical application scenarios remains to be verified.

4. The author claims that tube-wise is more flexible. How is this flexibility demonstrated and supported in the paper?

5. Some typos exist, e.g., “GD” in abstract and several definitions lack references.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper consider a tensor compressive sensing problem, with tube-wise linear measurements as defined in Definition 1, and under a low tubal rank assumption. They show that using gradient descent with a Burer-Monteiro factorization of the matrix is able to recover the original tensor and provide several theoretical guarantees.\n\nSTRENGTHS AND WEAKNESSES:\nThe topic of the paper is quite interesting. Although the applications of the paper are limited, as the model considered by the authors seldom arises in applications, the mathematical analysis behind the results makes up for that.  They provide a very complete analysis of the algorithm, under random settings, and obtain results in line with other lines of work. Moreover, they obtain an interesting dependence between sample complexity and the dimensions of the tensor $(O(r^5 n_3/\\min(n_1, n_2))$, where the tensor is $n_1\\times n_2 \\times n_3$.\n\nQUESTIONS:\n- Line 48: Tubal rank is yet to be defined. Here the authors should point to the definition of tubal rank (Definition 8).\n- Line 129: The definition of $\\mathcal{I}$ is inconsistent. The authors alternate between using $\\mathcal{I}$ and $\\mathcal{I}_n$. I think dropping the index and using $\\mathcal{I}$ is fine as it is clear its definition in the context.\n- Line 146: It should be made more clear at this time what the second term does in the loss does. In particular I believe it should be mentioned that the minima are the balanced factors defined in 162, up to an orthogonal transformation (which is referred in equation (11)).\n- The norm in (10) is for a matrix, yet the authors refer to a tensor norm.\n- In section 4, let us suppose that another random model for $\\mathcal{Z}$ also possesses an RIP condition. Would that imply the results obtained also hold for that random model?\n- The authors consider the $t$-rank of a tensor, which is the $M$-rank of a tensor when $M$ is the DFT matrix. Would the results obtained hold for other $M$-products?\n\nLIMITATIONS:\nThe authors address the limitations adequately.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper studies spectral-initialized factorized GD for tensor sensing problem under the so-called local measurement model, where each one-dimensional slice along the third axis $\\mathcal{X}(i, j, :)$ is measured by i.i.d. Gaussian matrices. Under a somewhat stronger coherence condition than the usual one, it is proved that factorized GD converges linearly when the number of measurements $\\gtrsim r^5  n_3 n$, where it is assumed $n=n_1=n_2$.

STRENGTHS AND WEAKNESSES:
## Strengths
The local measurement model is relatively new (built upon the idea of [58]). The overall approach appears rigorous, though I have not checked all the details in the proof.

## Weaknesses
- It is not clear whether the local measurement model is of significant practical relevance. It would be beneficial to discuss potential real-world applications. 
- It is not adequately addressed in the paper whether the local measurement model is simpler or harder to deal with than the more common models. Given the similarities of the proof to well-known proofs in the literature, the main technical contribution of the paper should be highlighted better.

QUESTIONS:
Please refer to the Weaknesses part.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03