Towards a Geometric Understanding of Tensor Learning via the t-Product
Authors: Andong Wang
Yuning Qiu
Haonan Huang
Zhong Jin
Guoxu Zhou
Qibin Zhao
Venue: NeurIPS 2025 poster ()
Keywords: tensor decomposition
tensor learning
t-SVD
tensor approximation
Primary area: general_machine_learning
TL;DR: We develop a geometric framework for t-product-based tensor learning, define smooth t-manifolds, establish theory for manifold testing and function approximation, and illustrate the approach with a conceptual image modeling application.

Abstract: Despite the growing success of transform-based tensor models such as the t-product, their underlying geometric principles remain poorly understood. Classical differential geometry, built on real-valued function spaces, is not well suited to capture the algebraic and spectral structure induced by transform-based tensor operations. In this work, we take an initial step toward a geometric framework for  tensors equipped with tube-wise multiplication via orthogonal transforms. We introduce the notion of smooth t-manifolds, defined as topological spaces locally modeled on structured tensor modules over a commutative t-scalar ring. This formulation enables transform-consistent definitions of geometric objects, including metrics, gradients, Laplacians, and geodesics, thereby bridging discrete and continuous tensor settings within a unified algebraic-geometric perspective. 
On this basis, we develop a statistical procedure for testing whether tensor data lie near a low-dimensional t-manifold, and provide nonasymptotic guarantees for  manifold fitting under noise. We further establish approximation bounds for tensor neural networks that learn smooth functions over t-manifolds, with generalization rates determined by intrinsic geometric complexity. This framework offers a theoretical foundation for geometry-aware learning in structured tensor spaces and supports the development of models that align with transform-based tensor representations.

Towards a Geometric Understanding of Tensor Learning via the t-Product
Andong Wang RIKEN AIP
Yuning Qiu
Haonan Huang RIKEN AIP
RIKEN AIP
andong wangOriken jp
Brrinn.qiuuriken. jp
haonan huangoriken. jp
Zhong Jin
Guoxu Zhou
China University of Cetroleum-Beiiiiee Karamay inonngin@cuppkcoom edu. en
Guangdong University of Technology gx. gx.zhou@gdut. edu cn
Qibin Zhao *
RIKEN AIP
qibin wwa..zhhhorreeee....
Abstract
Despite the growing success of transform-based tensor models such as the t-product, their underlying geometric principles remain poorly understood. Classical differ- ential geometry, built real-valued function spaces, is not well suited to capture the algebraic and spectral structure induced by transform-basee tensor operations. In this work, take an initial step toward geometric framework for tensors equipped with tube-wise multiplication via orthogonal transforms. We introduce the notion of smooth t-manifolds, defined topological spaces locally modeled on structured tensor modules over a commutative t-scalar ring. This formulation enables transform-consistent definitions of geometric objects, including metrics, gradients. Laplacians, and geodesics, thereby bridging discrete and continuous tensor settings within unified we develop statistical procedure for testing whether tensor data lic near low- dimensional t-manifold, and provide nonasymptotic guarantees for manifold fitting under noise. We further establish approximation bounds for tensor neural networks that learn smooth functions over t-manifolds, with generalization rates determined by intrinsic geometric complexity. This framework offers theoretical founda- tion for geometry-ave -ware learning in structured tensor spaces and supports the development of models that align with consfoommbseed tensor representations.
perspective. On this basis,
Introduction
Tensor-based modeling has emerged as a powerful paradigm for representing and analyzing structured data, with widespread applications in machine learning, computer vision, signal processing, and quantum AI [64, 65, 78, 53, 23, 39, 61, 62, 83]. Among these, the t-SVD framework [33, 32] introduces a unique t-scalar representation, where each mode-3 fiber (uube) is treated an indivisible algebraic unit called a t-scalar, following a transform-basse based multiplication rule (see Section 2.1 for details). Built this foundation, 2D image be viewed vecect whose entries t-scalars, while a 3D video can be interpreted as a t-matrix [79, 41, 33]. This representation enables low-rank
Qibin Zhao the corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

I-module
(E)
be seen
t-linear space
Sliced smooth manifolds the transform domain
E-odule
K
(A)
t-scular K.
discrete
t-vector in K
Local homcomorphivon
Decompose
Support
form
hypothesis testing of t- t-manifold
Jorm
(D)
I-manifold fitting from noisy data function learning on t- t-manifold
(B)
t-vector K
continuous
E-scalar in Kg
(F) t-manifold
Learning theory of t-product geometry
Figure 1: Overview the proposed t-product geometric framework. (A, B) Discrete and continuous
1-scalars in K, and x form high-dimensional t-vectors (C. D) elements of K Kg (E) A t-module is a t-linear space spanned by t-vectors with coefficients t-scalars. (F) A t-manifold (finitely generated free) t-module (G), generalizing classical manifolds to the transform-base tensor setting. (I) In the transform domain, the I-manifold decomposes into multiple frequency-wise manifolds, which support spectral smoothness. (H) These constructions support the development of learning theory over t-manifolds, including hypothesis testing, fitting from noisy data, and smooth function learning with tensor neural networks.
is defined as smooth space locally homeomorphic
modeling in the transform domain, effectively capturing structured dependencies such as spatial, spectral, and channel-wise correlations [32, 31, 83, 68, 27, 71].
central concept in many t-SVDDbased methods, such as Tensor Robust PCA [41] and Tensor Low- Rank Representation (TLRR) [83], is the structure that be viewed I-module (see Section 2.1 and Fig. 1-E), which serves as an analog of linear space defined over the t-product algebra [30, 6]. For instance, TLRR models utilize t-modules to express the self-representation property of tensor data [83, 68, 69, 30]. While effective, such linear structures are inherently limited in their ability capture nonlinear interactions or encode higher-order geometric properties. Moreover, existing approaches are primarily formulated based on discrete tube structures [83, 68, 69. 30], typically corresponding unne-indexed sequences with finite sampling, such video frames [83]. These methods lack unified perspective that bridges discrete-time tubes [83] and continuous-ting time tube representations [67, 47, 66] within coherent geometric framework.
Recent work has begun to explore geometric aspects of the t-product, including Grassmannian and Stiefel manifolds defined over t-scalars [45, 19]. However, these developments remain narrowly focused and lack a unified theoretical framework. In particular, a general theory of differential geometry that compatible with the t-product and capable of defining smooth structures, Riemannian metrics, geodesics, and other geometric objects in a transform-consisteen manner has yet to be established. At the same time, t-SVD-baedd models have demonstrated strong empirical performance in graph-structured learning tasks [13, 51, 24, 70], where graphs may be viewed as discrete samples from an underlying manifold. Yet, the intrinsic geometric structure of such tensorized data remains largely unexplored.
These observations point important yet underdeveloped direction: developing geometric framework for tensor learning that extends differential geometry to the algebra induced by the t- product. Such framework essential not only for unifying discrete and continuous tensor modeling, but more importantly, for establishing principled theoretical foundation for structure-aware learning in tensor settings. In this work, we make an initial attempt toward this goal by posing the following theoretical questions:
Q1: Can we construct a rigorous differential geometric frometerrrrwwrrrrrrr frumework over 1-scalars that system- atically unifies discrete and continuous tensor representations, and gives rise to well-defied geometric objects such as t-manifolds?
Q2: Can such a framework support the theoretical analysis of smooth function learning on dineeninni t-manifolds from high-dimensional tensor data?

---

To address these questions, we propose a theoretical framework of t-product geometry (see Fig. 1) with the following contributions:
Geometry. We introduce the concept of I-manifolds, a new class of smooth manifolds defined over the t-scalar ring. formalized via for defining waanform-consistenn geometric structures such as tangent spaces and Riemannian metrics (Section 2), directly addressing Q1.
construction. This lays a principled foundation
Learning theory. We develop hypothesis testing and fitting methods for identifying low- dimensional t-manifold structures from sigh-dimensional tensor data, and establish theoretical guarantees for learning smooth functions defined on such manifolds using tensor neural networks. These results offer rigorous theoretical foundation for geometry-awe learning in the tensor setting (Sections 3. 3.123.2,, thereby addressing Q2.
Beyond the theoretical developments, we further illustrate the modeling potential of the proposed framework through a conceptual example image modeling, where bidirectional structures based on the t-product formulation are utilized to enhance clustering and tensor recovery" (Section 4). To our knowledge, this work provides the first framework that formulates tensor learning within differential- geometric celferential-geometricc space defined by the t-product algebra, bridging transform-batt tensor analysis and smooth manifold theory. The appendix provides detailed related work, proofs, algorithms, and experiments.
2 Differential Geometry over the t-Product Algebra
To address Q1, we develop a differential geometric framework over the t-product algebra by intro- ducing class of smooth manifolds, which refer to t-manifolds. These manifolds constructed over the t-scalar ring and support conssssent notions of smoothness and locality. We begin by introducing the algebraic foundations necessary for this formulation.
2.1 Preliminaries on t-Scalar-Based Representation
Discrete t-scalars and the ring K.. A discrete t-scalar is a third-order tensor a E R Ixxexe which may represent a row in a 2D image (with e columns, see Fig. 1-A) [83, 68] or the RGB channels of pixel (when 3) [41]. We define the t-product between t-scalars via orthogonal transform M Rexe (e.g., DCT) [61, 62], letting a b (M + M(b)). where O the Hadamard product. This turns the space into commutative ring [6], denoted K, = R x xx +, *), with identity element == M (1) where 1 e R is the all-ones vector.
multiplication R gixixe
Continuous t-scalars and the ring K. In the continuous setting, a continuous I-scalar is smooth function ʃ C* (R), representing, e.g., continuous-timg signal (see Fig. 1-B) [67]. Let M be unitary operator on C ()) (e.g., multiplication or translation operator). Then multiplication is defined as f g := M ((M (M))), and the ring' is denoted as K == (C*((() +)) For unit element
simplicity, we assume the existence
M -1)
T-vectors and t-module. A r-vector is a tuple of d t-scalars (see Figs. 1-C and D). the discrete case, p p= (p...... Pa) Kd with each p; Rixixe, in the continuous case, f (f1,.... fa) K This allows us to represent structured objects such as image rows, multichannel signals, or time- varying slices [61, 83, 67]. The Kd forms free right module over the t-scalar ring IK (K., K. supporting addition and scalar multiplication defined componentwise [6]. We refer to such space as 1-module, which field (e.g., R or C)) is replaced by the t-product algebra KK (see Fig. 1-F). This t-module structure serves the model space for defining local charts in our construction of t-manifolds (Definition l).
be regarded as natural generalization of linear subspace, where the scalar
"This example intended demonstrate the modeling perspective inspired by the t-product framework rather than to serve as an empirical benchmark for t-manifold learning.
AAternatively, one may define K., over C(R) rather than C (IR), provided that the transform M preserves continuity and ensures closure of the induced multiplication in C(LR), and that the unit element M (1) exists. For instance, when M a translation or smooth multiplication operator, e is smooth and lies C (R). In contrast, if M is the Fourier transform, then = ((t) is tempered distribution lying outside C(R)): this can be addressed by extending the scalar ring the space of distributions D' (R) or adjoining formal identity.

---

2.2 Smooth t-Manifolds with Intrinsic Dimension
We propose the notion of I-manifolds as a generalization of classical smooth manifolds (see Fig. 1- F), which real-valued coordinates are replaced by structured t-scalars from commutative ring K E {IK,, ). and the local structure is modeled by a t-module (see Fig. I-G). This provides a unified algebraic-geometric view of discrete and continuous tensor data.
Definition (Smooth t-Manifold of Dimension p). Let & N31- A topological space M is called a om-soooth t-manifold of intrinsic dimension p over K. if the following conditions satisfied:
(I) Topological structure. M is Hausdorff and paracompact topologiaal space.
(2) Local charts. There exists a locally finite atlas (Un; (ea:PPa).m sP)) A such that each U,, is open in M, and each Pa is homeomorphism onto an open subset Va S KP, where KP denotes the free K-module of rank p (with K E (Ko, K.. The manifold M can be regarded as embedded in higher-rank free K-module K (d 3/ p), equipped with the product topology-Euclidean K K. and Fréchet Kyz.
K
(3) Component-wise smoothness and independence. For any overlapping charts yau and yp with Ue a Up # Ø. the transition map fa o 993 is C-ssmooth in the transform domain, and acts independently' across frequency components. Specifically:
(Discrete case) K = Kg; For each frequency index k I,.... the k-th slice
[M Pa "p o M 1 R² - R
M
a -m-smooth diffeomorphism, and different slices are mutually independent. Consequently, M where each (M is -m-oooothh p-dimensional manifold.
M ((M) M
(Continuous case) K = Kg For each frequency parameter t I,
[M o Pa o pp' o M R R
C (9-smooth, and the map - [M Pa 93 M continuous in the C -toooooyyy Assuming component-wise independence across the transform-domain representation satisfies M(M) @ M dt, where (M(() ]EER forms continuously parameterized family of C smooth manifolds.
This definition generalizes classical smooth manifolds: when K R and M Id, t-manifold reduces to an ordinary 8 -mmooh manifold modeled on R", with standard smooth transition maps. More generally, when K E (Ke, } and the product topology is imposed, the free module K" becomes a trivial C -mooth t-manifold and serves as the local model space for general t-manifolds. Definition thus extends the notion of smooth manifolds to the algebraic setting induced by the t-product, where smoothness is enforced in the transform domain to ensure compatibility with underlying tensor operations. This definition strictly adheres to the core philosophy of the t-SVD [33, 32, 6]: by treating tubes algebraic units and operating in the transform domain, it captures low-rank structures and coherent variations along spectral modes while ensuring frequency wise independence among transform components.
In data modeling, the intrinsic dimension p quantifies the local degrees of freedom required to parametrize the manifold M, while the ambient dimension d corresponds to the number of t-vector coordinates (e.g., image rows or spatial locations). In most applications, we expect >> p. consistent with the manifold hypothesistthe assumption that structured tensor data concentrate near low- dimensional t-manifold embedded in a high-dimensional (-vector space. This hypothesis will be theoretically examined in Section 3.1.
'Throughout this work, t-manifolds defined under the assumption of frequency-wise independence in the transform domain, meaning that local smooth structures are specified separately for each frequency component. This assumption simplifies the definition of differential and geometric objects but rules out cross- trossffreyeency interactions that could model spectral coupling. Relaxing this independence assumption in manners like [63] would allow for more expressive geometric structures and constitutes interesting direction for future research.

---

2.3 Differential Geometry on t-Manifolds
To develop a coherent geometric framework on I-manifolds, classical differential geometry needs to be extended in a manner that respects the algebraic structure induced by the I-product. Traditional geometric objects such as vector fields, differential forms, and Riemannian metrics are defined functions. In contrast, t-manifolds are locally modeled t-scalars drawn from transform-basse commutative ring K (c.g., Kr or Kx,,, rather than R. This shift leads to a basic issue: since t-scalars structured objects, such discrete tubes smooth functions, basic operations including directional derivatives and linear functionals cannot be defined by direct scalar-basee analogies. Instead, they must be formulated in a manner consistent with the underlying transform algebra.
over
To address this, we adopt sheaf-theoretic formulation rooted in commutative algebra [7, 22, 20], which provides a unified language for organizing frequency-wise smooth functions and their differential relations. In this setting. all geometrie objects are defined with respect to the structure sheaf OM of K-smooth functions, ensuring wransform-consittee wise independence. Tangent vector fields are realized as KKlliear derivations OM generalizing directional derivatives to the setting of structured scalars, while differential 1-forms are defined as elements of the Kähler differential module [22] over O M,, satisfying the Leibniz rule and serving as duals to vector fields under evaluation.
local definitions even under frequency-
We now introduce the algebraic structures underlying differential geometry t-manifolds.
Algebraic structures for geometry on t-manifolds. We generalize classical differential constructions to the setting of K-smooth functions using a sheaf-theoreti approach that is compatible with the cansform-based algebra underlying the t-product.
Definition (K- Et-oooooth Function). Let U be an open subset of a t-manifold MM and M be the transform defining the t-product. A function f U - K is called K-smooth if its transform-doman components are S'S-mooth. i.c.,
[M E O ([M(U))) for K = Ke,
[MS(()) E C ((M(U))()) for IK = K
This notion extends classical smoothness to the structured t-scalar algebra by enforcing frequency- wise regularity consistent with the t-product'.
Definition (Structure Sheaf). Let M -moooh t-manifold over K. The structure sheaf O assigns to each open set U c M the set of all K-smooth functions f - K:
Om(U) = U - K| I K-smooth }.
This generalizes the classical ring of smooth real-valued functions to structured scalars in K and serves as the algebraic base for all geometric constructions. Elements of (U) can be viewed as generalized smooth scalar fields that take values in the t-product algebra K, respecting smoothness the transform domain. Having defined the structure sheaf OM, we now introduce the tangent space, which captures infinitesimal variations of K-smooth functions on t-manifolds via K-linear derivations.
Definition 4 (Tangent Space). The tangent space Tm(U) over U c M is the space of K-lincar derivations acting OM(U):
D(f g) = D(f) + f D(g).
TM(U) = D: DD:0MU)) - (M(()
D(a f b+ g) a * D(J) + b D(g), Va, ba K
Each D generalizes a directional derivative in the K-valued setting and plays the role of vector field.
'The definition of K-smooth functions is motivated by the notions of tubal functions [49] and I-fanctions [43]. Throughout this work, all K-smooth functions and related geometric objects are assumed to satisfy frequency-wis smoothness in the transform domain. This assumption is considerably stronger than that of general vector-valued smooth functions, which may allow inter frequeeyy dependencies. While simplifies both definitions and theoretical analysis, it also limits the expressiveness of the framework for modeling more complex cross frequenyy behaviors. Relaxing the frequency- wise smoothness assumption therefore represents promising direction for future research.
5

---

Definition 5 (Differential I-Forms). The module of 1-forms M (U) is defined as the Kähler differential module over (U(,), generated by formal symbols df subject to the Leibniz rule: d(f * f * dg + g + df. It satisfies df(x) := X(f) for all X Tm(U) and f OM(UU,, where (M) denotes the module of K-linear derivations on (U))
These definitions establish the algebraic foundation for Riemannian geometry on t-manifolds, includ- ing Riemannian metric, gradients, Laplacians, and geodesies, which we develop next.
Riemannian geometry on t-manifolds. We equip each tangent space with a Riemannian metric that respects the t-scalar algebra K E (IK,, ] and transform-domatt smoothness, enabling computation of distances, angles, and gradients.
Definition 6 (Riemannian Metric on t-Manifolds). A Riemannian metric on smooth t-manifold M is a symmetric, K- -ossitive-definiee and IK-bilinear form
9: (U) TM(U) Om(
defined on each open set U M, satisfying the following conditions:
(I) Symmetry: For all X, Y E TM(U') and every x E U, the metric satisfies (x, Y) = (a(Y,)), where ya (X,Y) (x) e KK denotes the value of the K-valued function g Y) x.
(2) ---positivedeffiite: For every x E U and nonzero X E (U), the element dz (X, X) E KK is pointwise positive in the transform domain. That is, [Mgx X) 0 for all & when K Ke. and [Mgx(X. ()()) >
for all when K- Ko
(3) Smoothness: For all X,Y Y E (U) x) E (U), i..., it is K-smooth.
Intuitively, a Riemannian metric on t- t-manifold assigns an inner product between vector fields, but with values in the t-scalar ring K. In the transform domain, this corresponds family of classical real-valued inner products computed slice-wise weeewwieeooo for each frequency index & (in the discrete case) or each parameter (in the continuous case) preserving compatibility with the algebraic structure of the t-product [32, 33].
The Riemannian metric enables the definition of gradients, which are essential for optimization tasks on t-manifolds. Since functions on M are K-valued, the gradient must be defined in way that respects both the algebraic structure of the t-product and the underlying transform.
Definition 7 (Gradient). Let M be a t-manifold with Riemannian metric g. The gradient f of function f Om(UU) is the vector field in (U) satisfying g( df) for all X E (U).
While defined abstractly, the gradient operates slice-wise in the transform domain, preserving the structure of t-scalars and enabling differentiable learning in frequency-
tensor spaces. Definition 8 (Divergence and Laplacian). Let X E (U). The divergence div is defined via the Lie derivative of the volume form induced by g. The Laplacian of function f is defined as Af ;= ffv
The Laplacian governs harmonicity and diffusion on t-manifolds and reduces to classical Laplace operators slice-by-slic in the transformed domain. This structure enables geometric regularization and smoothing in tensor spaces. In particular, it provides a principled generalization of spectral methods used in t-SVD-based graph models [13, 51], where our construction now endows such models with intrinsic manifold-aware L Laplacians.
Definition 9 (Levi-Civita Connection and Geodesics). A connection V on TM is called the Levi- Civita connection if it torsion-free and metric-compatible: VxY V = [X, Y] and Xg(Y, 2) o( V Y, Z) + g(Y, Z). A curve 7 - M is geodesic if it satisfies V V 0.
This generalizes shortest-path and constant- constant-velocity velocity flows to the t-manifold setting. In practice, geodesics respect the transform structure and can be computed slice-wise, providing a bridge to structure-awart modeling in tensor dynamics.
These constructions establish a differential geometric framework for t-manifolds, addressing Q1 by unifying discrete and continuous tensor data with transform-consister metrics, operators, and flows which are crucial for extending traditional manifold learning techniques [46] to the t-scalar setting. This foundation also enables learning theory on t-manifold (Q2) investigated in the next section.

---

3 Learning Theory on t-Manifolds: Testing, Fitting, and Function Learning
Leveraging the t-manifold geometry framework, we tackle Q2 by testing and fitting a low-dimensional I-manifold, then modeling its functions with tensor neural networks.
3.1 Theory of t-Manifold Hypothesis Testing and Fitting
We begin our study of Q2 by examining the geometric structure underlying tensor data. Motivated by the manifold hypothesis [18], we ask: can t-vector data in K be well approximated by low-dimensional smooth t-manifold?
When tensor-valued data concentrate around a low-dimensional t-manifold M, learning can be performed in reduced, structure-aligned space, which enhances generalization and interpretability. To model such structure, we exploit the algebraic property of K2 = IRixIxe, +, *), where the t-product defined through an orthogonal transform M Rexe that decouples tensor operations into frequency-wise matrix multiplications. This yields natural frequency-domain representation: t-manifold M IK is characterized by its frequency slices M2 {[MaJe x E M} Rd which jointly describe the global tensor geometry.
Specifically, extend the manifold hypothesis framework [18] the t-product setting. Given sam- ples {=:)<=1 c: K we test whether, for each frequency index k. the transformed slice {[Mx;]a la 3 lies near p-dimensional manifold Ma. The global t-manifold candidate is then defined as the inverse transform of the slice manifold collection (M&)āgn- To quantify data concentration, we aggregate the per-slice deviations _ dist([Ma:]J... M&)² and test whether their average remains below given resolution e.
To formally characterize the complexity of t-manifold M, we define its volume and reach via its transform- domain components:
Definition 10 (Spectral Volume and Reach). Let M = IK be p-dimensional t-manifold, and let M₁ = [[Ma] x E M} c R2 denote its k-th frequency slice. We define the spectral volume and spectral reach of M as Volspec == >i=1 Hbp (MR)) and reachspec == miniskse reachg (M&), where Hd denotes the p-dimensional Hausdorff measure in Rdd, and reachę4 is the reach measured in the Euclidean metric.
Theorem 1 (t-Manifold Hypothesis Testing). Let (21)[=1 Bgg be i.i.d. samples normalized the unit ball. 6 Fix intrinsic dimension p. upper bound V on the spectral volume, lower bound on the spectral reach, resolution >> O, and confidence level 5 E (0, 1). Then there exists statistical test that, for sufficiently large n, distinguishes with probability at least ô between the following two situations:
(I) (Near case) There exists M G(p, ((p.CV,T/C) such that 1
n E dist? & Ce.
(II) (Far case) For all M e V/C,CTC 1 E._, dist? (), c/C.
Here, Gl, V,T) denotes the class of C 1-manifolds with spectral volume most V and spectral reach at least T, and C is positive constant depending only on and c.
This result suggests that low-dimensional t-manifold structures can, in principle, be detected from high-dimensional tensor data, even when the underlying geometry is implicit and embedded in structured transform- based representation.
While Theorem 1 confirms the presence of t-manifold structure, addressing Q2 further requires reconstructing the manifold itself. We now consider the fatting problem: given noisy samples [2:)2_ = K2 near p-dimensional smooth I-manifold, can we recover smooth estimator that approximates up to geometric accuracy?
Theorem 2 (t-Manifold Fitting). Let I4 = 2, 2 & i where 2; Unif( is sampled from t-manifold M G(p, V, and & NO, o² Lod) represents additive Gaussian noise. Assume the manifold dimension p and noise level
known. If O (v+3) and
'Rescaling so that ||as| & does not affect the generality of the result, since both spectral reach and volume scale homogeneously under dilation.
7

---

is sufficiently small, then with high probability, the estimator M is p-dimensional C² t-manifold embedded K.g. satisfying the Hausdorff distance bound:
du (M & Co log(1/o),
1/2 where the distance is measured under the K. norm | :- (2x [[Mx]ol3) C' depends on c,p, T,T, and the unitary transform M.
and the constant
The theorem establishes the pointwise and global proximity of M the target t-manifold while preserving its Ca regularity. This underscores the consistency of the proposed contraction-based estimator within the K geometric setting.
3.2 Function Learning Theory on t-Manifolds
To further address Q2, we consider the problem of learning functions defined on a p-dimensional t-manifold M c IK which may be suitable model in structured prediction tasks such as image or video analysis. The goal is to approximate an unknown Km-sthhth function fo . - K. that maps nigh-dimensional t-vector inputs to structured t-scalar outputs, such pixel intensities or compressed features. While the low- -ow-dimeniiona geometry of M enables efficient representation, the algebraic complexity of the t-scalar ring K,,, defined via transform- mansfrdn-bbeedd multiplication, poses unique challenges. In particular, any approximation architecture must respect both the non-Euclidean geometry of M and the algebraic structure of K...
To address this, we adopt tensor neural networks (TNNs) specifically designed to operate over t-product spaces [44, 61, 48]. A TNN approximator is constructed composition of L layers:
hy ((Wi hx-1), 1= 1=1,...,
where bu = I2 E K is the input tensor, Wi E mm 1xc are learnable weight tensors, and the activation function is defined as o = (xx) x M. where denotes the elementwise ReLU function. This transform-done activation design follows [44], and E Rexe is a fixed orthogonal transform, such the DCT.
The network outputs a function f(x,; 0) :- h2 E K2, where 0 (Wi,... Wr denotes all model parameters. Training is performed via empirical risk minimization under the observation model:
y: fo(:;) + 4, = 1,..., n,
(2)
where yi Ke is the response and e; denotes zero-mean Gaussian noise with covariance o² le. The empirical objective is defined directly in the transform domain:
in E arg min 1 E Elu f(2)). : : - JeF, n
This formulation preserves transform consistency and ensures compatibility with the t-product struc- ture of K,, It also reveals the core challenge of t-manifold learning: designing neural architectures capable of approximating -valued functions over domains that are both geometrically curved and algebraically structured.
We introduce regularity assumptions under which TNNs admit provable generalization guarantees. Assumption (Modeling Assumptions). The following conditions hold:
(AI) Data distribution regularity: The data points (a.) = TK! lie within a compact p-dimensional @ t-manifold M with spectral reach at least and spectral volume most V. The sampling distribution v is supported on compact subset of M and is absolutely continuous with respect to the spectral volume measure.
(A2) Target function smoothness: The function : - IK is CS-K,-smooth with bounded norm:
[ := sup sup 120 [M ya (xx) & Bo.
(A3) Model class constraint: The hypothesis class F,, consists of TNNs with width No. depth Lo- total parameter count S, and output norm bound B, using the t-product defined via a fixed orthogonal transform M.
8

---

Remark 1. Assumption reflects natural and interpretable conditions adapted to the spectral geometry of t-manifolds: Specifically, (AI) extends the classical manifold hypothesis to the t-product setting via spectral volume and reach [29]; (A2) requires frequency-slin smoothness of fo. t-product variant a standard assumption in geometric deep learning [35]: (A3) reflects practical TNN design and enforces algebraic compatibility with K [44, 61].
We are now ready to state the convergence guarantee for TNNS trained via empirical risk minimization on t-manifolds.
Theorem 3 (TNN Approximation t-Manifolds). Define PenPett Oplogged)) the effective dimension determined by the intrinsic complexity of M. Under Assumption 1, there exists TNN class Fu such that the empirical risk minimizer I'a of Problem (3) satisfies
Bl foll2(())
Far+22
20
where C depends on (p, Bu, V, T, o, c, d, log Here, Ellf fulliz(w) represents the expected squared -rror between the TNN estimator fn and the true target function fo. measured with respect to the sampling distribution on M.
This result shows that TNNs can approximate smooth functions on t-manifolds with rates determined by the effective dimension Opp log(cd)). The sample-dependent term benefits from the low effective complexity of the t-manifold, achieving near-optimal nonparametrie regression rate under strict manifold support. The bound scales polynomially with ambient and spectral parameters, aligning with classical manifold approximation theory [29, 35] and extending it to the t-manifold settings.
4 Modeling Implications of t-Product Geometry
This section examines how the framework of t-product geometry can inspire new modeling per- spectives, complementing the theoretical results developed in Sections 2 and 3. Rather than val- idating the theory through experiments, we focus on clarifying how the algebraic structures of t-scalars, t-modules, and t-manifolds translate into modeling constraints and design principles for high-dimensional tensor data, such as how linearity, locality, and curvature are represented in the transform domain.
At the foundational level, the t-scalar serves as a basic modeling unit, such as a row or column of an image [83], while preserving its internal spectral structure. Building upon this, the t-module provides a flat t-linear space formed by linear combinations of t-scalars. At a more general level, t-product geometry allows for nonlinear structures, such as curvature twisting, arise when linear t-modules are combined, constrained, or glued together in nontrivial ways. From this viewpoint, the central modeling perspective of t-product geometry is to understand how structured constraints on t-scalar representations give rise to effective geometric structure.
Building on this understanding, we explore how t-product geometry can inform model construction. Section F. introduces the Bidirectional Tensor Representation (BTR) formulation, which incorpo- rates dual t-module constraints for structured learning tasks such clustering and tensor recovery. Preliminary evaluations across several data modalities, including images, videos, hyperspectral and multispectral images, point clouds, and thermal sequences, suggest that BTR offers a coherent and flexible modeling framework. Broader extensions and discussions are provided in Section F.2.
Example: Bidirectional Tensor Representation (BTR). Images naturally exhibit row- col-column symmetry, which aligns with the dual-module structure implied by the t-product: two-dimensional image can be regarded as a t-vector in either Ky. (row-wise) or IK (column-wise). The BTR formulation leverages this bidirectional structure by applying low-rank regularization in both modules through tensor nuclear norm surrogates, thereby promoting coherence from both perspectives.
Although KÅ and KY are each flat (linear) modules, enforcing low-rankness jointly in both induces a coupling that geometrically /wists the representation space. From a geometric viewpoint, this coupling gives rise to an effective, constraint-induced nonlinearity in the representation space, which empirically associated with improved generalization. The objective formulation and optimization details are presented in Appendix F.I.

---

We evaluate BTR on image clustering and video denoising tasks. As shown in Table 1, BTR achieves consistent gains across clustering metries (ACC, NMI, PÚR) and yields the highest PSNR values in YUV video denoising under 20% noise (Figure F.2). More experiments on Poisson tensor completion are provided Appendix F.1.3. These results illustrate how incorporating geometric principles from the t-product perspective can enhance structured tensor modeling.
Table 1:
Clustering performance comparison on five benchmark datasets using accuracy (ACC), normalized mutual information (NMI), and purity (PUR).
R-TPCA
OR-TPCA
R-TLRR
OR-TERR
BTR (Progosed)
Dataset
Metric
DFT
DCT [40]
[82]
DFT[83]
DCT [73]
DFT [68]
DCT [68]
DFT
DCT
ACC
0,7613
0.7777
0,7644
0,8429
0.3400
0.8358
0.7386
0.8591
0.8594
FRDUE
NMI
0.9093
0,9140
0,9127
0.9511
0.9510
0,9470
0.9045
0.9578
0.9567
PUR
0.7955
0.8077
0.7990
0.8760
0.8643
0.7776
0.8901
0,8890
ACC
0.7906
0.7911
0,7974
0,8000
0.8002
0.8657
0.7616
0.8860
0.8769
FRDUE-100
NMI
0,9133
0,9136
0.9221
0,9529
0.9526
0.9564
0.9055
0.9643
0.9605
PUR
0,8197
0.8200
0.8271
0,8882
0.8881
0.8926
0.7956
0.9128
0.9032
ACC
0.3970
0.3703
0,3965
0.52.0
0.6452
0,6162
0.5535
0,5645
0.6670
Olivetti
NMI
0,5990
0,5809
0,5987
0 0.6920
0.7905
0,7732
0.7395
0.7290
0.8080
PUR
0.4242
0.3983
0,4200
0.5483
0.6755
0.6492
0.5880
0.5960
0.7005
ACC
0.4276
0.4268
0.5401
0.5897
0.5831
0.4594
0.1975
0.6000
0.6110
PIE-10
NMI
0.6674
0.6621
0.7361
0,7562
0.7618
0,7049
0.5150
0.7697
0.7778
PUR
0.4469
0.4462
U,5593
0.6059
0.5999
0.4803
0,2050
0.6187
0.6325
ACC
0,3548
0.3537
0.3369
0.4257
0,4088
0.5191
0.4499
0.5799
0.5339
USPS
NMI
0,3066
0.2986
0.2828
0,3834
0.3915
0.5104
0.4490
0,5860
0.5390
PUR
0,4470
0,4542
0,4425
0.5264
0,5076
0,6330
0,5905
0.6862
0.6662
5 Concluding Remarks
We introduce a general framework of t-product geometry that extends differential geometry to the 1-product algebra for tensor learning. Through sheaf-theoretic formulation, we define metrics, gradients. Laplacians, and geodesics over both discrete and continuous t-scalars, unifying transform- domain representations within a coherent geometric structure. To our knowledge, this is the first systematic development of differential geometry on t-scalars, enabling structured modeling of tensor data. We also present a theoretical study of learning on t-manifolds, encompassing hypothesis testing, manifold fitting, and function learning. The potential applicability of this framework is illustrated through examples in image clustering and video denoising. These results offer a principled foundation for geometry aware tensor learning.
Limitations and future work. This work focuses on the theoretical foundations of t-product geometry, emphasizing conceptual definitions, assumptions, and provable guarantees rather than empirical evaluations. The authors believe that developing a mature and comprehensive theory in this direction highly nontrivial task. The framework presented here is preliminary and necessarily incomplete, yet it aims to shed light on what a general theory of t-product geometry might entail and to provide foundation for continued theoretical development in this direction.
This work has limitations in scope, idealized assumptions, and algorithmic development:
Scope of study. This paper focuses primarily on the theoretical formulation of t-product geometry rather than empirical validation. A natural next step lies in extending the framework richer modeling paradigms, such as generative modeling, geometric graph networks, temporal dynam- ics, manifold optimization, manifold learning, and federated tensor learning, where geometric consistency across spectral modes could offer new algorithmic principles and inductive biases.
Idealized assumptions. For theoretical clarity, several idcalized assumptions were made: (1) transform-domain smoothness, (2) the existence of unit element in the t-scalar ring, and (3) frequency-wist independence among spectral components. These assumptions make analysis tractable but restrict realism. In particular, relaxing the independence assumption to allow structured cross- frosuffryy coupling may reveal richer geometric behaviors and better reflect the complexity of real-world tensor data.
Algorithmic extensions. Beyond theory, the computational side of t-product geometry remains largely unexplored. Developing numerically stable and scalable algorithms that leverage the pro- posed geometric structures, for instance in optimization over i-manifolds, spectral regularization, or tensor-valued neural architectures, poses both challenges and opportunities.
10

---

Acknowledgments and Disclosure of Funding
The authors sincerely thank the Area Chair and the four anonymous reviewers for their detailed and constructive feedback. Their suggestions have greatly enhanced the quality and clarity of this paper. This work was supported in part by the National Natural Science Foundation China under Grant Numbers 62203 124, 62562065; JSPS KAKENHI Grant Numbers JP25K21283, JP24K20849, JP23K28109, JP24K03005, JP25K21 288; JSPS Bilateral Program Number JPJSBP 120557420; and RIKEN Incentive Research Project 100847-202301065 1. Yuning Qiu was supported by the RIKEN
Special Postdoctoral Researcher Program.
References
[1] N. Aigerman, K. Gupta, V. G. Kim, S. Chaudhuri, J. Saito, and Groucix. Neural jacobian fields: learning intrinsic mappings of arbitrary meshes. ACM Transactions Graphics (TOG), 41 (1): 2022.
[2] M. Anthony and P. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge, 1999.
[3] R. G. Baraniuk and M. B. Wakin. Random projections of smooth manifolds. Found. Comput. Math., (1):51-77, 2009.
[4] L. Bartlett, N. Harvey, C. Liaw, and A. Mehrabian. Nearly-tight VC-dimension and pseudodi- mension bounds for piecewise linear neural networks. J. Mach. Learn. Res., 20:Paper No. 63, 17, 2019.
[5] M. Belkin and P. Niyogi. Semi-supervised learning on riemannian manifolds. Machine learning, 56(1):209 2004.
[6] K. Braman. Third-ordes tensors as linear operators on space of matrices. Linear Algebra and its Applications, 433(7):1241-122) 1253, 2010.
[7] G. E. Bredon. Theory, volume 170 of Graduate Texts in Mathematics. Springer, 2nd edition, 1997.
[8] Y. Cao and Y. Xie. Poisson matrix recovery and completion. IEEE Transactions on Signal Processing, 64(6)):609-1620, 2015.
[9] M. Chen, H. Jiang, W. Liao, and Zhao. Nonparametric regression on low-dimensional manifolds using deep relu networks. 190.00/90..0/4// 2019.
[10] M. Chen, H. Jiang, and T. Zhao. Efficient approximation of deep relu networks for functions low dimensional manifolds. Advances in Neural Information Processing Systems, 2019.
[11] H. Chung, B. Sim, D. Ryu, and J. C. Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25666, 2022.
[12] W. Davis and Sharma. Background-ssutracctnn using contour-based fusion of thermal and visible imagery. Computer vision and image understanding, 106(2-3):162-182, 2007.
[13] L. Deng, X.-Y. Liu, H. Zheng, X. Feng, and Y. Chen. Graph spectral regularized tensor completion for traffic data imputation. IEEE Transactions on Intelligent Transportation Systems, 23(8):00996-1110, 2021.
[14] S. Esposito. Q. Xu, K. Kania, C. Hewitt, O. Mariotti, L. Petikam, J. Valentin, A. Onken, and O. Mac Aodha. Geogen: Geometry-aware generative modeling via signed distance functions. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7479-7488, 2024.
[15] C. Fefferman. Whitney's extension problem for "m Annals of Mathematics. 164(1):313-359 2006.

---

[16] C. Fefferman, S. Ivanov, Y. Kurylev, M. Lassas, and H. Narayana. Fitting putative manifold to noisy data. In Conference On Learning Theory, pages 688 388-720. PMLR, 2018.
[17] C. Fefferman, S. Ivanov, M. Lassas, and H. Narayanan. Fitting& manifold of large reach to noisy data. Journal of Topology and Analysis, 17(02):315-366, 2025.
[18] C. Fefferman, S. Mitter, and H. Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4)9983- 2016.
[19] K. Gilman, D. A. Tarzanagh, and L. Balzano. Grassmannian optimization for online tensor completion and tracking with the t-SVD. IEEE transactions signal processing, 70:2152-2161 2022.
[20] A. Grothendieck and J. Dieudonné. Eléments de géométrie algébrique. 1964.
[21] L. Györfi, M. Kohler, A. Krzyžak, and H. Walk. A Distribution- Free Theory of Nonparametric Regression. Springer- Verlar, New York, 2002.
[22] R. Hartshorne. Algebraic Geometry, volume 52 of Graduate Texts in Mathematics. Springer, 1977.
[23] J. Hou, F. Zhang, H. Qiu, J. Wang, Y. Wang, and D. Meng. Robust low-tubal-rank tensor recovery from binary measurements. IEEE Transactions Pullern Analysis and Machine Intelligence, 2021.
[24] Z. Huang, X. Li, Y. Ye, and M. K. Ng. Mr-gen: Multi-relational graph convolutional networks based on generalized tensor product. In IJCAI, volume 20, pages 1258-12644 2020.
[25] B. Hui, Z. Song. H. Fan, P. Zhong, W. Hu, X. Zhang, Ling, H. Su, W. Jin, Y. Zhang, et al.
A dataset for infrared detection and tracking of dim-small aircraft targets under ground/air background. China Sci. Data, 5(3):291-300 2020.
[26] A. J. Izemman. Introduction to manifold learning. Wiley Interdisciplinary Reviews: Computa- tional Statistics, 4(5):439-44)) 2012.
[27] J. Ji and S. Feng. Anchor structure regularization induced multi-view subspace clustering via enhanced tensor rank minimization. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 19343-19352, 2023.
[28] Q Jiang and M. Ng. Robust low-tubal-rank tensor completion via convex optimization. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 2649- 2655. AAAI Press, 2019.
[29] Y. Jiao, G. Shen, Y. Lin, and J. Huang. Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics, 51(2):691-716, 2023.
[30] E. Kernfeld, S. Aeron, and M. Kilmer. Clustering multi-way data: A novel algebraic approach. arXiv preprint
2014.
[31] E. Kernfeld, M. Kilmer, and S. Aeron. Tensor-tensor products with invertible linear transforms. Linear Algebra and its Applications, 485:545-570, 2015.
[32] M. E. Kilmer, K. Braman, al. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM J MATRIX ANAL A, 34(1):148- 172, 2013.
[33] M. E. Kilmer, L. Horesh, H. Avron, and Newman. Tensor-tensor algebra for optimal representation and compression of multiway data. Proceedings of the National Academy of Sciences, 118028):e201585111 2021.
[34] W. Kühnel. Differential geometry, volume 77. American Mathematical Soc., 2015.
[35] D. Labate and J. Shi. Optimal approximation of smooth functions on riemannian manifolds using deep relu neural networks. Available at SSRN 5117796.
12

---

[36] J. M. Lee. Riemannian manifolds: an introduction to curvature, volume 176. Springer Science & Business Media, 2006.
[37] X. Li, M. K. Ng. G. Xu, and A. Yip. Multi-relational graph convolutional networks: Generaliza- tion guarantees and experiments. Neural Networks,
2023.
[38] Liu, Musialski, P. Wonka, and Ye. Tensor completion for estimating missing values in visual data. IEEE TPAMI, 35(1):208-224 2013.
[39] X. Liu, S. Aeron, V. Aggarwal. and X. Wang. Low-tubal-rank tensor completion using alternat- ing minimization. IEEE TIT, 66(3):1714-17377 2020.
[40] C. Lu. Transforms based tensor robust pea: Corrupted low-rank tensors recovery via convex optimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 115,,1152,2 2021.
[41] C. Lu, J. Feng. W. Liu, Z. Lin, S. Yan, et al. Tensor robust principal component analysis with new tensor nuclearr norm. IEEE TPAMI, 2019.
[42] C. Lu, X. Peng, and Y. Wei. Low-rank tensor completion with new tensor nuclear norm induced by invertible linear transforms. In CVPR, pages 5996-6004, 2019.
[43] K. Lund. The tensor t-function: definition for functions of third-order tensors. Numerical Linear Algebra with Applications. 27(3):22288, 2020.
[44] O. A. Malik, S. Ubaru, L. Horesh, M. E. Kilmer, and H. Avron. Dynamic graph convolutional networks using the tensor m-product. In Proceedings of the 2021 SIAM international conference on data mining (SDM). pages 729-737. SIAM, 2021.
[45] X.-P. Mao, Y. Wang. and Y.-N. Yang. Computation over t-product based tensor stiefel manifold: A preliminary study. Journal of the Operations Research Society of China, pages 1-49, 2024.
[46] M. Meilä and H. Zhang. Manifold learning: What, how, and why. Annual Review of Statistics and Its Application, 11(1):93-417, 2024.
[47] U. Mor and H. Avron. Quasitubal tensor algebra over separable hilbert spaces. arXiv preprint vrXiv:2504.16231, 2025.
[48] E. Newman, L. Horesh, H. Avron, and M. Kilmer. Stable tensor neural networks for rapid deep 2018.
learning. arXiv preprint
[49] E. Newman, L. Horesh, H. Avron, and M. Kilmer. Stable tensor neural networks for efficient deep learning. Frontiers in Big Data, 7:1363978, 2024.
[50] N. Parikh, S. Boyd, al. Proximal algorithms. Foundations and Trends@ in Optimization, 1(3):127-239, 2014.
[51] K. Pena-Pena, D. L. Lau, and G. R. Arce. T-HGSP: Hypergraph signal processing using t-product tensor decompositions. IEEE Transactions on Signal and Information Processing over Networks, 9:329-345, 2023.
[52] F. Qian, Z. Liu, Y. Wang, Y. Zhou, and G. Hu. Ground truth-free 3-d seismic random noise attenuation via deep tensor convolutional neural networks in the time-frequency domain. IEEE Transactions on Geoscience and Remote Sensing, 60:1-17 2022.
[53] H. Qiu, Y. Wang. S. Tang, Meng, and Q. Yao. Fast and provable nonconvex tensor RPCA. In International Conference Machine Learning, pages 18211-18249. PMLR, 2022.
[54] Y. Qiu, G. Zhou, A. Wang, Q Zhao, and S. Xie. Balanced unfolding induced tensor nuclear norms for high-order tensor completion. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[55] S. T. Roweis and K. Saul. Nonlinear dimensionality reduction by locally linear embedding. science, 29005500) 2000.
13

---

[56] T. Sakai. Riemannian geometry, volume 149. American Mathematical Soc., 1996.
[57] G. Sardanashvily. Lectures on differential geometry of modules and rings: Application to Quantum Theory. LAP LAMBERT Academic Publishing, 2012.
[58] Schmidt-Hieher, Nonparametric regression using deep neural networks with ReLU activation function (with discussion). Ann. Statist.,
2020.
[59] G. Song, M. K. Ng, and X. Zhang. Robust tensor completion using transformed tensor singular value decomposition. Numerical Linear Algebra With Applicationn, 27(3):e2299, 2020.
[60] R. Tang and Y. Yang. Adaptivity of diffusion models to manifold structures. In International Conference Artificial Intelligence and Statistics, pages 1648-1656. PMLR, 2024.
[61] A. Wang, C. Li, M. Bai, Z. Jin, G. Zhou, and Q. Zhao. Transformed low-rank parameterizaatin can help robust generalization for tensor neural networks. In Advances in Neural Information Processing Systems, 2023.
[62] A. Wang, Y. Qiu, M. Bai, 7. Jin, G. Zhou, and Q. Zhao. Generalized tensor decomposition for understanding multi-output regression under combinatorial shifts. In Advances in Neural Information Processing Systems, 2024.
[63] A. Wang, Y. Qiu, H. Huang, Z. Jin, G. Zhou, and Q Zhao. Refining dual spectral sparsity in transformed tensor singular values, 2025.
[64] A. Wang, Y. Qiu, Z. Jin, G. Zhou, and Q. Zhao. Low-rank tensor transitions (LoRT) for transferable tensor regression. In International Conference on Machine Learning, volume 267, 2025.
[65] A. Wang, G. Zhou, Z. Jin, and Q. Zhao. Tensor recovery via *s-spettral k-support norm. IEEE Journal of Selected Topics in Signal Processing. 15(3):522 2244,,534444 2021.
[66] C. Wang, X.-L. Zhao, Y.-B. Zheng, B.-Z. Li, and M. K. Ng. Functional tensor singular value decomposition. SIAM Journal on Scientific Computing, 47(4):A1180-A2209 2025.
[67] J. Wang and X. Zhao. Functional baased low-rank tensor factorization for multi- dimensional data recovery. In European Conference on Computer Vision, pages 39-56. Springer, 2024.
[68] T. Wu. Robust data clustering with outliers via transformed tensor low-rank representation. In International Conference on Artificial Intelligence and Statistics, pages 1756-1764. PMLR, 2024.
[69] T. Wu and W. U. Bajwa. A low tensor-rank representation approach for clustering of imaging data. IEEE Signal Processing Letters, 25(8): 10111199-110000 1200, 2018.
[70] Z. WU, L. Shu, Z. Xu, Y. Chang, C. Chen, and Z. Zheng. Robust tensor graph convolutional networks via t-SVD based graph augmentation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2090-2099, 2022.
[71] Y. Xie, D. Tao, W. Zhang, representations for clustering by tensor multi-rank minimization. International Journal of Computer Vision, 126 126(11):115--1179 2018.
Y. Liu, I. Zhang, and Y. Qu. On unifying multi-view self-
[72] Y. Xu, R. Hao. W. Yin, and Z. Su. Parallel matrix factorization for low-rank tensor completion. Inverse Problems and Imaging, 9())600-622, 2015.
[73] J. Yang, C. Chen, H.-N. Dai, M. Ding, Z.B.. Wu, and Z. Zheng. Robust corrupted data recovery and clustering via generalized transformed tensor low-rank representation. IEEE Trunsuctions on Neurul Networks and Learning Systems, 2022.
[74] Z. Yao, Su, B. Li, and S.-T. Yau. Manifold fitting. arXiv preprint, 2023. arXiv:2304.07680.
[75] 7. Yao, J. Su, and S.-T. Yau. Manifold fitting with cyclegan. Proceedings of the National Academy of Sciences, 12135)2221143311,, 2024.
14

---

[76] Z. Yao and Y. Xia. Manifold fitting under unbounded noise. Journal of Machine Learning Research, 26(45):1-55, 2025.
[77] D. Yarotsky. Error bounds for approximations with deep relu networks. Neural Networks, 94:103-114, 2017.
[78] X. Zhang and M. K.-P.NN. Low rank tensor completion with poisson observations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[79] 7. Zhang and S. Aeron. Exact tensor completion using t-SVD. IEEE TSP, 65(6):1511-1526,
2017.
[80] Z. Zhang. G. Ely. Aeron, et al. Novel methods for multilinear data completion and de-noising based on tensor-SVD. In CVPR, pages 3842-3849, 2014.
[81] Z. Zhang, K. Zhang, M. Chen, Y. Takeda, M. Wang, T. Zhao, and Y.-.. Wang. Nonparametric classification on low dimensional manifolds using overparameterized convolutional residual networks. Advances in Neural Information Processing Systems,
2024.
[82] P. Zhou and J. Feng. Outlier-robust tensor pea. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2017.
[83] P. Zhou, C. Lu, J. Feng, Z. Lin, and S. Yan. Tensor low-rank representation for data recovery and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5):1718-1732. 2019.
[84] B. Zhu, J. Z. Liu, S. E. Cauley, B. R. Rosen, and M. S. Rosen. Image reconstruction by comain-transform manifold learning. Nature, 555(7697):4877-99) 2018.
15

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] is generally preferable to "[No]" it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction clearly state the three main contributions: (1) the definition of t-manifolds and associated differential geometric structures, (2) a theoretical framework for t-manifold learning including hypothesis testing and function approximation, and (3) a conceptual application illustrating empirical implications. These align with the main technical developments the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
16

---

Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper includes a dedicated discussion of limitations in the conclusion, ac- knowledging its primary focus on theoretical development. It highlights several simplifying assumptions made for conceptual clarity, such as transform- -omomnn smoothness and the existence of unit element. The discussion also notes that complex practical algorithms and broader model classes are beyond the scope of this initial exploration.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well sell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, authors should discuss possible limitations of their approach to address problems of privacy and faimesss
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For each theoretical result,including the t-manifold hypothesis testing theorem, manifold fitting theorem, and the TNN approximation theorem, the paper clearly states all necessary assumptions and outlines the corresponding results. Full proofs are deferred to the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
17

---

4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The main experimental results, such as the clustering and denoising case study using BTR, are presented with detailed metrics, dataset names, comparison methods, and tables. The appendix is indicated to provide further details including objective functions and optimization procedures, ensuring reproducibility in relation to the conceptual claims.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The authors have made the code publicly available and indicate in the supple- mental material that it includes all necessary scripts, data links, and instructions to reproduce the main experimental results such clustering accuracy and PSNR in denoising, which supports faithful replication.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurIPS code and data submission guidelines (kttps://nips.ce cc/
for more details.
18

---

While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. httccc/pppbl..c guides/ Codeeuomiisssse
huto:////miides/oeeeuubbiiiiii www.eeuummssiooooiiooooooooo onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: While the main paper focuses on theoretical contributions, refers to the appendix for algorithm details. The case study involving the proposed models specifies datasets, metrics, and comparison baselines, and the supplementary material reportedly includes further information on hyperparameters and optimization, satisfying reproducibility requirements.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper presents performance metries (e.g.. ACC. NMI, PUR, PSNR) through tables and figures, focusing on conceptual illustration rather than exhaustive bench- marking. As the empirical results aim demonstrate the potential utility of the proposed framework, statistical uncertainty measures such as error bars or significance tests are not emphasized.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
19

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative eтror rates).
If error bars reported in tables plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The supplemental material includes information about the computational setup, including hardware specifications and memory.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https http:s:////euuiit http///nnuri.ppeeeee
Answer: [Yes]
Justification: The paper focuses on theoretical and methodological advancements in tensor geometry and learning, uses only public datasets for illustrative experiments, and does not involve sensitive data, human subjects, or deployment risks, thus conforming to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., ... if there is special consid- eration due to laws or regulations their jurisdiction..
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The paper is a purely theoretical work focused on foundational geometry and learning theory for tensor data. As such, it does not raise direct societal concerns impacts that would typically require discussion.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
The paper does not involve the release of models or datasets with high risk for misuse. It focuses on thcoretical frameworks and illustrative experiments using standard public datasets.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper uses publicly available datasets and baseline models, all of which are properly cited in the references. There is no indication of license violations, and terms of use appear to be respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The paper introduces the geometry-inspired models and associated code, which is made available with documentation and usage instructions in the supplemental material, supporting reproducibility and clarity for users.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve any crowdsourcing experiments or research with human subjects, so this question is not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The study does not involve human participants, so there are no associated risks or requirements for IRB approval.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
The core method development in this research does not involve LLMs as any important, original, non-standard components. In accordance with the NeurIPS LLM policy, any use of LLMs was limited to language polishing or code formatting and does not affect the methodology, scientific rigor, originality of the work.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve L.MMs as any important, original, non non-standard components.
Please refer to our LLM policy (https : /nturiise..////// wwc/onoeeeeccs////neeee /LLM) for what should or should not be described.
23


Reviews:
[{"review_id": "qDYCESIxdm", "reviewer": "Reviewer_gABx", "summary": "The authors define a t-manifold, on discrete or continuous t-scalars, by generalizing the classic notion of a smooth manifold.\nAppropriate generalizations of the classic definitions of the Riemannian metric, gradient, Laplacian and geodesics are defined, as well as theorems on manifold hypothesis testing [4] and manifold fitting. \nA theorem on the Tensor neural networks (TNNs) approximation error to discrete t-manifolds is also proven. \nThe efficacy of the proposed BTR (Bidirectional Tensor Representation)  is demonstrated on a clustering dataset for images and a denoising dataset for videos,\nby a comparison with alternatives such as R-TLRR [37].", "questions": "Please refer to the questions inherent in the first two weaknesses mentioned earlier.", "limitations": "Yes, the authors clearly mention certain limitations in Sec. 5.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper appears to be theoretically strong as evidenced by Theorems 1, 2 and 3.\n\n2. The empirical comparison in Sec. 4 indicates that the proposed method outperforms alternatives.  \n\n3. The authors have provided code, which helps with reproducibility.\n\nWeaknesses\n\n1. The authors do not provide a sufficient description for the optimization procedure used to obtain the minimizer in eq (3).\n\n2. The authors mention that the intrinsic dimension $p$ is much lower than the observed dimension $d$, but do not discuss how $p$ is estimated for the image and video datasets in Sec. 4.\n\n3. Typos, e.g, line 84: \"t-salars\"", "was_revised": true, "final_justification": "I wish to thank the authors for satisfactorily addressing my concerns in their rebuttal. \nSpecifically, I wish to point out the following aspect:\nI was originally worried that the t-manifold dimension estimate, required in the theoretical bounds, might render the practical benefit of this method, in the \"generally unknown until experimentation is done\" category, given that tensor nuclear norm regularization merely side-steps this problem. But I was satisfied by the authors' rebuttal regarding the directions that could be considered as part of future work. \n\nSo I will retain my (positive) rating.", "rebuttal": "We sincerely thank the reviewer for the encouraging assessment and thoughtful feedback. We appreciate the recognition of our theoretical contributions (Theorems 1–3), empirical evaluations, and reproducibility efforts.  \n\nThis work proposes a general geometric framework over the t-product algebra for tensor learning, aiming to unify discrete and continuous tensor modeling under a transform-consistent structure. To the best of our knowledge, it represents the first formal attempt to construct algebraic-geometric tools compatible with the t-product—drawing on sheaf-theoretic ideas from algebraic geometry to define transform-aligned differential-geometric structures, including metrics, gradients, and learning formulations, in a manner consistent with the underlying algebra. We further demonstrate its modeling potential through the bidirectional regularization strategy embodied in BTR.\n\nBelow we respond to the reviewer’s specific concerns in detail.\n\n---\n\n> **Weakness 1:** The authors do not provide a sufficient description for the optimization procedure used to obtain the minimizer in Eq. (3).\n\n**Response:**\n\nWe appreciate the reviewer’s attention to this point. Eq. (3) plays a theoretical role in our paper: it defines an empirical risk minimization problem over a t-manifold using Tensor Neural Networks (TNNs), serving as the foundation for the generalization bound presented in Theorem 3.\n\nOur primary aim in introducing Eq. (3) is to support theoretical analysis—not to propose a new optimization algorithm. Nonetheless, the training procedure implied by Eq. (3) follows **standard practices from the TNN literature**. Specifically:\n\n- The function $f_\\theta$ can be instantiated as a multi-layer TNN with t-product layers and transformed ReLU activations, as in [20].\n- The empirical loss $\\mathcal{L}(\\theta) = \\frac{1}{n} \\sum_{i=1}^n \\\\| f_\\theta(x_i) - y_i \\\\|_{\\mathbb{K}_c}^2$ is computed in the transform domain to ensure consistency with the t-product algebra.\n- The minimization can be carried out using standard optimizers (e.g., Adam) and autodiff frameworks such as PyTorch or TensorFlow, where the t-product operations are efficiently implemented.\n\nThis training paradigm is consistent with prior works on dynamic TNNs and tensor graph models (e.g., Refs. [20, 22]), and no additional implementation challenges arise in our setting. We will clarify this point in the revision to better connect the theoretical formulation with established optimization techniques.\n\n---\n\n> **Weakness 2:** The authors mention that the intrinsic dimension $p$ is much lower than the observed dimension $d$, but do not discuss how $p$ is estimated for the image and video datasets in Sec. 4.\n\n**Response:**\n\nWe appreciate the reviewer’s insightful question regarding intrinsic dimensionality. While we mention in the text that the intrinsic dimension $p$ is much lower than the ambient dimension $d$, we do not estimate $p$ explicitly for the image or video datasets in Section 4. This is because the experiments are not intended to compute $p$ directly, but rather to explore the modeling benefits enabled by the t-product geometry—particularly its ability to represent low-dimensional structure through transform-consistent constraints.\n\nInstead, they demonstrate how the proposed t-product geometry can inform data modeling by coupling dual t-modules in orthogonal directions, thereby introducing non-flat geometric structure into the representation. This bidirectional regularization enables the model to capture richer spatial dependencies than traditional one-sided tensor methods, reflecting the *modeling flexibility and structural expressiveness offered by the t-SVD algebra*—capabilities that are not naturally accessible in conventional vector-based approaches.\n\nIn our formulation, this effect is achieved through *tensor nuclear norm regularization*, which encourages low-dimensional structure *without requiring explicit estimation of $p$*.\n\nMore broadly, the reviewer’s question raises an important and novel subproblem in our framework: **t-manifold dimension estimation**. Unlike classical manifold learning, where intrinsic dimension estimation is typically addressed using local PCA, correlation dimension, or spectral decay, the t-product setting introduces *new challenges due to its transform-consistent structure and tensor-algebraic neighborhoods*.\n\nPotential directions for addressing this include:\n\n- Estimating local tubal ranks via patchwise t-SVD;\n- Analyzing the effective rank of learned self-representation coefficients;\n- Extending spectral or entropy-based estimators to structured tensor domains.\n\nWe view this as a technically meaningful extension beyond the scope of this paper, which focuses on establishing the theoretical foundations of t-product geometry. We will clarify this point in the revision and emphasize that dimensionality is not treated as a tunable or estimated parameter in our empirical section. We thank the reviewer for raising this thoughtful and forward-looking question.\n\n> **Weakness 3:** Typos, e.g, line 84: \"t-salars\"\n\n**Response:**\n\nWe thank the reviewer for pointing out the typo in line 84 (“t-salars”). We will correct it to “t-scalars” in the final version, and will carefully proofread the manuscript to eliminate any remaining typographical errors."}, {"review_id": "eWJvSj3dar", "reviewer": "Reviewer_3y4K", "summary": "The authors aim to develop a comprehensive differential geometry framework for tensors equipped with t-product. Their goal is to create a theoretical structure that can handle both discrete and smooth t-scalars (respectively, a row in three dimensions and a real smooth function). To achieve this, they introduce the concept of t-manifolds, which generalizes traditional differential geometry for transform-based structures. The paper explores how this framework can be applied to extract low-dimensional structures from high-dimensional tensorial data, as well as provide theoretical guarantees for learning smooth functions on low-dimensional t-manifolds using tensor neural networks. To demonstrate the practical value of analyzing the geometric properties of transform-based structures, they evaluate the Bidirectional Tensor Representation (BTR) approach (inspired by bi-module structure in t-manifolds) on image clustering and video denoising tasks.", "questions": "- As highlighted in the related work, studies such as [1] have demonstrated that the third-order tensor Stiefel manifold is a Riemannian manifold, enabling the application of Riemannian manifold optimization techniques for these t-product based models. Their work illustrates how this framework can be applied to optimization tasks such as tensor approximation with missing entries, joint diagonalization, joint t-SVD and (sparse) tensor PCA. Does your framework support the optimization of similar problems? If so, could you provide insights into how the two approaches compare in terms of optimization potential?\n\n- I recommend clarifying that the empirical results presented in the experiments are not directly related to the \"Learning Theory on t-Manifolds:Testing, Fitting, and Function Learning\" section, as one might initially expect. Providing a clearer distinction between these two parts would enhance the overall structure and reader understanding.\n\n- The paper [1] presents \"preliminary\" experiments to evaluate the feasibility of conjugate gradient descent on their Riemannian manifold.  Could similar experiments be conducted on simpler problems using the analyzed learning theory approach? It would be valuable to verify the theoretical guarantees by testing them in such controlled settings.\n\n[1] MAO, Xianpeng, WANG, Ying, et YANG, Yuning. Computation over Tensor Stiefel Manifold: A Preliminary Study. Journal of the Operations Research Society of China, pages1–49, 2024.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is clearly written and well-structured\n- While I did not verify every proof in detail, the paper presents a solid and comprehensive theoretical framework. The authors introduce novel theoretical concepts and support them with a substantial number of well-structured theorems and corresponding proofs.\n- The t-product based approaches have received growing attention in recent literature and have demonstrated their effectiveness in a range of important applications such as computer vision or signal processing. Developing geometry-aware tensor learning approaches for transform-based tensorial models holds significant potential.\n\nWeaknesses\n- The experimental section does not adequately address the proposed hypothesis testing and fitting methods for identifying low-dimensional structures from high-dimensional tensor data, nor does it explore the learning theory approach. It would be beneficial to include experiments that directly evaluate these aspects to better align the empirical results with the theoretical framework.\n- Given the lack of application-based experiments, it is difficult to assess how this more general geometric framework compares to the more specific Riemannian optimization for t-product settings approaches introduced in earlier studies, especially in terms of their application to particular problems.", "was_revised": true, "final_justification": "I have raised my score from a 3 (Borderline Reject) to a 4 (Borderline Accept). The authors’ detailed rebuttal was effective, and the changes they made were substantial enough to move the paper across the acceptance threshold in my view.\n\n**Issues Resolved:**\n\nMy primary concern was the lack of empirical evidence to ground the paper’s abstract theory. The authors addressed this by adding a new synthetic experiment. While simple, it provides a much-needed proof-of-concept showing that their framework’s predictions align with observable behavior. This was the key missing piece.\n\nI also questioned how the work distinguished itself from prior optimization methods such as [1]. Their rebuttal provided a clear and convincing comparison, positioning their contribution as a more general, foundational modeling framework rather than a competing algorithm.\n\n**Unresolved Issues:**\n\nThe main limitation that remains is the gap between theory and practical, scalable algorithms. The work remains highly theoretical, and its immediate applicability is limited.\n\n**Weighting and Rationale:**\n\nI assign significant weight to the new experiment. For a foundational theoretical paper, having at least some empirical grounding is critical. Its addition shifts the work from being purely speculative to offering verifiable insights. While the absence of a mature algorithm prevents a higher score, I now see this as an acceptable limitation for the scope of the paper, rather than a fatal flaw. The paper is now strong enough to warrant publication.\n\n[1] MAO, Xianpeng, WANG, Ying, et YANG, Yuning. Computation over Tensor Stiefel Manifold: A Preliminary Study. Journal of the Operations Research Society of China, pages1–49, 2024.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive review. We are grateful for the recognition of the paper’s theoretical rigor, its structured presentation, and its relevance to emerging t-product tensor models in signal and vision applications. Below, we clarify the scope, contributions, and experimental design in response to the reviewer’s concerns.\n\n**Summary of Contribution and Scope**\n\n- This work establishes a *general framework for transform-consistent tensor geometry* based on the t-product algebra, aiming to unify discrete and continuous tensor modeling under a common algebraic structure for tensor learning. \n- Our primary contribution is the development of an *algebraic-geometric foundation*—drawing from sheaf-theoretic tools—that supports *differential-geometric notions* aligned with the algebraic properties of the t-product. This framework enables the formulation of manifold testing and fitting, learning guarantees, and geometric regularization strategies. \n- While the primary focus is theoretical, we also demonstrate *practical implications* via a bidirectional modeling example (BTR). \n- We agree that further integration with large-scale optimization remains a promising direction for future research.\n\n---\n\n> **Weakness 1 & Question 3 (Experimental Validation of Theoretical Results)**: “The experimental section does not adequately address the proposed hypothesis testing and fitting methods for identifying low-dimensional structures from high-dimensional tensor data, nor does it explore the learning theory approach.”\n“Could similar experiments be conducted on simpler problems using the analyzed learning theory approach?”\n\nWe appreciate the reviewer’s concern and agree that directly validating the theoretical results is important. Our original experiments focused on showcasing the modeling utility of the BTR framework via clustering, denoising, and inverse recovery tasks, but did not directly verify the theoretical results.\n\nTo better support the theoretical components—particularly the **t-manifold fitting theory**—we conducted a new set of *controlled synthetic experiments* aimed at examining whether *t-manifold structure can be recovered* from noisy observations. Specifically:\n\n- **Setup**: We sample data points from known low-dimensional t-manifolds (namely, a t-circle and a t-torus) defined in the transform domain. These constructions serve as illustrative examples where the underlying geometry is analytically known and can be perturbed with controlled Gaussian noise. \n- **Procedure**: Our proposed manifold fitting algorithm is applied to the noisy observations in an unsupervised manner, without access to the ground-truth structure during estimation.\n- **Metric**: We evaluate both the average and maximum pointwise distances between recovered and ground-truth samples, averaged over 10 independent trials.\n- **Results**: As shown below, recovery error consistently decreases with sample size.\n\n|Sample Size|t-circle Avg Dist|t-circle Max Dist|t-torus Avg Dist|t-torus Max Dist|\n|-|-|-|-|-|\n|50|0.0364 ± 0.0044|0.1377 ± 0.0316|0.1111 ± 0.0072|0.3102 ± 0.0065|\n|100|0.0212 ± 0.0013|0.0775 ± 0.0131|0.0711 ± 0.0037|0.2919 ± 0.0196|\n|200|0.0151 ± 0.0014|0.0569 ± 0.0069|0.0414 ± 0.0025|0.2077 ± 0.0230|\n|400|0.0116 ± 0.0006|0.0473 ± 0.0079|0.0258 ± 0.0009|0.1210 ± 0.0124|\n|800|0.0096 ± 0.0004|0.0383 ± 0.0053|0.0187 ± 0.0005|0.0893 ± 0.0073|\n\nThe observed monotonic decrease in both average and maximum recovery error with increasing sample size provides empirical evidence that our fitting procedure operates in a manner consistent with the theoretical guarantees, under moderate Gaussian noise.\n\nWe note that these experiments use simple synthetic t-manifolds as sanity checks to demonstrate viability under controlled settings. Extending to more complex or real-world manifolds and analyzing robustness to assumption violations is an important direction for future work.\n\n---\n\n> **Weakness 2 & Question 1 (Relation to Prior Riemannian Optimization Approaches):** (i) the lack of application-based experiments makes it difficult to assess the practical value of the framework compared to prior work such as [1], and (ii) it remains unclear whether the framework supports the same class of optimization problems and how the optimization potential of our approach compares to [1].\n\nWhile [1] introduces a Riemannian manifold structure over the set of 3rd-order tensors satisfying $\\mathcal{X}^\\top \\* \\mathcal{X} = \\mathcal{I}$ (termed the *tensor-Stiefel manifold*) and develops associated tools such as tangent spaces, Riemannian gradients, and retractions to support conjugate gradient descent (CGD), our work adopts a broader *algebraic-geometric perspective* rooted in the t-product algebra.\n\nThe differences can be summarized as follows:\n\n* **Modeling Scope**:\n  [1] defines a fixed manifold tailored to orthogonality-constrained tensor models.\n  Our framework instead introduces a family of *transform-consistent t-manifolds* constructed from arbitrary t-modules, supporting a wider range of structural assumptions and data types (including discrete and continuous settings).\n\n* **Mathematical Tools**:\n  [1] builds on classical Riemannian geometry, using projections, retractions, and vector transports to enable manifold-constrained optimization.\n  Our approach develops *algebraic-geometric constructs*—such as t-scalar modules, sheaf-induced differentials, and transform-aligned charts—that are intrinsic to the t-product structure and support learning-theoretic analysis.\n\n* **Types of Guarantees**:\n  [1] provides *optimization-theoretic tools* for solving constrained problems like tensor completion.\n  Our framework focuses on *statistical learning guarantees*, including manifold testing, fitting, and generalization bounds, which address identifiability and sample complexity beyond algorithmic behavior.\n\n* **Optimization Potential**:\n Although we do not develop optimization solvers, our geometric components—metrics, gradients, and bi-module regularizers—can be incorporated into Riemannian optimization pipelines. \n  - When the *orthogonality constraint* is imposed and the *metric is defined as the sum of frequency component inner products*, it aligns closely with the tensor-Stiefel geometry [1].\n\n  - Our framework is more general, supporting *constraints beyond orthogonality*, such as fixed tubal-rank. In tensor completion, t-manifold optimization can enforce low-rankness without orthogonality, improving flexibility while remaining compatible with [1]'s CGD. \n  - Our framework also allows a *$\\mathbb{K}$-valued metric*, enabling finer-grained spectral representation and frequency-selective positive-definiteness, which may improve stability in noisy settings, for example by prioritizing low-frequency components to mitigate noise. \n\n  We will clarify these optimization insights in Appendix F.2 of the revision. While *realizing these potentials may require non-trivial algorithmic design and is beyond the current scope of this work*, our framework can, in principle, support conjugate-gradient–style methods for tasks such as tensor completion, joint diagonalization, and sparse PCA. Rather than replacing [1], it provides **a unified modeling foundation** to enhance the flexibility and expressiveness of transform-based optimization frameworks.\n\n    \nTo summarize the comparison more concretely:\n\n| **Aspect**     | **Tensor-Stiefel [1]**                                 | **Our Framework**                                                 |\n|-|-|-|\n| **Object**     | Stiefel manifold of 3rd-order tensors             | *Family* of t-manifolds induced by arbitrary t-modules            |\n| **Toolset**    | Classical Riemannian tools (CG, retraction, projection) | Transform-aligned charts, sheaf differentials, t-Laplacians, etc.       |\n| **Guarantees** | Optimization theory for specific tasks                  | Statistical learning guarantees: testing, fitting, generalization |\n| **Usage**     | Direct solver for orthogonality-constrained problems    | Structural priors or geometric regularizers in learning pipelines |\n\n**Preliminary Application Support**\n\nAlthough our current experiments do not involve explicit optimization over t-manifolds or Riemannian tools as in [1], they reflect the modeling implications of our framework. The BTR model, built on dual t-module coupling, derives from the proposed geometry and is evaluated on denoising, image clustering, and Poisson tensor completion—covering unsupervised learning, inverse recovery, and robustness to non-Gaussian noise (Appendix F.1). BTR’s dual t-module coupling suggests potential for regularizing [1]’s model in non-Gaussian settings like Poisson completion.\nWhile not optimization benchmarks, these examples show how the framework supports diverse modeling tasks. Geometric components like transform-consistent regularization and directional coupling may serve as inductive priors in future solvers.\n\nThus, our work complements [1]: while [1] applies Riemannian tools to a fixed manifold, we provide a general geometric foundation for broader tensor models. Though not yet integrated into solvers, our structures may inform future optimization via descent guidance or spectral regularization.\n\n---\n\n> **Question 2 (Clarifying the Relationship Between Theory and Experiments)**\n\nWe fully agree that this clarification is necessary. In the revision, we will state explicitly that:\n\n* Sec. 3 presents the theoretical framework and statistical guarantees over t-manifolds;\n* Sec. 4 demonstrates modeling consequences of t-product geometry, especially via the BTR structure in clustering and denoising;\n* The new t-manifold fitting experiment directly validates a core theoretical result and bridges the above aspects.\n\nWhile unified by t-product geometry, these sections address distinct aims: theoretical analysis versus modeling illustration. We will emphasize this distinction in the revision for clarity."}, {"review_id": "j7PjL845IT", "reviewer": "Reviewer_vx6o", "summary": "This manuscript focused on the geometric understanding of tensor learning via the t-product, and extended the differential geometry to the t-product algebra.", "questions": "[Q1] Continuous tensor learning has been well developed, e.g., [R1], so how to understand the necessity of ``is essential not only for unifying discrete and continuous tensor modeling''?\n\n[R1] Functional transform-based low-rank tensor factorization for multi-dimensional data recovery, ECCV, 2024.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: A unifying framework for discrete and continuous tensor modeling is developed, and a principled theoretical foundation for structure-aware tensor learning is established. The manuscript is well prepared, and the theoretical results are very sufficient.\n\nWeaknesses: 1. The description of motivation is a little ethereal. Actually, I'm more interested in the practical implications of expanding differential geometry to the algebra induced by the t-product. The utilization of t-manifold geometry to enhance model performance is not theoretically verified. 2. Despite the abundance of theoretical results, it is rather limited in its practical support due to the not always valid assumptions. 3. Comparatively, the experimental validation is relatively flimsy.", "was_revised": true, "final_justification": "Anyway, the authors' rebuttal did resolve my concerns. Overall, I would maintain a positive recommendation.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback. We appreciate the recognition of our theoretical development and your helpful suggestions for clarifying motivation, assumptions, and empirical interpretation. Below, we address each of the raised concerns in detail.\n\n> **Weakness 1**: Motivation is ethereal; practical implications unclear; no theoretical verification of impact on model performance\n\nWe appreciate the reviewer’s comment and fully agree that grounding the motivation in concrete modeling considerations is essential. Our work is driven by the increasing use of transform-based tensor models—particularly those based on the t-SVD—in applications such as video modeling, multi-modal learning, and graph analysis. These models operate in the transform domain, yet lack a unified geometric framework that:\n\n- Formalizes the algebraic role of t-scalars (e.g., tubes or time-domain functions) in structured representations;\n- Bridges discrete (e.g., frame sequences) and continuous (e.g., temporal signals) tensor data under a common formulation;\n- Enables transform-consistent differential tools (metrics, gradients, Laplacians) for principled analysis and regularization.\n\nTo address these gaps, we introduce the framework of t-product geometry, which offers a principled foundation for modeling and learning in tensor spaces governed by transform-based algebra. By grounding geometric reasoning in the structure of the t-product, the framework supports consistent reasoning across spectral modes and provides a theoretical basis for learning over such spaces.\n\nWhile this paper focuses on establishing the conceptual and theoretical underpinnings, we also explore initial modeling implications through the Bidirectional Tensor Representation (BTR), which draws directly on the t-product structure. In particular, BTR couples orthogonal t-modules to encode bidirectional regularities, departing from traditional flat models. This design reflects the potential of geometric constraints to guide modeling, and BTR achieves competitive results in image clustering and video denoising. We emphasize, however, that these examples serve to illustrate how the theory may inform future model design, rather than exhaustively validate the framework.\n\nFurthermore, to provide preliminary empirical support for the theoretical results beyond conceptual modeling, we conducted controlled synthetic experiments for t-manifold fitting. Specifically, we sampled data from frequency-domain-defined ground-truth structures (t-circle and t-torus), added Gaussian noise, and applied our proposed fitting procedure. The recovery error consistently decreased with sample size, aligning with the convergence guarantees established in our theoretical results:\n\n|Sample Size|t-circle Avg Dist|t-circle Max Dist|t-torus Avg Dist|t-torus Max Dist|\n|-|-|-|-|-|\n|50|0.0364 ± 0.0044|0.1377 ± 0.0316|0.1111 ± 0.0072|0.3102 ± 0.0065|\n|100|0.0212 ± 0.0013|0.0775 ± 0.0131|0.0711 ± 0.0037|0.2919 ± 0.0196|\n|200|0.0151 ± 0.0014|0.0569 ± 0.0069|0.0414 ± 0.0025|0.2077 ± 0.0230|\n|400|0.0116 ± 0.0006|0.0473 ± 0.0079|0.0258 ± 0.0009|0.1210 ± 0.0124|\n|800|0.0096 ± 0.0004|0.0383 ± 0.0053|0.0187 ± 0.0005|0.0893 ± 0.0073|\n\n\nThese results provide preliminary empirical evidence for the viability of our t-manifold estimation procedure and its consistency with the theoretical learning framework.\n\n> **Weakness 2**: Assumptions may not always hold, limiting practical support\n\nWe appreciate the reviewer’s thoughtful observation. As this work represents the first attempt to formalize a differential geometric framework over the t-product algebra, we adopt several simplifying assumptions aimed at ensuring conceptual clarity, mathematical consistency, and alignment with existing practices in transform-based tensor modeling.\n\n- **Transform-domain smoothness** is imposed on the frequency components of tensor data rather than on raw entries. This mirrors the slice-wise structure of t-SVD and reflects the spectral regularity widely observed in image, video, and signal processing tasks. It enables geometry to be defined in a transform-consistent manner. Further discussion is provided in Appendix Remarks 2 and 4.\n- **Existence of a unit element** is guaranteed under common discrete transforms (e.g., DFT, DCT), where the unit corresponds to the inverse image of an all-ones vector. In the continuous case (e.g., Fourier transform), the unit becomes the Dirac delta distribution, which lies outside the smooth function space. As noted in Footnote 2, this can be addressed by extending the scalar ring to include distributions or by adjoining a formal identity via standard techniques from C*-algebra.\n- **Algebraic interpretation of t-scalars** as abstract units (rather than physically observed signals) allows the construction of coherent tensor modules. This perspective supports the extension of differential operators in a consistent way, particularly under the sheaf-theoretic framework employed in our geometric formulation.\n\nWe acknowledge that these assumptions may not hold in every practical setting. However, they allow for a unified, algebraically grounded definition of smoothness, locality, and geometry that is compatible with the t-product. They are consistent with the structures underlying many successful t-SVD-based models. Relaxing these assumptions—for instance, to allow piecewise smooth transforms, multiscale dictionaries, or weakly-defined differential operators—is an important and promising direction for future research.\n\nRegarding the concern on “limiting practical support,” we emphasize that this work is theory-driven: its primary contribution lies in establishing a principled geometric foundation for tensor learning with the t-product. Rather than aiming to exhaustively address diverse application scenarios, our objective is to develop a tractable and expressive formalism that can serve as a basis for future extensions. As illustrated in the BTR case study, even partial realizations of the proposed geometry can already inform effective modeling strategies. We believe that a rigorous theoretical foundation is a necessary step toward the systematic advancement of practical tensor geometric methods.\n\n> **Weakness 3**: Experimental validation is relatively flimsy\n\nWe appreciate this concern and would like to clarify the intended role of the empirical section. This work is theory-centered, aiming to introduce a geometric foundation for transform-based tensor learning. Accordingly, our experiments are not designed as exhaustive performance evaluations, but as targeted illustrations of how the proposed framework can inform model design in diverse settings. We highlight three concrete directions:\n\n- **Image clustering and video denoising via BTR**: The Bidirectional Tensor Representation (BTR) model draws on the structural insights of t-product geometry. By coupling two orthogonal t-modules, it encodes directional low-rank constraints that depart from flat tensor models. This provides a concrete instantiation of the idea that curvature can be introduced through algebraic interactions in the t-product framework.\n- **Poisson tensor completion**: This task extends BTR to inverse problems under non-Gaussian observation models. The fact that the same geometric regularization principle adapts naturally to photon-limited settings underscores the flexibility of the approach, especially in low-sample regimes that challenge standard Euclidean or matrix-based assumptions.\n- **t-manifold fitting (new):** As noted in our response to **Weakness** **1**, we have additionally conducted small-scale synthetic experiments to assess whether t-manifolds can be reliably recovered from noisy data. These experiments are intentionally limited in scope and not intended as comprehensive validation. However, the observed trends—decreasing recovery error with increasing sample size—are consistent with the convergence behavior predicted by our theory. We view these results as exploratory, serving primarily to illustrate the feasibility of the theoretical framework under controlled settings rather than to establish practical robustness.\n\nTogether, these case studies demonstrate the modeling implications of the proposed geometry. While we do not claim exhaustive experimental validation, the presented results show that the framework is not only theoretically well-structured but also practically adaptable to a range of modeling scenarios.\n\n> **Question 1**: Why is unifying discrete and continuous tensor modeling essential, given existing work such as [R1]?\n\nWe thank the reviewer for pointing to [R1], which presents a functional factorization approach for continuous tensor data. While [R1] contributes an important tool for signal recovery, our work addresses a different and more general question: *how to endow transform-based tensor representations with a coherent geometric structure that enables reasoning across both discrete and continuous domains*.\n\nIn particular, our framework differs from [R1] in three key ways:\n\n- It defines **manifolds of structured tensors** by modeling local neighborhoods via t-modules, rather than relying on coordinate-dependent factorizations.\n- It develops **transform-consistent differential operators**, including metrics, gradients, Laplacians, and geodesics, enabling spectral geometric reasoning.\n- It supports **theoretical learning guarantees** for manifold testing, fitting, and function approximation under t-product-based structure.\n\nThe resulting framework subsumes both discrete tubes (e.g., image rows, video frames) and continuous signals (e.g., time-varying slices) within a unified algebraic-geometric language. By treating t-scalars as basic geometric entities and generalizing classical smoothness to the transform domain, we introduce a new class of non-Euclidean spaces tailored to tensor representations—beyond what is addressed in [R1]."}, {"review_id": "9PMxfq5nJY", "reviewer": "Reviewer_pcAJ", "summary": "The paper introduces smooth t-Manifolds which extend classical differential geometry to the algebraic structure induced by the t-product. It allows concepts from differential geometry such as metrics, gradients and geodesicity to transform-based tensor operations. This concept is then used with tensor neural networks for geometry aware learning. The learning part is tested on a simple case study with image clustering and denoising.", "questions": "- What kind of problems could possibly break the assumptions?\n- What is the limit of the method when pushing experimental results, in other words how well can the method scale and what will the bottlenecks become?\n- What would other applications be that this approach would fit well?", "limitations": "yes", "rating": 4, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Positive:\n- The paper is well written and structured.\n- The proposed framework is generic by focusing both on discrete and continuous cases.\n- The paper studies both on how the t-mainfold can be generated and how tensor networks can learn on it. \n- The Bidirectional Tensor Representation (BRT) seems an interesting structure that fits well the discrete tensor representation of an image.\n\nNegative:\n- There are some critical assumptions in the paper such as transform-domain smoothness and the existence of a unit element.\n- The denoising experiments seem to structurally fit very well to the BRT representation, which makes it hard to judge how well the method works for other tasks.\n- The paper did not push the experiments to see when the assumptions are not valid anymore and how this would affect the results.", "was_revised": true, "final_justification": "I read the rebuttal and it clarified some of my questions but I keep my score", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive comments, and for recognizing several key strengths of our work, including the generality across discrete and continuous settings, the development of t-manifold theory for learning, and the modeling value of the Bidirectional Tensor Representation (BTR). We respond to each concern in detail below.\n\n---\n\n> **Weakness 1**: Critical assumptions such as transform-domain smoothness and the existence of a unit element\n\nWe appreciate this important observation. As our work represents the first attempt to develop a differential-geometric framework grounded in the algebraic structure of the t-product, we have adopted assumptions that enable conceptual clarity, mathematical consistency, and compatibility with existing t-SVD-based models.\n\n- **Transform-domain smoothness** is imposed on frequency components, rather than on raw tensors. This aligns with the slice-wise nature of t-SVD, and reflects well-established spectral regularities in real-world data such as images, videos, and signals. This assumption facilitates the definition of local charts and geometric operations consistent with transform-based algebra. We further elaborate on this point in Remarks 2 and 4 of the appendix.\n- **Existence of a unit element** is standard in discrete settings (e.g., DFT/DCT), where the unit corresponds to the inverse transform of an all-ones vector. In continuous settings, the unit becomes a distribution (e.g., Dirac delta), which falls outside smooth function spaces but can be addressed via standard extensions—such as working in distributional rings or applying unitization techniques from C*-algebra. These extensions are discussed in Footnote 2.\n\nWe fully acknowledge that these assumptions may not hold universally. Nonetheless, they allow us to define smoothness, curvature, and differential operators in a tractable and algebraically meaningful way. Future work will aim to relax these assumptions, e.g., by exploring weak derivatives, piecewise smooth transforms, or multiscale settings.\n\n---\n\n> **Weakness 2**: Denoising experiments structurally align with BTR—what about generalization?\n\nWe agree that the denoising task naturally benefits from the bidirectional structure of BTR. However, our intention is not to present BTR as a universal model, but rather to use it as an **illustrative instantiation** of how ideas from t-product geometry—particularly dual t-module coupling—can inform the design of structured tensor models.\n\nTo demonstrate generality beyond the denoising scenario, we also present in Appendix F.1:\n\n- **Image clustering** tasks using the same BTR formulation, showing its utility in unsupervised settings;\n- **Poisson tensor completion**, a challenging inverse problem involving sparse, signal-dependent noise. This task is not structurally aligned with denoising, and highlights the adaptability of the framework to non-Gaussian data.\n\nThese examples cover a range of regimes including unsupervised learning, inverse recovery, and robustness under complex noise distributions. While our experiments are scoped to controlled settings, they collectively show that the proposed geometric perspective is not limited to a narrow application domain.\n\nMoreover, as emphasized in our conclusion, this work primarily aims to establish a theoretical foundation for t-product-based geometry. Practical extensions—such as more advanced models, scalable solvers, and applications in large-scale systems—are part of a broader research trajectory, several directions of which are outlined in Appendix F.2.\n\n---\n\n> **Weakness 3**: The paper does not experimentally explore assumption violations or failure modes\n\nWe fully agree that understanding the limitations of the framework under assumption violations is important. While the scope of this paper focuses on theoretical development, we have taken a first step by conducting **synthetic t-manifold fitting experiments** to empirically validate convergence under controlled noise conditions.\n\nIn these experiments, we generate noisy samples from ground-truth t-manifolds (t-circle and t-torus), and apply our proposed fitting procedure. As shown below, both average and maximum pointwise recovery errors decrease consistently with increasing sample size, thereby supporting our theory:\n\n|Sample Size|t-circle Avg Dist|t-circle Max Dist|t-torus Avg Dist|t-torus Max Dist|\n|-|-|-|-|-|\n|50|0.0364 ± 0.0044|0.1377 ± 0.0316|0.1111 ± 0.0072|0.3102 ± 0.0065|\n|100|0.0212 ± 0.0013|0.0775 ± 0.0131|0.0711 ± 0.0037|0.2919 ± 0.0196|\n|200|0.0151 ± 0.0014|0.0569 ± 0.0069|0.0414 ± 0.0025|0.2077 ± 0.0230|\n|400|0.0116 ± 0.0006|0.0473 ± 0.0079|0.0258 ± 0.0009|0.1210 ± 0.0124|\n|800|0.0096 ± 0.0004|0.0383 ± 0.0053|0.0187 ± 0.0005|0.0893 ± 0.0073|\n\n\nWhile these toy-scale studies do not exhaustively explore failure scenarios, they provide encouraging evidence that the estimation procedure is robust under moderate perturbations. Extending these results to stress-test the framework under sharp discontinuities, corrupted transforms, or ill-conditioned structures is an exciting direction for future work.\n\n---\n\n> **Question 1**: What types of problems could break the assumptions?\n\nThe framework may become inadequate under the following conditions:\n\n- **Highly irregular or non-smooth signals**, where frequency-domain regularity does not hold;\n- **Use of non-orthogonal, non-invertible, or data-driven transforms**, which may not preserve t-product algebra;\n- **Severe data degradation**, such as extreme sparsity, heteroscedasticity, or unstructured noise, which challenges the stability of spectral low-rank models.\n\nRobust extensions may require multiscale analysis, adaptive basis learning, or regularization techniques beyond low-rank priors.\n\n---\n\n> **Question 2**: What are the scalability limits and bottlenecks?\n\nThe primary bottleneck lies in the use of t-SVD and tensor nuclear norm computations, which require multiple SVDs over transformed tensor slices and scale cubically in spatial dimensions. To enhance scalability, several directions are promising:\n\n- **Approximate or truncated t-SVD** using randomized or low-rank approximations;\n- **Sparse or structured transform representations** that reduce per-slice complexity;\n- **Neural surrogates or geometry-aware models** that emulate t-product operations without full decomposition.\n\n---\n\n> **Question 3**: What are other applications where this framework may be useful?\n\nWe appreciate the reviewer’s interest in the broader applicability of the proposed framework. As summarized in Appendix F.2, the t-product geometry introduced in this work provides a general modeling foundation that may inform multiple lines of research involving structured tensor data. While the present paper focuses on the theoretical formulation, we outline below several areas where the proposed tools could be potentially useful:\n\n* **Representation learning**: The use of t-modules enables spectral-aware learning without flattening, which may benefit structured data modalities such as hyperspectral imaging, EEG, or multi-view video.\n* **Low-rank regularization**: Bi-module structures support regularization along multiple modes, which may offer improved robustness to direction-dependent noise or missingness, as demonstrated in our Poisson tensor completion experiments.\n* **Optimization over structured manifolds**: The framework introduces transform-consistent gradients and charts, which may be incorporated into algorithms for constrained tensor optimization, analogous in spirit to Riemannian methods.\n* **Graph-based tensor learning**: The t-Laplacian supports frequency-adaptive graph construction over tensor data, which may improve local structure modeling in spatiotemporal or multimodal tasks.\n* **Generative modeling**: By leveraging low-dimensional t-manifolds, the framework may provide a means of encoding geometric priors in generative models, potentially improving coherence in the transform domain.\n* **Spectral manifold learning**: Transform-slice geometry allows defining neighborhoods and distances directly in the frequency domain, offering an alternative perspective to traditional Euclidean manifold learning for tensor data.\n\nWhile these directions remain exploratory at this stage, they follow naturally from the geometric and algebraic constructions introduced in this work. We hope that the framework may serve as a foundation for future advances in both theoretical modeling and practical algorithm design."}]

Author rebuttal:

Meta-review: This paper presents a geometric analysis of tensor learning based on the t-product, with its main contribution being the development of a differential geometric understanding of the t-product–based formulation.

The key strength of the paper lies in its strong theoretical analysis of t-product geometry and the associated learning theory. In particular, it is valuable that the authors go beyond geometric properties to also establish and analyze a learning theory, which is both novel and insightful for the community.

The primary weakness of the paper is its presentation. The content is quite dense and therefore not easy to follow. While this difficulty is partly inherent to the mathematical nature of the work rather than due to poor writing, there is still room for improvement in clarity and accessibility. I strongly encourage the authors to take reviewers’ suggestions into account and revise the presentation for the camera-ready version.

Some reviewers also raised concerns about the empirical evaluation, practical benefits, and novelty over prior work. However, the authors addressed these concerns effectively in the rebuttal, and after discussion between reviewers and the AC, all reviewers are now positive about acceptance.

In summary, I believe the proposed learning theory of t-product geometry represents a fundamental contribution that can encourage further development in this area. I therefore recommend acceptance of the paper.
Decision: Accept (poster)