Tensor Wheel Decomposition and Its Tensor Completion Application
Authors: Zhong-Cheng Wu
Ting-Zhu Huang
Liang-Jian Deng
Hong-Xia Dou
Deyu Meng
Venue: NeurIPS 2022 Accept (2022)
Keywords: Tensor wheel decomposition
Computer vision
Tensor completion

Abstract: Recently, tensor network (TN) decompositions have gained prominence in computer vision and contributed promising results to high-order data recovery tasks. However, current TN models are rather being developed towards more intricate structures to pursue incremental improvements, which instead leads to a dramatic increase in rank numbers, thus encountering laborious hyper-parameter selection, especially for higher-order cases. In this paper, we propose a novel TN decomposition, dubbed tensor wheel (TW) decomposition, in which a high-order tensor is represented by a set of latent factors mapped into a specific wheel topology. Such decomposition is constructed starting from analyzing the graph structure, aiming to more accurately characterize the complex interactions inside objectives while maintaining a lower hyper-parameter scale, theoretically alleviating the above deficiencies. Furthermore, to investigate the potentiality of TW decomposition, we provide its one numerical application, i.e., tensor completion (TC), yet develop an efficient proximal alternating minimization-based solving algorithm with guaranteed convergence. Experimental results elaborate that the proposed method is significantly superior to other tensor decomposition-based state-of-the-art methods on synthetic and real-world data, implying the merits of TW decomposition. The code is available at: https://github.com/zhongchengwu/code_TWDec.

Tensor Wheel Decomposition and Its Tensor Completion Application
Zhong-Cheng Wu', Ting-Zhu Huang*., Liang-Jian Deng".,, Hong-Xia Dou?, Deyu Meng
School of Mathematical Sciences, University of Electronic Science and Technology of China
"School of Science, Xihua University
School of Mathematics and Statistics, Xi'an Jiaotong University
Pazhoo Laboratory (Huangpu)
wuzhch970166 63 com, wwwgzhiuuungggggtee @26 com, Langgiinn... www.eeestteeeeeu.ccomm
hongxia dou@mai xhu, edu. en, dymeng@mai xijta edu.cn cn
Abstract
Recently, tensor network (TN) decompositions have gained prominence in com- puter vision and contributed promising results to high-order data recovery tasks. However, current TN models are rather being developed towards more intricate structures to pursue incremental improvements, which instead leads to a dramatic increase rank numbers, thus encountering laborious aaper-parameter selection, especially for higher-order cases. In this paper, we propose novel TN decom- position, dubbed tensor wheel (TW) decomposition, is represented by set of latent factors mapped into specific wheel topology. Such decomposition is constructed starting from analyzing the graph structure, aiming to more accurately characterize the complex interactions inside objectives while maintaining a lower deficiencies. Furthermore, investigate the potentiality of TW decomposition, we provide its one numerical application, i.e., tensor completion (TC), yet develop efficient proximal alternating minimization-basse solving algorithm with guar- anteed convergence. Experimental results elaborate that the proposed method is significantly superior to other tensor decomposition -esssdd state-of- the-art methods on synthetic and real-world data, implying the merits of TW decomposition. The http::/hhhhuh..cm////momeee CodeeeWeeeeeedddWWeWeeee
which a high-order tensor
scale, theoretically alleviating the above
code is available
Introduction
Tensors, as higher-order generalization of matrices, naturally preserve the complex interactions inside high-order tensor data, e.g., multispectral images (MSIs), color videos (CVs), and hyperspectral videos (HSVs). Tensor decompositions refer to the approximation of high-order tensor data by using latent factors, which can be considered as the principal components or features, leading to the reduced storage cost and computational complexity in large-scale and intractable tensor data analysis. Previ- ously, different tensor decompositions have been proposed and widely applied various fields, e.g., signal processing [6] 7 19), machine learning 255 29 D1, and medical imaging (31). Among them, tensor singular value decomposition (1-VVD) 16), especially CANDECOMP/PARAPE
(CP) [3
and Tucker 226 decompositions,
the most classical and widely investigated [7 34
More recently, tensor network (TN) decompositions 227 9 14 [T] have consecutively been involved in computational mathematies and numerical analysis, and exhibit superior properties, such super compressing and efficient computing 15. Particularly, some TN models have been introduced
"Corresponding authors.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

h
12
1
Ix
R
IN
R
R:
R
R2
Ru
RN
I
R
L
R
(a)
TT decomposition
1
L
1x
L
1
h
I
R
12
R4.
I
R
Ry.
ly
I3
1
R
La
L
R+
R
I1
Ry
1
R4
1
(b)
TR decomposition
(c)
TW decomposition
Figure I: Graphical illustration of decomposition, TR one, and the proposed TW one.
to the computer vision community as powerful tools for high-order data recovery applications owing to their excellent characterization capabilities. The popular tensor train (TT) decomposition [2 is one such decomposition, in which an Nithorder tensor expressed as strictly ordered multi-linear products over one matrix, followed by N-2 third-order tensors and another matrix, graphically
illustrated Figure M a). Afterwards, tensor ring (TR) decomposition [35] 23 21,, considered as a generalized form of TT decomposition, instead expands the near-border two matrices in TT factors to two third-order tensors and then constructs the cyclically sequential multi- products over N Figure b).. Currently,
third-order tensor factors, fully-com cullycconnected tensor network (FCTN) decomposition (37. has also been explored by establishing a structure evolved from the complete graph in graph theory, where all decomposed Nthhorder factors are connected to each other. As of now, TT, TR, and FCTN formats have been properly evaluated for their effectiveness and successfully applied to variety of fields, e.g., classification [30], compressive
sophisticated decomposition, referred
sensing [5, and image/video recovery [ 10 36 3
28
37 [2).
Despite the great success achieved by TT, TR, and FCTN decompositions, three intrinsic limitations are rather reflected in their TN structures. More specifically, (i) TT, even TR, models only establish the connection over adjacent latent factors while ignoring other possible interactions between non- adjacent dimensions, leading to an inadequate relation construction. (ii) From the perspective of tensor subspace, the intricate features inside Nthhorder tensor can be well-preserved in the same dimensional tensor space without destroying the structures. However, when N 2 4, TT and TR models express Nth-order tensor employing only third even lower-order factors, which definitely cannot form the desired space. (iit) From FCTN-ranks, FCTN model generates (N² N)/2 hyper- parameters, which scale quadratically with tensor order N. Thus, numerically specifying an optimal collection of FCTN-ranks can be challenging for higher order, making it a costly and tedious task.
Given the above limitations, argue that more rational TN model should include three aspects. Namely, (i) all factors with dimension modes (i.e., It, k = 1,2, .N) are certainly interconnected with their adjacent ones, while remaining equivalently close possible the others. (ii) At least one otherrder factor is required to physically inherit the complex interactions from an N tt-orrrer tensor, thus expecting superior characterization for high-order tensors without sacrificing parameter storage. (iii) The TN structure keeps the scale of of ranks. Remarkably, the first two folds correspond the characterization capabilities, which are crucial for TN models in high-order data recovery. Consequently, we propose the tensor wheel (TW) decomposition, which decomposes an Nith-order tensor into both N fourth-order ring factors and an N Nth-order core factor, and establishes the multi-linear products by wheel topology (see Figure [c)), aiming agree above superior properties. The main contributions of this paper are as follows:
lower, i.e., the smaller collection
We analytically propose a novel TW decomposition, which allows more expressive characterization for correlation than TT and TR formats, especially preserving the linear scaling for the number of hyper-parameee (TW-ranks) with increased tensor dimension (Section 2.3h. Also, present an alternating least squares (ALS)-based algorithm for learning TW decomposition (Section 2.4).
We further provide one numerical application of TW decomposition, i.e., tensor completion (TC), and then formulate a TW recomposiition-aaee TC model, abbreviated TW-TC. Moreover, we develop an efficient proximal alternating minimization (PAM)-based algorithm to solve the proposed TW-TC model and establish the thcoretical proof of algorithm convergence (Section 3

---

J1
Ju
Im Jm
a-based banfokiing
Jnu Jau
0.22:
Ud Ry folding
Imu
Jns
J1
Ju
Jmn tn Jm
Jun Jun
Imter t Imt
Immss
Jisos
Is
un based bunfolding
I ss Jni
X
1:
tramppsing
X Ka
Figure 2: Graphical illustration the generalized tensor k-contraction Definition
2 Tensor Wheel Decomposition
2.1 Notations and Preliminaries
In general, scalars, vectors, matrices, and tensors are denoted by lowercase letters, e.g., æ, lowercase bold letters, c.g., x, uppercase bold letters, e.g.. X, and calligraphic letters, e.g., X, respectively. For an Nth-order tensor X' E xp/xxxx---xxx its element on index (i1, ... iin) is expressed by
X (i1, 12,, ... ;in). The Frobenius norm is defined as ||>||| VV .N X i2, , For p, q N and S q. we especially denote by Xp:g the sequence (Xp. ;,)) for brevity.
2.2 Basic Theory
To numerically construct the TW decomposition, firstly develop several new tensor definitions. Definition 1 (Tensor Permutation). Given Nth-order tensor X / IN. Assuming that vector N = (M1, ,IN) ) is a specific reordering of vector 112..... ,N,, then we define the vector n n-base permutation of tensor X as a tensor X E RRln x X X..XIx,,, whose elements obey
X" (Iags bag: .ix))
(2,,,,,"".. ,iN).
(1)
Definition 2 (Generalized Tensor k-Unfolding). Let n = (n1, T...... JNN) a reordering of vector (1, 1, ... .N), then for an Nth-order lensor X E 1 1111xxxx.xIx the generalized k-unfolding (0 < k S N, E Z) of X is defined as a matrix [|nik] E Rlli= Ju x[T]=e+1 Io;, which requires
X Xin:b] (1n bng ... .n Pay 1 1 (nx)
2n (img ings" ing);
(2)
where X" is the tensor permutation of X, the multi-indices ing fng " ing and inyt Payto ... bay are defined by 1 + (i, 11) 1)ITi== In, and ΣN Ex=41 (in, 1) 1)I1 Iny, respectively. When k is and N, X|x:0] E IR I I Ing and X|a:N] E IRl.. InpX1 x1 imply two generalized vectorizations. Conversely, its inverse operator yields = Folddmss (msk)) or X Fold (X for k 0, N.
Following Definition with associated multi-indices, X (A.1.2,... 1-1,84 N);1] be specialized as X()). which is the classical mode-k unfolding in 17 Also, ((., 2... k- 1):1] can be specialized X used in TR decomposition (35) Correspondingly, the mode-k folding operators of these two special unfoldings can be expressed as Fold(a) (-) and Fold<ı> (.), respectively.
Definition 3 (Generalized Tensor k- &-Contraction), Given an Mth-order tensor X 1 and an N Nth-order tensor y RR x/1x.x J2x... x..xXXx with & common modes (1 S k < min(M,N) k Z). Assume that two vectors m (m1, m2, ... ,mis)) and (M1,n2," n2, ... ...."",, respectively indicate the reordering of vectors (1, 2...... ..... ....... .M) and (1,2, ....... N), satisfying I Jn, for i 1.2, k. m&+1 < M242 <... < m& and nk++1 < T144 between X and y along the k modes specifies an (M+N- 2h)th-rrder tensor X x _m 3a in RR. mmpas X mmgg x Jugss x.. xxxx X
< nN. Then the generalized k-contraction mag. my y
which is given (also intuitively shown in Figure as follows,
X
ma m2. my y Fold .MNN MMNN22):MMMM Ri.
T mm:k) [m:ka))
where X mma] and Y [n:] are the m- m-based and n n-based k-unfoldings of tensors X and Y, respectively. Significantly, the generalized tensor k-contruction operator requires a left-side precedence and com-cummuleeive property. That is, neither the associative nor commutative properlies are satisfied.
3

---

2.3
Tensor Wheel Model
As illustrated in Figure c), let X' E R be an Nthh-rrder tensor, then TW decomposition aims to parameterize it by both N fourth-order ring factors GR E R R4 x T,xt4 R& 11, k = 1, 2. .N, and an Nith-order core factor C E R x2x by xLx When & N, RN+1 stands for R1. Mathematically, TW model establishes the element wise relation as follows,
Rs
Rx
L1
LN E Ix=1
(11, 12, .. ,iN) = ΣΣ
Σ
(4)
9=1n==
rx=1q==
Ge(Tk, ik, R,, Tk+1) ... GN iN, IN. ri (1, ,NN)--
According to the wheel topology, when the tensor contractions are executed in the simplest manner,
TW model 4 can be naturally expressed in the tensor form by X' Gi x1 92 xf... ... x1* Ge x GN x 2,4,.. .2N 1,2,... .N C, which provides the actual numerical computation for TW decomposition. Symbolically, we employ TW[G, G2. GN: CJ or more compactly, T )== C]] to denote TW decomposition. Moreover, the ring factors GE, k 2, .N, and core factor C are collec- tively called 7'W factors. Apart from the above basic form, TW decomposition also has numerous generalized tensor forms, but enumerating all the cases is laborious and pointless. In what follows, we exhibit two special tensor forms by Theorems I [nd aiming to illustrate its invariance property.
x 1,4 2N.1
Theorem 1 (Core-Centee Centered Circular Invariance). Given an Nth-order tensor X' E R x/2xxxxx IN and its TW decomposition 7W[[9 ) 2 Cl. Assume that n (M1, "2.."...... .NN') is the circular re- ordering of vector (1,2, ,N,, then the core-centente invariance gives X" = [W J*. Theorem 2 (Core-Connected) Invariance). Assume that the TW decompesition of X E IR..xx...xx is X" = (9. C where n = (M1, n22 ... ,Nx) is any vector that circularly shifts vector (1, 2, ............ ,N). Let vector (n1, nk, n2, ". 1,-1, Mk+1,'"' ,NN) (3 s k N, 1s E Z), then
X= (9m, xi{ (mx) x 3,4,- A+1,N+2
1,3,4, NN - A+4
1,,, 222-3,2 119.--26--,.6222mmm 2nng. n 2(N KNN-K)223.... 2(N-k)+ Vnsti. 18N2
where Unger m1-1 Gna x
xi 2(k 2) grz_s and V Vneti
Gozt1 x x 2(N k) Gax.
Theorem reveals any two non-adjacent ring factors with dimension modes (i.e., Ts., k = 1,2, N) can be connected by the core factor. Accordingly. TW topology can comprehensively establish all possible mode interactions of a high-order tensor without being impeded by other ring factors, which endows TW decomposition with adequate relation construction over TT and TR decompositions.
Theorem 3 (Tensor Subwheel Equation). Assume that the TW decomposition of X E 1
.Rx) ) is any vector that circularly shifts vector (1, 2, .... ,,, N). Let m = (N + 1, N + 2, 1, 2, 2.... ,N) and v = (2.4.... 1,3,, ,,2N 1).
is X" TW]lGmm J. 1 C" where n = (n1,122
then there inherently exists the following two tensor subwheel equations,
XenN> (G.x)(2) (M [m:]] and X [|INN,0] = oj (N fc)),N)
(5)
where M MAny E IR. Bug X!ng x..XTax-- x BuN xIn is an (N + 2)th-order subwheel tensor, which
merges all TW factors but GuNs ie., MĄnN Gnx-1 3.5.. 2N C*, and Nzc E IRto x x... xhay is another 2Nth-order subwheel 11.2,...,N
Gns xt X²² gn, x1 22+2
x2N-2
tensor obtained by only merging {Gm IN ie. Nwc- = Gng x? Gue x1 2k+2
x1,4
2N,1
Gax-
Theorem 3 clarifies that all TW factors, i.c., (9 and C, can be individually separated from TW] Hi CJJ. which contributes to those alternating direction-based algorithms, e.g., ALS and PAM ones in Sections 2.4 and 3.1 respectively. Furthermore, X|I.N.o] cl. (N yc) [v:N] in
formula 5 is essentially X C"
1.2.... .N .2 Nycc. appearing as tensor subspace representation.
2,4
Since all modes of X=" are contained in Nyc. C" is purely a coefficient tensor, and Niz0 can be regarded as a basis tensor. From such perspective, the intrinsic structure of high-order tensor can be well- peelerreeereee by the core factor C. leading to high characterization ability of TW decomposition.
Theorem 4. Assume that X' = H C1 with N ring factors GR E IR Rs x x La Re=1, then
k+1
Rank X ((k) = Rank X Rank <xx>)= S Lx
k= 1=,,,... N. Rs:
(6)
i=k
4

---

Algorithm The Alternating Least Squares Algorithm for TW Decomposition (TW-ALS).
Input: An Nth-order tensor X' E [R1xxxxxxxIN the manually given TW-ranks and threshold 6. Initialization: The randomized G E x Lax R k = ,, N, and C E R glxx gglxxaxx- Lax-xxx
while not converged do
LN 2: Record the last-update result Thass f TW 1019HK--1-0 : N do 4: Obtain the subwheel tensor Ma and vector m by Theorem
3: for
5:
(G))(2) arg min |X (G)) (M (mall (Gx)c2
6:
end for
7: Obtain the subwheel tensor Nye and vector y by Theorem 3
8:
C[1:N,o] <- arg min Xx::N.0] Ci1-N.o| /c)//v]]
c1.8.0
9: Check the convergence criterion: [[T
Tinal Thasel <6
10: end while
Output: TW factors Gk. k- = ... 1.,,...,,A and
Theorem justifies that scalars Rg and Lx. = 1,2, .N, can upper bound the rank of all dimension-mode tensor unfolding and control the low-rankness behavior to a certain extent. Thus, they are assigned as the TW-ranks, usually collected by a vector r (Ri, .....,,N, L1, .LN) E R2N The TW-ranks essentially determines the actual TN structure, e.g., Tucker topology when Rs- 1 and TR topology when LA- = 1 with k = 1,2, .... .N, graphically proving the superiority of TW decomposition over both Tucker and TR decompositions. Along such a perspective, TT, TR, and FCTN decompositions can be viewed as special case of the adaptive TN (ATN) decomposition [[]] which adaptively identifies TN structures by eliminating the relevant edges from the complete graph topology. Nevertheless, provided that decomposing the same N Nth-order tensor, TW decomposition exactly excluded from the setting of ATN decomposition since the latter cannot fit the structures with an internal core tensor, e.g., Tucker (26.. hierarchical Tucker [9.. and projected entangled state pairs (PEPS) 227 topologies. Compared with FCTN decomposition with (N2 / ranks, TW decomposition forms the collection of 2.V ranks, which grows only linearly rather than quadratically with the tensor order N, thereby improving its flexibility for higher-order applications.
The proposed TW topology allows for more expressive characterization capability using the linear number of sypee-parameters. Nonetheless, such a structure inevitably leads limitation on the storage costs. For a given N Nthhorder tensor X' E R IXX--XX whose TW-ranks is assumed as the same value R, i.e., r R, ................................ R), then TW decomposition requires O R + R^) parameter costs for storage. It is clear that the O + R^) scales exponentially with the tensor order N, resulting in greater storage burden for larger-scale and higher-order tensor recovery problems.
Limitation.
Unlike Tucker decomposition, TW one considers the potential relationship between adjacent factors and establishes a connection for higher characterization capacity. Actually, such strategy also reduces the loadings of core factor, which contributes a smaller 3 in TW-ranks than Tucker decomposition, thus alleviating the limitations of high storage and computational complexity.
2.4 Alternating Least Squares (ALS) Algorithm
In this section, we develop ALS-based algorithm to perform TW decomposition, called TW-ALS. Given X' E I xl2xx.xxxx and its predefined TW-ranks, then the TW-ALS algorithm mainly aims to seek set of TW factors such that relative error between X' and TW[ IN 1 C] is minimum, i...
min G1.62," Cv.CC || - TW
(7)
Based on Theorem 3 minimization problem 7 can be optimized under the ALS framework by alter- natively solving N + least-squares subproblems, i.e., min(G.))) ||X - (2) (M ( A 1,2, N. and minepu.o.o [|x]=N1o] Cp.v.o (Nrc) lr. Compared to formula (5), since
utilize the simplest case without permutation for updating C, X|INNo] and C[I:N,o] imply X x and 2t [[: with n 21,2,... ..,,,, ,N), respectively. The numerical procedure detailed in Algorithm
5

---

Algorithm 2 The Proximal Alternating Minimization (PAM)-Based Solver for TW-TC Model.
Input: The observed tensor F E IR x12x--IIN the location set 92 the TW-ranks r, the step size g. = =020, 0.001, Emax = 1000, J indicating the complementary set of 92, and e = 10 Initialization: The iteration step = 0, x = F,, and the randomized g
x R 1111,
k 1,2, .N, and (( 1 121xx1xxxxxx,
1: while not converged and t < mmax do
2: Determine the subwheel tensor M and vector m relying upon Theorem
#k
3: g((+1) (2 (x <ke
(M
#k [m:3] p
#k)m33
)2)) (())) (MM))/((M)))
(t)
#/ Ak) [m:3] (M
[ma] pI)}
4: Determine the subwheel tensor A and vector relying upon Theorem
5: i > 200 and mod(t, s) + 0 (i.e., modulo is not 0) then
6:
c(++1) = C(
7: else
c(++1)= = Fold IN (LNN,)) (C))
KUN + PO( (CN (C) (N [e) + (D))-
8:
9:
end if
x(++1)
Pn- { " +200)))))+++
Pa(F).
11:
Check the convergence criterion: ||(((++1) - X. 1=/1/001 <c.
t+
13:
end while
Output: The recovered tensor X' E RRxx4x-XIx
3 Numerical Application to Tensor Completion
To validate the rationality and superiority of TW decomposition, we employ it to one classical tensor recovery problem, i.e., tensor completion (TC), whose objective is recovering the missing entries from a partially observed tensor, then formulating TW decomposition- The proposed TW-TC method is committed to exploring latent TW factors from the partially observed tensor, then predicting the missing entries using these constructed factors. This essentially evaluates the potential capability of TW decomposition to characterize the high-order tensors.
TC (TW-TC) model.
3.1 Model and Algorithm
Given partially observed tensor F E - at location set Ω, then the proposed TW-TC model aims to identify an optimum-TW-ranke approximation X' E RR, xxxxxxxx. of the tensor F. Mathematically, the TW-TC model can be formulated as the unconstrained problem follows,
0, X' (C: Pa(C) = Pa(F)); X. X.91.N.C C 2 51 - TW ((X) with ((x) = oo, otherwise,
min
where Pa(()) is projection function keeping the entries in Ω while forcing all the others to zeros.
Obviously, the object function in formula (8) is not jointly but independently convex for all variables. Thus, we develop proximal alternating minimization (PAM) (2) based algorithm to alternately and recursively optimize each variable, leading the following procedure for A 1,2,-.. NN.
g(l+1) E arg min Gn
{ 11
TW[g Gk. (t) 1 K4HINN C
11
P f o }
1
2
cl(+1) E min e
12(0
TwIg +
P 2 ||C c( c [3:]
2
1 | ( + G1lx - x + ((x)).
x(++1) E arg
Bargmin min
x
where p >> is a proximal parameter. Following Theorem again, each univariate minimization problem in formula (9) can certainly be reduced to a linear least-squares problem, which is expressed by matrix form the supplementary material, as in Section 2.4 Then, the corresponding closed-form solutions can easily be given. In most numerical TC experiments, the variable C typicaaly tends to be

---

(PAM)
TR-TC (PAM)
FCTN-IC (PAM)
Tucker IC (PAM)
(PAM)
-0.3
order Sik- 2016
5ith order
2R-20
SIR 2018
1e (RSE)
10 (RSE)
Lot (RSE)
.2.5
-2
-1.5
-2.5
-3.5
0
20
40
60
80
100
120
100
200
300
400
500
600
700
10
15
20
25
30
35
Recevery Time
Recovery Time
Reconery Time (4)
-0.5
Long (RSS)
Los (SS6)
Log to (8S1))
-1.5
Ard order SR 40%
5th order
SR=4050
5
20
40
60
80
100
120
140
0
100 200 300
400
500
600
700
10
20
30
40
50
Recovery Time
Recovery Time
Recovery Time (6)
Figure 3: Recovery performance (i.e., (RSE) against recovery time. The TT-TC (PAM), TR-TC (PAM), FCTN-TC (PAM), Tucker-TC (PAM), and proposed TW-TC algorithms are evaluated on synthetic data under six different cases, The first column: 64 third-order tensor experiments with SR=20% and 40%: The second column: 81 fourth-order tensor experiments with SR=20% and 40%: The third column: 32 fifth-order tensor experiments with SR=20% and 40%. The smaller (RSE) value implies higher recovery accuracy.
stable when it exceeds 200 iterations. Therefore. we utilize the modulo operation (marked mod(-, with a step size to intermittently update after 200 iterations, aiming to save computing resources. The PAM-based solving procedure for the developed TW-TC model is summarized in Algorithm 2
3.2 Computational Complexity Analysis
For simplicity, we assume that all dimensions of an Nthh-rrdrr tensor X' E R x I x are I and all values of TW-ran ranks R, i... I4 and R₂ L1= R for k 2, .N. Then, the com- putational complexities involved in updating factors { C, and X' O(N Ei=2 T'
NI R IN)), O 122N), ), and O RV^) respectively. Therefore, the computational complex- ity for each nonintermittent iteration (i.e., t E {t: 0 < < 200 or mtd((, 0}) in Algorithm 2 is O(N) > 1 I'R + NI R3 + I [22) Akin to the storage parameters in the Limitation of Section 2.3] the computational complexity also increases exponentially with dimensionality N, implying that more computing resources required for higher-order tensor recovery tasks.
3.3 Convergence Analysis
This section consists in theoretically proving the convergence of Algorithm 2 Before that, we employ the 2 = (X, GINN, C) and (Z) to denote the variables and objective function in (). respectively.
Theorem 5. Let (2 JeEN be the sequence generated by Algorithm 2 then globally converges to critical point (i.e., local minimum point) 2 = =Xx, C*) of the objective function 4 ((
Proof. According to the finite length property (see [4] Theorem pag 479]), the sequence (2( {Z()) HEN
can converge to the critical point Z of (2),, since the following Lemmas 4 are satisfied.
Lemma 1. The objective function (Z in formula (8) a Kurdyka--ojasiewiic: (KL) function.
Lemma 2 (Sufficient Decrease Condition). Let (2(0) be the sequence generated by Algorithm 2(2
Then, the sequence [ Z heN explicitly satisfies T (()) $ ((((++1)
2(()
where [(++) 2 13 = [[((++)) - 22000 + Σx= [1g g + +[(((t)) - C/
Lemma 3 (Relative Error Condition). Let ( (2(0) HeN be the sequence generated by Algorithm 2 Then, there exists [ < (LD + (N + (1) - (00 where Lo sums the Lipschitz constants of (0g. N and 0 (Z), i.e., La EN= Lg, + Lc-
7

---

Lemma 4. Let HEN be the sequence generated by Algorithm 2 then it is bounded.
Considering the page limitations, the proof Lemmas is detailed in the supplementary material.
3.4 Numerical Experiments
In this section, we design substantial numerical experiments on synthetic and real- data to verify the superiority of the proposed TW-TC method compared with others, which are constructed based on several commonly used tensor decompositions. All the experiments are implemented in MATLAB (R2021a) on a computer of 64Gb RAM and Intel(R) Core(TM) 19-10900KF CPU: @3.70 GHz.
3.4.1 Synthetic Data Completion
To testify the superiority of TW decomposition over four related decompositions, i.c., TT 22). TR (35). FCTN (37), and Tucker (26) ones, we firstly conduct the numerical experiments on synthetic tensor data by horizontally comparing the TW-TČ method with their PAM-based TC algorithms, i.., TT-TC (PAM), TR-TC (PAM), FCTN-TC (PAM), and Tucker-TC (PAM), respectively. Since the Tucker factors comprise an underlying high-order structure, which may be closer to reality, we construct the synthetic tensors by Tucker decomposition using Tucker factors sampled from the uniform distribution (0, 1). More specifically, the synthetic data consists of 64 third-order, 81 fourth-order, and 32 fifth-order tensors, whose sizes are {11 x 12 x 13 4, Iz, 13 E (45, 50, 55, 60)). {I x 12 I3 x 14 11. Iz. I3, I4 E { (18, 20, 22 and {1 x I2 x I3 14 x Is I, Iz. Is. I, Is E {7,8)}, and Tucker-ranks are (6, 6, 6). (5, 5, 5, 5). and (3, 3, 3, 3).). respectively. All synthetic data are numerically renormalized into [0. 1]. Subsequently, the partially observed tensors are generated by random sampling with two sampling rates (SRs): 20%, 40%. For each case, the hyper-parametert i... TT-ranks, TR-ranks, FCTN-ranks, Tucker-ranks, and TW-ranks, also the storage complexities are provided in Appendix C. The performance is assessed by the residual standard error (RSE), i.e., - -xx-eeaall / ////t F. where is the recovered tensor.
Figure 3 depicts the recovery result versus recovery time for five PAM-based TC models under six different cases. Compared with TT-TC (PAM), TR-TC (PAM), and FCTN-TC (PAM), the TW-TC method achieves the optimal RSE values for the majority of samples using both competitive computing efficiency and lower storage complexity (see Appendix C), which experimentally demonstrates the superior characterization ability of TW decomposition and strongly supports the theoretical analysis presented in Sections [ and 2.3 Moreover, the faster computational efficiency of TW-TC over the TC model implies that TW decomposition can alleviate the curse of high-order core factor on computational burden, achieving greater applicability than Tucker decomposition.
3.4.2 Real-Worke Data Completion
Furthermore, we investigate the TW-TC method real-world data experiments by comparing it with several tensor decomposition based state-of-the-art methods, including HaLRTC [18) t-SVD [33] TMacTT B3, TRLRF [32), and FCTN-TC B7). All dyper-parameten adopted in these competitors are fine-tuned within a specific range suggested by their authors to achieve better performance. The real-world data mainly comprises two types, i.e., the multispectral image (MSI) and the video data. Similarly, all experimental data are numerically pre-normalized into JO, 1J. Afterwards, the partially observed tensors created by random sampling with three SRs: 5%. 10%, 20%. A popular metric, i.e., the peak signal-to-noie ratio (PSNR), is employed for quantitative evaluation. When applied to the MSI and video data, the mean of PSNRs across all channels is measured, termed MPSNR.
MSI Data. The tested MSI data sizes 200 x 200 x 31 (i.e., height x width x spectral), called Toy, which is cropped from the CAVE f Regarding TW-ranks, we empirically assign R2 Los and Rg = L = L2 based on observations of multiple third-order realdworld data experiments, and then select ranks R1. R2 and R3 from the candidate sets (3. 4, 5). {10, 15, 20, 25} and {2, respectively. Numerically, Table (Data Index: Toy) reports the MPSNR values under three different SRs and the average computing costs of all compared methods. Also, Figure presents their visual performances and the corresponding error maps when SR=20%. From the quantitative and qualitative results, we observe that our TW-TC model achieves higher MPSNR values and better visual reconstruction than the other methods, confirming its effectivvness for real-world third-order data.

---

Observed
Hal RTC
CR
1-VD
TMacTT 3
TRIRF
22
FGIN-TC
R71
TW-TC
Ground-truth
022
Figure 4: Visualization of the recovered results on MSI Toy (SR=20%) and CV News (SR=10%) experiments. The first two rows: the visual compositions (31-th. 20-th, and 10-th spectral bands R. G, and B channels. respectively) and the corresponding residual images for the MS1 Toy data; The bottom compositions (13-th frame) and corresponding residual images for the News data.
the visual
Table Quality metric (i.e., MPSNR) values average computational costs all compared methods under three SRs: 5%, 10%, 20%. (Bold: best; Underline: second hest)
Method
Data Index
Observed HaLRTC [8
L-SVD 33
TMacTT
TRLRF [32 FCTN-TC 37 TW-TC
5%
11.156
19.446
25.379
27.59
30.126
29.446
30.689
10%
11.391
24.087
29.527
32.134
35.428
34.169
37.121
Toy
20%
11.904
29.944
35.097
36.791
41.010
40.453
44.009
Time (s)
4.74
15.72
63.50
181.25
26.27
154.67
5%
8.806
15.185
26.791
25.972
26.942
25.064
28.887
10%
9.041
19.579
28.748
29.213
29.305
30.272
32.551
News
20%
9.553
23.935
34.533
32.367
32.923
34,897
36.206
Time (s)
7.64
31.76
36.40
521.30
128.62
342.36
5%
4,600
18,273
27.979
23.681
27.333
28.364
29.473
10%
4.834
21.198
31.690
26.646
28.834
33.721
34.459
Container
20%
5.344
24,901
35,460
35.204
35.726
37.536
38.259
Time (8)
8.88
23.70
71.35
204.78
121.31
339.04
5%
7.494
11.579
39,736
42.459
42.501
42,675
48,999
10%
7.729
22.795
44.756
47.563
47.132
49.067
$2.743
HSV
20%
8.240
32,631
50.123
51.065
51.961
53.566
54.678
Time (8)
9.32
19.57
144.98
459.79
101.77
493.38
Video Data. The tested video data contains two color videos (CVs): News and Container, and one hyperspectral vided" (HSV) [20]. The former two are size 144 x 176 x 20 (i.e., height x width x channel frame), and the latter is of size 40 x 40 20 20 (i.c., height width spectral X frame). When tested on two CVs, we also empirically set R3 = R4 and L1 = Lg = L3 = L1. and then specify ranks R1, R2, R3 and L1 from the candidate sets 3, 5). {10, 15, 20, 25), (3,4,35 and (2,3), respectively. More directly, we designate R₁ = R2 = Ra = R4 = 6 and I, T.2 = Tag La = 4 to the HSV data experiment for all three SRs, aiming to alleviate the challenge of determining TW-ranks for these data with more balanced dimensions. The numerical and graphical results of all compared methods are exhibited in Table I and Figure respectively. From the results, we detect that the TW-TC method invariably contributes the best performances and further enhanced advantages, which jointly illustrates the superiority of the TW topology for higher-order tensor characterization.
Besides, further studics, e.g., the effect investigations of TW-ranks, are conducted in Appendix D.
hhtp://traace.ussasu.cco edu/yuv/

---

Conclusion
This paper suggested a novel TN decomposition, i.e., TW decomposition, and presented an ALS-based algorithm to learn such a decomposition iteratively. The TW decomposition verified theoretically and numerically potential for more accurately characterizing the complex interactions inside higher-order tensors using only linear hyper-parameter scaling, significantly contributing to the advancement of TNs in machine learning. Moreover, we provided a TW decomposition based TC model, i.e., TW-TC, for high-order tensor data recovery and developed an efficient PAM-based solving algorithm. which enjoys a convergence guarantee. Substantial experiments on synthetic and real-world data confirmed that the TW-TC model markedly outperforms other decomposition- based state-of-the-- methods.
Acknowledgments and Disclosure of Funding
The authors would like to thank the area chair and anonymous reviewers for their insightful comments and constructive suggestions. This research is supported by NSFC (Grant Nos. 12171072, 1227 (083,, Natural Science Foundation of Sichuan Province (Grant No. 2022NSFSC0501, Key Projects of Applied Basic Research Sichuan Province (Grant No. 2020YJ0216), National Key Research and Development Program of China (Grant No. 2020YFA07 14001), Major Key Project of PCL (Grant No. (CC2202AA2)) and China NSFC projects under contract (Grant No. U1811461).
References
[1] A. Anandkumar, R. Ge, D. Hsu, S. M. Kakade, and M. Telgarsky. Tensor decompositions for learning latent variable models. J. Mach. Learn. Res., 15:2773 2-2277-2832,, 2014.
[2] H. Attouch, J. Bolte, and B. F. Svaiter. Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward backward splitting, and regularized Gauss-Seide methods. Math. Program., 137(1):91-129, 2013.
[3] J. A. Bengua, H. N. Phien, H. D. Tuan, and M. N. Do. Efficient tensor completion for color image and video recovery: Low-rank tensor train. IEEE Trans. on Image Process., 26(5):2466 2479, 2017.
[4] J. Bolte, S. -abach,, and M. Teboulle. Proximal alternating linearized minimization for noncon- and nonsmooth problems. Math. Program., (66611):4944999 2014.
[5] Y. Chen, T. Z. Huang, W. He, N. Yokoya, and X. L. Zhao. Hyperspectral image compressive sensing reconstruction using subspace-based nonlocal tensor ring decomposition. IEEE Trans. on Image Process., 29:6813 13-6888,, 2020.
[6] A. Cichocki, D. Mandic, L. De Lathauwer, G. Zhou, Q. Zhao, C. Caiafa, and H. A. Phan. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE Signal Process. Mag., 32(2):145-16) 2015.
[7] A. Cichocki, R. Zdunek, A. H. Phan, and S. Amari Nonnegative matrix and tensor factoriza- tions: Applications to exploratory multi-way way data analysis and blind source separation. Wiley, 2009.
[8] S. Gandy, B. Recht, and I. Yamada. Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Probl., 27(2):025010 2011.
[9] L. Grasedyck. Hierarchical singular value decomposition of tensors. SIAM J. Matrix Anal. Appl., 31(44):022-203 2010.
[10] L. Grasedyck, M. Kluge, and S. Kramer. Variants of alternating least squares tensor completion in the tensor train format. SIAM Sci. Comput., 3755)42424-422455 2015.
[11] M. Hashemizadeh, M. Liu, J. Miller, and G. Rabusseau. Adaptive tensor learning with tensor networks. Proc, NewrIPS 1st Workshop on Quantum Tensor Networks Machine Learning, 2020.
10

---

[12] W. He, Y. Chen, N. Yokoya, C. Li, and Q. Zhao. Hyperspectral super-resolution via coupled tensor ring factorization. Pattern Recognit, 122:108-288 280, 2022.
[13] L. Hitchcock. The expression of tensor or a polyadic sum of products. Math. Phys., 6(1-4):164-18)) 189, 1927.
[14] T. Huckle, K. Waldherr, and T. Schulte- -erburignnn Computations in quantum tensor networks. Linear Alg. Appl., 438(2)750--81, 2013.
[15] B. Khavari and G. Rabusseau. Lower and upper bounds the pseudo-dimension of tensor network models. In Proc. NeurlPS, 34, 2021.
[16] M. E. Kilmer, K. Braman, N. Hao, and R. C. Hoover. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl., 34(1):148-11)) 172, 2013.
[17] T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM Rev., 51(3):455- 500, 2009.
[18] J. Liu, P. Musialski, P. Wonka, and J. Ye. Tensor completion for estimating missing values in visual data. IEEE Trans. Puttern Anal. Mach. Intell., (51)):20--20, 2013.
[19] C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, and Yan. Tensor robust principal component analysis with new tensor nuclear norm. JEEE Trans. Pattern Anal. Mach. Intell., 42(4))925 998999993889611 2019.
[20] A. Mian and R. Hartley. Hyperspectral video restoration using optical flow and sparse coding. Opt. Express, 20(10): 10600):0668811673, 2012.
[21] R. Orús. A practical introduction to tensor networks: Matrix product states and projected entangled pair states, Aun. Phys., 349:1 17-158, 2014.
[22] I. V. Oseledets. Tensor train decomposition. SIAM J. Sci. Comput., 33(5):2295-2311 2011.
[23] D. PerezaGarcia, Verstraete. M. M. Wolf, and I. Cirac. Matrix product state representations. Quantum Inform. Comput., 7(5-6):401-433 2007.
[24] U. Schollwöck. The density-matrix renormalization group in the age of matrix product states. Ann. Phys., 326 2011.
[25] N. D. Sidiropoulos, L.. De Lathauwer, Fu, K. Huang, E. E. Papalexakis, and C. Faloutsos. Tensor decomposition for signal processing and machine learning. IEEE Trans. Signal Process., 65(13):3551-3588 2017.
[26] L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279-11, 1966.
[27] F. Verstraete, V. Murg, and J. I. Cirac. Matrix product states, projected entangled pair states, and variational renormalization group methods for quantum spin systems. Adv. Phys., 1432244 2008.
[28] W. Wang, V. Aggarwal, and S. Aeron. Efficient low rank tensor ring completion. In Proc. ICCV, pages 5697-5705, 2017. 2017.
[29] W. Wang. Y. Sun, B. Eriksson, W. Wang, and V. Aggarwal. Wide compression: Tensor ring nets. In Proc. CVPR, pages 9329-9338, 2018.
[30] Y. Yang, D. Krompass, and V. Tresp. Tensor-train recurrent neural networks for video classifica- tion. In Proc. ICML, pages 3891-3900, 2017.
[31] T. Yokota, K. Kawai, M. Sakata, Y. Kimura, and H. Hontani. Dynamic PET image reconstruction using nonnegative matrix factorization incorporated with deep image prior. In Proc. ICCV, pages 3126-3135. 2019.
[32] L. Yuan, C. Li, D. Mandic, J. Cao, and Q Zhao. Tensor ring decomposition with rank minimization on latent space: An efficient approach for tensor completion. Proc. AAAI, 3300119915--1158 2019.

---

[33] Z. Zhang and S. Aeron. Exact tensor completion using t-SVD. IEEE Trans. Signal Process., 65(6):1511-1526, 2017.
[34] Z. Zhang, G. Ely, S. Aeron, N. Hao, and M. Kilmer. Novel methods for multilinear data completion and de-noising based tensor-SVD. In Proc. CVPR, pages 3842-3849 2014.
[35] Q. Zhao, G. Zhou, S. Xie, L. Zhang, and A. Cichocki. Tensor ring decomposition. arXiv 1eeprimt:106.005333 2016.
[36] Q. Zhao, G. Zhou, L. Zhang, A. Cichocki, and S. I. Amari. Bayesian robust tensor factorization for incomplete multiway data. IEEE Trans. Neural Netw. Learn. Syst., 27(4):736-748, 2016.
[37] Y. B. Zheng, T. Z. Huang. X. L. Zhao, Q Zhao, and T. X. Jiang. Fully-connected tensor network decomposition and its application to higher-order tensor completion. In Proc. AAAI, 35(12)1110, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] Our abstract and introduction clearly describe our contribution, the algorithm, the theoretical results, and the experimental results.
(b) Did you describe the limitations of your work? [Yes] The limitations are clearly elaborated in later paragraph of Section 2.3 also in Section 3.2
(c) Did you discuss any potential negative societal impacts of your work? [No] This work is theoretical research and numerical application for tensor learning. For the time being, it does not present any foresceable negative societal impact.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results....
(a) Did you state the full set of assumptions of all theoretical results? [Yes] All assumptions are clearly stated and referenced in Section 2.2
(b) Did you include complete proofs of all theoretical results? [Yes] The proofs of all Theorems and Lemmas provided in Appendix A of the supplementary material.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] The code for TW-TC model is provided in Appendix E of the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section 3.4
(c) Did you report error bars (c.g., with respect to the random seed after running ex- periments multiple times)? [No] Testing our model is time-consuming and we lack sufficient resources/time for multiple repetitions of our experiments.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] The computational resources shown in the first paragraph of Section 3.4
4. If you are using existing assets (e.g., code, data, models) or curating/relcasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] See Section 3.4
(b) Did you mention the license of the assets? [No]
(c) Did you include any assets either in the supplemental material
URL? [Yes]
The code for TW-TC model is provided in Appendix E of the supplementary material.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [No] All datasets used in our work are publicly available. (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
12

---

5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "uP_0-6b8xHT", "reviewer": "Reviewer_Dt2K", "summary": "In this paper, a new Tensor Network (TN) topology is proposed, the Tensor Wheel (TW), which can be seen as a combination of Tensor Ring (TR) and Tucker network topologies. The paper explores the properties of TW and develop a Proximal Alternating Minimization (PAM) algorithm to make inference of missing values in tensor data. Experimental results of tensor completion on synthetically generated and real-world signals are presented and compared against some state-of-the-art tensor completion methods.", "questions": "-\tCould you please provide the details of the ranks used for each model used in Fig. 1 and Fig. 2? \n-      Can you compute the number of model parameters in each case (sum of number of entries in each core tensor for each model)?\n------------------------------------------------------\nI updated my scores after rebuttal. See my comments below", "limitations": "Yes, the authors have stated that the main limitation of their proposed model is its exponentionally grow of model parameters with the number of dimensions.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n-\tClear definition of the new topology and description of some of its properties\nWeaknesses\n-\t(major) I don’t agree with the limitation (ii) of current TN models: “At least one Nth-order factor is required to physically inherit the complex interactions from an Nth-order tensor”. TT and TR can model complex modes interactions if the ranks are large enough. The fact that there is a lack of direct connections from any pair of nodes is not a limitation because any nodes are fully connected through a TR or TT. However, the price to pay with TT or TR to model complex modes interactions is having bigger core tensor (larger number of parameters). The new proposed topology has also a large price to pay in terms of model size because the core tensor C grows exponentially with the number of dimensions, which makes it intractable in practice. The paper lacks from a comparison of TR/TT and TW for a fixed size of both models (see my criticism to experiments below).\n-\tThe new proposed model can be used only with a small number of dimensions because of the curse of dimensionality imposed by the core tensor C.\n-\t(major) I think the proposed TW model is equivalent to TR by noting that, if the core tensor C is represented by a TR (this can be done always), then by fusing this TR with the cores G_n we can reach to TR representation equivalent to the former TW model. I would have liked to see this analysis in the paper and a discussion justifying TW over TR.\n-\t(major) Comparison against other models in the experiments are unclear. The value of the used ranks for all the models are omitted which make not possible a fair comparison. To show the superiority of TW over TT and TR, the authors must compare the tensor completion results for all the models but having the same number of model parameters. The number of model parameters can be computed by adding the number of entries of all core tensors for each model (see my question about experiment settings below).\n-\t(minor) The title should include the term “tensor completion” because that is the only application of the new model that is presented in the paper.\n-\t(minor) The absolute value operation in the definition of the Frobenius norm in line 77 is not needed because tensor entries are real numbers. \n-\t(minor) I don’t agree with the statement in line 163: “Apparently, the O(NIR^3+R^N) scales exponentially”. The exponential grow is not apparent, it is a fact. \n------------------------------------------------------\nI updated my scores after rebuttal. See my comments below", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "oGe28T4F2c", "reviewer": "Reviewer_Fhbz", "summary": "This paper proposes a new tensor decomposition method to improve the downsides of existing ones. The proposed tensor wheel decomposition can be viewed as a hybrid between TR decomposition and Tucker decomposition, i.e., it is similar to TR decomposition but there is an additional core tensor of which the order is the same as the input data and all the factor tensors are connected (tensor-producted) to it. This has some advantages: (1) The Nth order structure of the input data can be preserved. (2) Nevertheless, it is more manageable than FCTN. (3) All the factors are now interconnected by the core tensor, unlike TR decomposition where some factors are far away so it is not easy to find direct relationships between them. Experiments show that, for a few synthetic and real data, the proposed method achieves the best performance.", "questions": "The TC experiments only use i.i.d missing conditions which are not realistic. How will it go if we use more realistic conditions, e.g., structured missing conditions?\n", "limitations": "I believe that this paper does not have any serious potential negative societal impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper proposes a clever way to bypass the downsides of the existing tensor network type decompositions. Unlike TT and TR decompositions, nonadjacent factors are connected via the core tensor and the original structure of the data is better preserved. At the same time, the number of connections is minimized (unlike FCTN) so that it is more manageable. The idea is quite intuitive and convincing, and accordingly, the superior performance is also convincing.\n\nThe proposed decomposition is found by a PAM-style method, and the convergence proof is provided. One shortcoming of the paper is that there is no deeper analysis of the decomposition. There are some theorems in the paper, however, they are about basic algebraic properties rather than deeper ones such as approximation error bounds.\n\nAnother thing that bothers me is the comparison with FCTN. In Figure 3, Why does FCTN-TC take a shorter time than TW-TC? This seems somewhat unintuitive, considering the main claim (TW decomposition being more manageable than FCTN). Is it because the best performing hyperparameter setting of FCTN happens to be very small? Moreover, it seems that FCTN-TC usually performs worse than TT-TC and TR-TC even though it has heavier connections. Why is this?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "RtMu9CNvSb", "reviewer": "Reviewer_Bfwm", "summary": "The paper proposed a new tensor network decomposition method, namely Tensor Wheel Decomposition. Different from prior tensor train or tensor ring decomposition, the tensor wheel decomposition introduced a new core tensor factors C in the factorization. An ALS algorithm (algorithm 1) and proximal solution (algorithm 2) are provided. Empirical study in completion task on synthetic data and real data shows that the new algorithm is better than the TT and TR based factorization methods. ", "questions": "[Q1] In the experimental results of Table1, since the rank are learnt different based on different algorithm, could the author list the rank parameter or the model size to understand the model complexity from different algorithm?\n\n[Q2] The tensor wheel seems to be more general as a combination of tucker decomposition and tensor ring decomposition. Could the author highlight the difference between the TW to Tucker decomposition, and include tucker decomposition as a comparable here? \n\n[Q3] In Figure 3, the data is generated via Tucker decomposition while the comparison is mainly between TT, TR, and TW. Will generating data via TW or TR with Tucker decomposition included, makes more sense for algorithm comparison? Additionally, from the systematic data, it seems TW algorithm is mostly slow and having large performance variance, where TR seems to be faster with smaller performance variance. Will there be any regularization methods to reduce TW performance variance? ", "limitations": "The paper proposed a new tensor wheel factorization method. Although it is a trivial but interesting extension, comparison to Tucker decomposition method is needed to demonstrate the improvement. From the existing results, model complexity and performance trade-off also needs to provided to demonstrate the efficiency of the new factorization method.  Last but not least, interpretations on the algorithms efficiency and performance variance is needed to show the advantage of TW as compared to TT and TR.  ", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper extends the existing tensor networks method to a new tensor factorization. The extension is trivial, but might still brings some new knowledge to the area. \n\nThe new tensor wheel factorization seems to be a combination of tensor ring factorization and Tucker factorization, without core tensor taking into considerations. While the comparison to Tucker factorization is not included in the paper. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "B0bOrAgVWKL", "reviewer": "Reviewer_CwGm", "summary": "This paper presents a new tensor network decomposition model for high-order tensor analysis. The proposed model factorizes a high-order tensor into a set of latent factors connected to a specific wheel topology. This can be viewed as a combination of the classic Tucker decomposition and the tensor ring decomposition. The numerical experiments of tensor completion application on both synthetic data and real-world examples demonstrate the superiority of the proposed method in terms of reconstruction accuracy. ", "questions": "- Considering the numerical results in Table 1, why are the reported PSNR values for competing methods TRLRF [A] and  FCTN-TC [B] different from those reported in Table 1 of [B] on the same data (container, news, and HSV)? \n- Why does the most complicated model FCTN-TC run much faster than others (e.g., 26.27s v.s. 154.67s) in Toy data\n\n[A] Tensor ring decomposition with rank minimization on latent space: An efficient approach for tensor completion, AAAI 2019  \n[B] Fully-connected tensor network decomposition and its application to higher-order tensor completion, AAAI 2021  \n\nIMO, this paper serves as a nice contribution to the tensor decomposition domain and should be accepted if the above question could be properly answered. \n***\n**post-rebuttal: I keep my original score though I feel the hyperparameter sensitivity issue should be extensively studied --- I'm not sure whether the superior results beyond other competing methods are partially owing to the test data selection and hyperparameter tuning. Missing to reveal this problem would definitely make the results weaker**", "limitations": "N.A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]  \n- Innovative construction of the tensor network model inheriting the merits of the classic Tucker decomposition model\n- State-of-the-art performance on real-world tensor completion applications\n- Linear scaling for the number of hyperparameters with increased tensor dimension\n\n[Weeknesses]\n- Increased computational burden compared with tensor-train and tensor-ring models\n- Expensive storage burden scaled exponentially with the dimension of tensor data -- sacrificing one of the significant benefits of the tensor-train model ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept