Title: Efficient and Learnable Transformed Tensor Nuclear Norm with Exact Recoverable Theory
Venue: NeurIPS 2023
Keywords: Efficient

Learnable Transformed Tensor Nuclear Norm

Exact Recoverable Theory
Abstract: The tensor nuclear norm represents the low-rank property of tensor slices under a transformation. Finding a good transformation is crucial for the tensor nuclear norm. However, existing transformations are either fixed and not adaptable to the data, leading to ineffective results, or they are nonlinear and non-invertible, which prevents theoretical guarantees for the transformed tensor nuclear norm. Besides, some transformations are too complex and computationally expensive. To address these issues, this paper first proposes a fast data-adaptive and learnable column-orthogonal transformation learning framework with an exact recoverable theoretical guarantee. Extensive experiments have validated the effectiveness of the proposed models and theories. 

Abstract

The tensor nuclear norm represents the low-rank property of tensor slices under a transformation. Finding a good transformation is crucial for the tensor nuclear norm. However, existing transformations are either fixed and not adaptable to the data, leading to ineffective results, or they are nonlinear and non-invertible, which prevents theoretical guarantees for the transformed tensor nuclear norm. Besides, some transformations are too complex and computationally expensive. To address these issues, this paper first proposes a fast data-adaptive and learnable column-orthogonal transformation learning framework with an exact recoverable theoretical guarantee. Extensive experiments have validated the effectiveness of the proposed models and theories.

1 Introduction

In real-life scenarios, many high-dimensional tensor data, such as hyperspectral images (HSIs), multispectral images (MSIs), and multi-frame videos, exhibit strong low-rank properties. Leveraging such low-rank structures of tensor data is crucial for solving tensor data restoration tasks, including but not limited to tensor completion (TC) [1, 2] and tensor robust principal component analysis (TRPCA) [3, 4]. Numerous methods have achieved outstanding results in practical applications by exploiting the low-rank property of tensors, such as video processing [5, 6], hyperspectral denoising [7, 8, 9], classification [10, 11].

There are various definitions of tensor rank, which differ from the rank used for matrices [12, 1]. Two well-known types of tensor decomposition are based on the CANDECOMP/PARAFAC (CP) and Tucker decompositions, which define the CP rank and Tucker rank, respectively [12]. These decompositions have been widely studied and have demonstrated competitive performance in lowrank tensor recovery. Computing the CP rank is known to be NP-hard, and a clear convex surrogate for this rank has not been established. On the other hand, computing the Tucker rank involves unfolding tensors along each mode into matrices, which may result in the loss of intrinsic high-order interactive information. In addition to these two ranks, the tensor tubal rank is also commonly used for tensor decomposition [13]. This rank is computed via tensor singular value decomposition (t-SVD), which was initially derived from a novel definition of the tensor-tensor (t-t) product [14]. Unlike other methods, t-SVD can operate on an integral third-order tensor without reshaping it into matrices, by using the discrete Fourier transform (DFT). For a third-order tensor A ∈ R n 1 × n 2 × n 3 , assuming that its third mode has a low-rank property, the transformed tensor A can be obtained as follows:

where × 3 denotes mode-3 tensor product [12], and L ∈ R n 3 × n 3 is corresponding DFT matrix which satisfies LL T = L T L = n 3 I . Then the definition of the tensor tubal rank of A is rank t ( A ) =

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

∑ n 3 i =1 rank ( A (: , : , i )) , where A (: , : , i ) is the frontal slice of A . Since the minimization of the tubal 34 rank is an NP-hard problem. Zhang et al. [15] built a convex surrogate of the tensor tubal rank, 35 named the tensor nuclear norm ( TNN ) by summing the matrix nuclear norm of each frontal slice 36 under DFT. Thus the DFT-transformed TNN is defined as: 37

Based on the DFT transformed TNN, Zhang and Aeron [2] and Lu et al. [3] give the exact recovery theorem for TC and TRPCA task by minimizing the TNN norm, respectively. Since then, many variants of DFT transformed TNN are proposed, such as weight TNN [16], partial sum of TNN (PSTNN) [17], Schatten-p norm TNN [18], p-shrinkage TNN [19], and many others [20, 21, 22].

Referring to Eq. (1), if we substitute the DFT matrix with another transform matrix/operator L , we can obtain a transformed tensor and corresponding induced TNN norms that differ from those obtained using DFT. Hence, a crucial question arises: what type of transform matrix/operator is appropriate? Intuitively, a suitable transform operator should satisfy the following three criteria:

Data adaptation. The design of transform operators must depend on the data to better utilize its characteristics, which is a recent viewpoint. Works such as S2NTNN [23], Q-rank [24], and SALTS [25] have employed various methods to learn transform matrices from data. S2NTNN uses deep neural networks, Q-rank introduces a new algebraic definition, and SALTS uses SVD decomposition. Although only Q-rank has theoretical guarantees, updating the transform matrix and tensor recovery are independent processes that take a long time, making it impractical for real-world tasks.

Theoretical guarantee Theoretical guarantees are crucial for both models and algorithms. Currently, the exact recoverable guarantees are based on fixed linear invertible transforms, such as DFT, discrete cosine transform (DCT) [26, 27, 28], wavelet transformation [29], and unitary transformation [30], but they lack adaptability to data. In addition, there are fixed complex transforms that do not have recoverable theoretical guarantees, such as framelet transform [31], and coupe transform [32].

Good Performance Good transforms should improve restoration performance.

To achieve these objectives, this paper leverages the tensor structure and exploits the low-rank property of the third mode of the tensor to learn an adaptive column-orthogonal matrix (COM) transform for each data instance. Specifically, we model the low-rank tensor to be restored as the product of a smaller-sized factor tensor and a COM. This modeling approach effectively captures the low-rank structure of the tensor and facilitates the learning of the COM transform. Moreover, due to the reduced size of the factor tensor compared to the original tensor, our proposed model achieves accelerated computation. Additionally, we provide theoretical guarantees for the recoverability of our proposed model. To facilitate comparison, we present some classical transform-based tensor nuclear norm (TNN) approaches in Table 1. It can be observed from the table that only our modeling approach can stand out by simultaneously considering data adaptability, theoretical guarantees, and computational efficiency. In summary, this article first presents an efficient learnable transformed tensor nuclear norm (TNN) model with recoverable theoretical guarantees.

2 Notations and Preliminaries

2.1 Notations

In this paper, we denote tensors by boldface Euler script letters, e.g., A . Matrices are denoted 74 by boldface capital letters, e.g., A ; vectors are denoted by boldface lowercase letters, e.g., a , and 75 scalars are denoted by lowercase letters, e.g., a. We denote I n as the n × n identity matrix. For a 76 3-order tensor A ∈ R n 1 × n 2 × n 3 , the frontal slice A (: , : , i ) is denoted compactly as A ( i ) . The tube 77

is denoted as A ( i, j, :) . The mode-n unfolding matrix of A is denoted as A ( n ) = unfold n ( A ) , and 78 fold n ( A ( n ) ) = A , where fold n is the inverse of unfolding operator. The moden product of a tensor 79 X ∈ R I 1 × I 2 × I 3 and a matrix A ∈ R J n × I n is denoted as Y := X × n A (see definition in [12]). 80 Some norms of vector, matrix and tensor are used. We denote the ‖ A ‖ 1 = ∑ ijk | a ijk | , the infinity 81 norm as ‖ A ‖ ∞ = max ijk | a ijk | and the Frobenius norm as ‖ A ‖ F = √∑ ijk | a ijk | 2 , respectively. 82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

2.2 Adaptive Transformation

For a third-order tensor A ∈ R n 1 × n 2 × n 3 , assuming that its third mode has low-rank property, it can be factorized as

where × 3 denotes mode-3 tensor product, U ∈ R n 1 × n 2 × r 3 , V ∈ R n 3 × r 3 ( r 3 ≤ n 3 ) satisfying V T V = I and r 3 = Rank ( A (3) ) . According to low-rank tensor decomposition (3), we have.

Therefore, if we regard U as a transformed tensor A , then V T can be regarded as the transform matrix L , and V is the inverse transform of V T . Then we denote the TNN under the COM learned from the data as the Adaptive TNN ( ATNN ), which can be reformulated as:

Remark 1 It should be noted that comparing Eq. (5) and Eq. (2), it can be seen that ATNN has faster solution efficiency than DFT-transformed TNN since the transformed tensor under COM transform has fewer slices. The stronger the low rank of the tensor, that is, the lower the r 3 /n 3 value, the higher the solution efficiency of ATNN can be obtained. However, since we want to ensure that the information of A with a rank of Rank ( A (3) ) before and after the transform will not be lost, i.e., A = A × 3 L T × 3 L is established, the condition r 3 ≥ Rank ( A (3) ) must hold.

2.3 T-product and T-SVD

Here, we give the definitions of t-product and t-SVD based on COM transform.

For A ∈ R n 1 × n 2 × n 3 , B ∈ R n 2 × n 4 × n 3 , the COM L T transformed tensor of A , B are A = A × L T ∈ R n 1 × n 2 × R , B = B × L T ∈ R n 2 × n 4 × R , respectively, via Eq. (1), then we define

Definition 1 (T-product) Let A ∈ R n 1 × n 2 × n 3 , B ∈ R n 2 × n 4 × n 3 and COM L T ∈ R r 3 × n 3 , ( r 3 ≤ n 3 ) satisfying L T L = I R , then the t-product under transform L T is defined as

where A = A × 3 L T ∈ R n 1 × n 2 × r 3 and B = B × 3 L T ∈ R n 2 × n 4 × r 3 .

According to the Definition 1, we have C = A ∗ L B ⇐⇒ C = AB since bfold ( C ) = C = C × 3 L T = bfold ( AB ) × 3 L × 3 L T = bfold ( AB ) × 3 ( L T L ) = bfold ( AB ) .

The t-product enjoys many similar properties to the matrix-matrix product. For example, the t-product is associate, i.e., A ∗ ( B ∗ C ) = ( A ∗ B ) ∗ C . We also need some other concepts on tensors.

Definition 2 (Transpose) The transpose of a tensor A ∈ R n 1 × n 2 × n 3 is the tensor A T ∈ R n 2 × n 1 × n 3 obtained by transposing each of the frontal slices.

Definition 3 (Identity tensor) A third-order tensor A ∈ R n × n × n 3 is called identity tensor if it 112 satisfies that each frontal slice is identity matrix, i.e., A ( i ) = I for all i = 1 , · · · , n 3 . 113

114

115

Definition 4 (Orthogonal tensor) A third-order tensor Q ∈ R n × n × n 3 is called orthogonal tensor if it satisfies that Q T ∗ L Q = Q ∗ L Q T = I .

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

Definition 5 (F-diagonal tensor) A tensor is called f-diagonal if each of its frontal slices is a diagonal matrix.

Theorem 1 (T-SVD) Let A ∈ R n 1 × n 2 × n 3 . Then it can be factorized as

where U ∈ R n 1 × n 1 × n 3 , V ∈ R n 2 × n 2 × n 3 are orthogonal, and S ∈ R n 1 × n 2 × n 3 is f-diagonal.

By replacing DFT transform with COM transform L T , we can prove the above Theorem [3].

Definition 6 (Tensor tubal rank [14] & TNN [3]) For A ∈ R n 1 × n 2 × n 3 , the tensor tubal rank, denoted as rank t ( A ) , is defined as the number of nonzero singular tubes of S , where S is from the t-SVD of A = U ∗ L S ∗ L V T . We can write glyph[negationslash]

And its tensor nuclear norm (TNN) is defined as

Using the t-product definition, we can get A = U ∗ L S ∗ L V T ⇐⇒ A = USV T , thus we have

by combing Eq. (5), Eq. (6) and Eq. (10).

3 Tensor Recovery via ATNN Minimization

3.1 Models

The observed tensor and the tensor that needs to be recovered are denoted as Y and X 0 , respectively. For the tensor completion (TC), the observation Y has the support set Ω ∼ Ber( ρ ) , i.e., P Ω ( Y ) = P Ω ( X 0 ) . For the tensor robust principal component analysis (TRPCA), the observation Y is corrupted with a sparse component E 0 (which may represent foreground and sparse noise), denoted as Y = X 0 + E 0 .

If the COM L T satisfying Eq. (5) is known, we can obtain the following two models:

Actually, it is often not possible to obtain L T that satisfies Eq. (5) in advance. Recall Eq. (5), where 135 the constraint A = A × 3 L T × 3 L shows that the information of A after the change and inverse 136 change will not be lost, as long as L is obtained from the SVD decomposition of X , Eq. (5) can be 137 satisfied. Hence, we can learn a suitable COM L from the data. By decomposing X as X = M × 3 L 138 and setting M = X × 3 L T , we can obtain the following alternative model to Eq. (12): 139

140

141

142

143

144

145

3.2 Incoherence Conditions

The incoherence condition is one of the most vital theoretical tools in low-rank recovery [33, 3, 4].

Below, we define ˚ e i as the tensor column basis and the tensor incoherence conditions similar to [3]. Definition 7 (Tensor Incoherence Conditions) For X 0 ∈ R n 1 × n 2 × n 3 with t-SVD rank R , it has the skinny t-SVD X 0 = U ∗ L S ∗ L V T . Then X 0 is said to satisfy the tensor incoherence conditions with parameter µ if

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

Algorithm 1 ADMMfor solving ATNN-RPCA model (13)

Input : Observation Y ∈ R n 1 × n 2 × n 3 , λ = 1 / √ max ( n 1 , n 2 ) , µ = 1 / ‖ Y ‖ ∗ , ρ = 1 . 25 , µ m = 1 e 7 µ , and the column number of learnable COM matrix r 3 .

1: Initialize Λ = E = O , M = bdiag ( U ) and L = V , where U , V is the low-rank tensor decomposition of among mode-3, i.e., unfold 3 ( Y ) = ( U ) × 3 L

2: while not convergence do

3: Update M := SVD 1 /µ (( Y -E + Λ /µ ) × 3 L T ) .

4: Update L := BD T , where [ B , C , D ] = svd ( unfold 3 ( Y -E + Λ /µ ) T unfold 3 ( bfold ( U ))) .

5: Update X := M × 3 L

6: Update E := S λ/µ ( Y -X + Λ /µ ) .

7: Update multipliers Λ := Λ + µ ( Y -X -E ) ;

8: Let µ = min { ρµ, µ m } .

9: end while

Output : recovered tensors X = M × 3 L and E .

3.3 Main results

We now demonstrate that both the model (12) and (13) possess exact recovery capability.

Theorem 2 (TRPCA Theorem) Consider ATNN-based TRPCA model (12) and (13). Suppose that X 0 ∈ R n × n × n 3 obeys the tensor incoherence conditions (14) and E 0 's support set, denoted as Ω 0 , is uniformly distributed among all sets of cardinality m . Then, there exist universal constants c 1 , c 2 > 0 such that ( X 0 , E 0 ) is the unique solution to model (12) and (13) when λ = 1 / √ n with probability at least 1 -c 1 ( nn 3 ) -c 2 , provided that

where ρ r , ρ s > 0 are some numerical constants.

Theorem 3 (TC Theorem) Consider ATNN-based TC model (12) and (13). Suppose that X 0 ∈ R n × n × n 3 obeys the tensor incoherence conditions (14) and Ω ∼ Ber ( p ) . Then, there exist universal constants c 0 , c 1 , c 2 > 0 such that X 0 is the unique solution to model model (12) and (13) with probability at least 1 -c 1 ( nn 3 ) -c 2 , provided that

Remark 2 It should be noted that although the model (12) and (13) are slightly different, they are the same in the proof of the exact recoverable theory. Assume that the optimal values of models (12) and (13) are ( ˆ X , ˆ E ) and ( ˆ M , ˆ L , ˆ E ) , respectively. A recoverable theory of model (12) requires proving ( ˆ X , ˆ E ) = ( X 0 , E 0 ) under the given L in advance. A recoverable theory of model (13) requires proving ( ˆ M × 3 ˆ L , ˆ E ) = ( X 0 , E 0 ) under the final learned ˆ L .

3.4 Solving Algorithm

This subsection derives efficient algorithms for solving the ATNN-based TRPCA and TC problem via the Alternating Direction Method of Multipliers (ADMM) framework [34].

We first write the augmented Lagrangian function of the TRPCA problem in Eq. (13) as:

where µ is the penalty parameter and Λ is the lagrange multiplier.

Due to page limitation, we provide Algorithm 1 for solving Eq. (17) using the soft-thresholding 168 operator S τ ( · ) [35] and the singular value soft-thresholding operator SVD τ ( · ) [36]. Additionally, for 169 the ATNN-TC model (13), we provide Algorithm 2 directly. For more detailed information, please 170 refer to the supplementary material. 171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

Input : Observation Y ∈ R n 1 × n 2 × n 3 with support set Ω , µ = 0 . 1 , ρ = 1 . 05 , µ m = 1 e 7 µ , and the column number of learnable COM matrix r 3 .

1: Similar initialization with Algorithm 1.

2: while not convergence do

3: Update M , L , X , Λ via the similar way in Algorithm 1.

4: Update E := P Ω ( Y -X + Λ /µ ) , where P Ω is projection operator.

5: Let µ = min { ρµ, µ m } .

6: end while

Output : recovered tensors X = M × 3 L .

3.5 Computational Complexity Analysis

As depicted in Algorithm 1 and 2, each iteration of the algorithm involves updating M through smallscale SVD computations, updating L through small-scale SVD computation, updating E through soft thresholding operations, and some matrix multiplications. For a third-order tensor X ∈ R n 1 × n 2 × n 3 , the time complexity of the soft threshold operator is O ( n 1 n 2 n 3 ) , the time complexity of solving L is O ( n 3 r 2 3 ) , and the time complexity of solving M is O ( r 3 n 1 n 2 2 ) . Thus, the overall time complexity of Algorithm 1 and 2 is O ( r 3 n 1 n 2 2 + n 3 r 2 3 + n 1 n 2 n 3 ) . Similarly, for the DFT-transformed TRPCA and TC models, the time complexity is O ( n 3 n 1 n 2 2 + n 1 n 2 n 3 ) . By comparing the two time complexities mentioned above, it can be observed that their ratio is positively correlated with r 3 /n 3 . Therefore, as the low-rank property of the tensor in the third dimension becomes stronger, the acceleration capability of the proposed algorithm in this paper also becomes stronger.

4 Experiments

In this section, we present numerical experiments to validate the main results stated in Theorems 2 and 3. Following the suggestion of Theorem 2, we set λ = 1 / √ max { n 1 , n 2 } for the TRPCA task in all experiments. However, it should be noted that further performance improvements can be achieved by carefully tuning the value of λ . The suggested value in the theory provides a useful guideline in practical applications. All simulations were conducted on a PC equipped with an Intel(R) Core(TM) i5-10600KF 4.10GHz CPU, 32 GB memory, and a GeForce RTX 3080 GPU with 10 GB memory.

4.1 Simulated Experiments

In this section, we will verify the correct recovery guarantee of Theorem 2 and 3 on randomly generated problems. We generate a tensor with tubal rank R as a product X 0 = P ∗ L Q T , where P and Q are n × R × n tensors with entries independently sampled from N (0 , 1 /n ) distribution and the COM L ∈ R r 3 × n is generated by orthogonalizing the random matrix with entries independently sampled from N (0 , 1) . For the TRPCA task, the support set Ω (with size m ) of E 0 with independent Bernoulli ± 1 entries is chosen uniformly at random, and the observation tensor is set as: Y = X 0 + E 0 . For the TC tasks, the observation Y is set as Y = P Ω ( X 0 ) .

Next, we investigate how the tubal rank of X 0 and the sparsity of E 0 (and missing ratio of X 0 ) affect the performance of model (12) and (13). We consider n = 50 and two values of r 3 , i.e., r 3 = 5 , 20 . We vary the sparsity ρ s of E 0 as [0 . 01 : 0 . 01 : 0 . 5] , the missing ratio ρ of X 0 as [0 . 01 : 0 . 02 : 0 . 99] , and tubal rank of X 0 as [1 : 1 : 50] , respectively. For each combination of ( R,ρ s ) and ( R,ρ ) , we perform 10 test instances and declare a trial successful if the recovered tensor ˆ X satisfies ‖ ˆ X -X 0 ‖ F / | X 0 ‖ F ≤ 0 . 01 . The fraction of successful recoveries are plotted in Figure 1. From Figure 1, we observe that there is a significant region where the recovery is correct for both models. Furthermore, two notable phenomena can be observed from the figure:

The phase transition diagram in the first row of Figure 1 closely resembles the second row, indicating that even if we don't know the correct COM L in the model (12), we can learn the COM L through model (13).

The phase transition diagram of r 3 = 5 is much better than that of r 3 = 20 for both TRPCA and TC tasks, which shows that it is necessary to consider the low-rank property of mode 3.

211

0.01

0.1

0.2

0.3

0.4

0.5

sparsity

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.01

0.1

0.2

0.3

0.4

0.5

sparsity

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.01

0.2

0.4

0.6

0.8

0.99

missing ratio

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.01

0.2

0.4

0.6

0.8

0.99

missing ratio

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

TRPCA under

r

3

= 5

TRPCA under

r

3

= 20

TC under

r

3

= 5

TC under

r

3

= 20

0.01

0.1

0.2

0.3

0.4

0.5

sparsity

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.01

0.1

0.2

0.3

0.4

0.5

sparsity

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.01

0.2

0.4

0.6

0.8

0.99

missing ratio

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.01

0.2

0.4

0.6

0.8

0.99

missing ratio

1

10

20

30

40

50

tubal rank R

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

TRPCA under

r

3

= 5

TRPCA under

r

3

= 20

TC under

r

3

= 5

TC under

r

3

= 20

4.2 Real Experiments

To validate the effectiveness of the proposed ATNN model in tensor recovery task, we conducted 212 experiments on various datasets, including hyperspectral images (HSI), multispectral images (MSI), 213 color video images, and surveillance videos. Due to page limitations, we have included the results of 214 robustness analysis, parameter settings for robustness, convergence verification, and more detailed 215 experimental outcomes in the Supplementary Material. 216

For comprehensive comparison, we have included additional state-of-the-art methods except those 217 listed in Table 1. These methods include CTV [42] and TCTV [4] for the TRPCA task, LRMC [33], 218 HaLRTC [1], UTNN [29], and OITNN [43] for the TC task, and GODEC [37], DECOLOR [38], 219 OMoGMF [39], RegL1 [40], and PRMF [41] for background modeling. Before conducting this 220 experiment, the gray value of each band was normalized into [0, 1] via the max-min formula. 221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

4.2.1 Hyperspectral and Multispectral Image Recovery

Two HSI images, i.e., WDC 1 and PaviaU 2 datasets are used. The sizes of the two data are 256 × 256 × 191 and 256 × 256 × 93 , respectively. Two MSI images in CAVE dataset 3 , i.e., Cloth and Beans are used. The size of the two data is 512 × 512 × 31 .

For the TRPCA task, we conducted experiments with six different levels of salt and pepper noise variance: 0.1, 0.2, 0.3, 0.4, 0.5, and 0.6. Table 2 reports the performance metrics of each method under a variance of 0.6, demonstrating that our ATNN outperforms all competing methods. Notably, our method achieves superior performance despite only utilizing the low-rank property of tensors, surpassing the performance of CTV and TCTV, which additionally exploit the local smoothness and low-rank property of images. Furthermore, our method exhibits comparable computational efficiency to RPCA, indicating that the introduction of the learnable COM matrix effectively reduces the time complexity of the model. To better visualize the comparison, we choose three bands of HSI to form a pseudo-color image to show four representative competing methods' visual restoration performance, as shown in Figure 2. From the images, it is evident that our proposed ATNN model can effectively remove noise and preserve more detailed information.

For the TC task, since all the methods achieve very accurate recovery results when the sample ratio (SR) is high, we test four different SRs: 0.01, 0.05, 0.1 and 0.2. The metric of each tested algorithm under an SR of 0.05 is placed in Table 3. As can be seen from the metrics in the table, our proposed method excels in recovery performance and running time.

4.2.2 Background Modeling from Surveillance Video

The aim of this task is to separate the background and foreground from Surveillance Video. We choose nine video sequences in Li dataset 4 with the known foreground of size 144 × 176 × 20 for testing, as shown in Table 4. It can be seen from the table that our proposed model is far ahead in

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

Clean: PSNR/SSIM

Noisy: 6.43/0.021

RPCA: 29.64/0.932

TNN: 19.85/0.355

CTV: 33.57/0.943

ATNN: 37.47/0.985

Clean: PSNR/SSIM

Observed: 6.24/0.014

TNN: 31.66/0.935

OITNN: 32.60/0.958

S2NTNN: 35.27/0.966

ATNN: 35.52/0.971

terms of evaluation metrics and running time. Even compared to the CTV model that simultaneously utilizes local smoothness and low-rank priors, our method outperforms it. It is worth noting that although tensor-based models have a higher performance ceiling than matrix-based models due to their ability to capture more complex structures, for TNN regularization, if the variation matrix is not well defined, the results can even be worse than matrix-based methods. This further highlights the necessity of learning the transform matrix.

4.2.3 Color Video Completion

We selected four color video sequences, namely Akiyo, Foreman, Carphone, and Mobile, from the open-source YUV video dataset 5 . To ensure efficient comparison, we considered the first 100 frames of each color video sequence. As the color video is represented as a fourth-order tensor in RGB format with dimensions 144 × 176 × 3 × 100 , we reshaped it into a tensor of size 144 × 176 × 300 . We adopted similar sample ratio (SR) settings as mentioned in Subsection 4.2.1. The performance metrics of all competing methods are presented in Table 5. It is evident that our proposed model consistently ranks within the top three, outperforming TCTV even under the Akiyo dataset. In comparison to other TNN models with fixed transform matrices, our model exhibits superior performance and remarkable computational efficiency. Furthermore, we provided the recovered images of some competing methods in Figure 3 for better visual comparison. For the convenience of observation, we have enlarged a part of the picture and placed the repair indicator below the picture. It can be seen that our proposed ATNN model has a strong ability to preserve the local information of the data.

5 Conclusion

In this paper, we introduce an efficient and learnable transformed tensor nuclear norm (TNN) model with a provable recovery guarantee. Our approach leverages the low-rank property of the third mode of the tensor to represent the tensor to be repaired as a combination of a small-sized tensor and a column-orthogonal matrix. The column-orthogonal matrix serves as an adaptively learned transform matrix derived from the data. By employing the nuclear norm on the small-sized tensor, our model achieves higher computational efficiency compared to existing methods. Additionally, we provide a theoretical framework that guarantees exact recovery for our proposed model with a column-orthogonal transform matrix. Extensive experimental results demonstrate the effectiveness of our approach and the validity of our theoretical findings.

Limitations There are two shortcomings in our work. Firstly, the recoverable theory does not explain how the low-rank property of the third dimension of the tensor affects the model's restoration performance. Secondly, the ATNN model only learns the low-rank property of the tensor, without incorporating image priors. These two points will be the focus of our future research.

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

References

Ji Liu, Przemyslaw Musialski, Peter Wonka, and Jieping Ye. Tensor completion for estimating missing values in visual data. IEEE transactions on pattern analysis and machine intelligence , 35(1):208-220, 2012.

Zemin Zhang and Shuchin Aeron. Exact tensor completion using t-svd. IEEE Transactions on Signal Processing , 65(6):1511-1526, 2016.

Canyi Lu, Jiashi Feng, Yudong Chen, Wei Liu, Zhouchen Lin, and Shuicheng Yan. Tensor robust principal component analysis with a new tensor nuclear norm. IEEE transactions on pattern analysis and machine intelligence , 42(4):925-938, 2019.

Hailin Wang, Jiangjun Peng, Wenjin Qin, Jianjun Wang, and Deyu Meng. Guaranteed tensor recovery fused low-rankness and smoothness. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023.

Johann A Bengua, Ho N Phien, Hoang Duong Tuan, and Minh N Do. Efficient tensor completion for color image and video recovery: Low-rank tensor train. IEEE Transactions on Image Processing , 26(5):2466-2479, 2017.

Wenrui Hu, Dacheng Tao, Wensheng Zhang, Yuan Xie, and Yehui Yang. The twist tensor nuclear norm for video completion. IEEE transactions on neural networks and learning systems , 28(12):2961-2973, 2016.

Yao Wang, Jiangjun Peng, Qian Zhao, Yee Leung, Xile Zhao, and Deyu Meng. Hyperspectral image restoration via total variation regularized low-rank tensor decomposition. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11(4):1227-1243, 2017.

Hongyan Zhang, Lu Liu, Wei He, and Liangpei Zhang. Hyperspectral image denoising with total variation regularization and nonlocal low-rank tensor decomposition. IEEE Transactions on Geoscience and Remote Sensing , 58(5):3071-3084, 2019.

Jiangjun Peng, Qi Xie, Qian Zhao, Yao Wang, Leung Yee, and Deyu Meng. Enhanced 3dtv regularization and its applications on hsi denoising and compressed sensing. IEEE Transactions on Image Processing , 29:7889-7903, 2020.

Pan Zhou, Canyi Lu, Jiashi Feng, Zhouchen Lin, and Shuicheng Yan. Tensor low-rank representation for data recovery and clustering. IEEE transactions on pattern analysis and machine intelligence , 43(5):1718-1732, 2019.

Jianlong Wu, Zhouchen Lin, and Hongbin Zha. Essential tensor learning for multi-view spectral clustering. IEEE Transactions on Image Processing , 28(12):5910-5922, 2019.

Tamara G Kolda and Brett W Bader. Tensor decompositions and applications. SIAM review , 51(3):455-500, 2009.

Canyi Lu, Jiashi Feng, Zhouchen Lin, and Shuicheng Yan. Exact low tubal rank tensor recovery from gaussian measurements. arXiv preprint arXiv:1806.02511 , 2018.

Misha E Kilmer, Karen Braman, Ning Hao, and Randy C Hoover. Third-order tensors as operators on matrices: A theoretical and computational framework with applications in imaging. SIAM Journal on Matrix Analysis and Applications , 34(1):148-172, 2013.

Zemin Zhang, Gregory Ely, Shuchin Aeron, Ning Hao, and Misha Kilmer. Novel methods for multilinear data completion and de-noising based on tensor-svd. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3842-3849, 2014.

Yang Mu, Ping Wang, Liangfu Lu, Xuyun Zhang, and Lianyong Qi. Weighted tensor nuclear norm minimization for tensor completion using tensor-svd. Pattern Recognition Letters , 130:411, 2020.

Taixiang Jiang, Tingzhu Huang, Xile Zhao, and Liangjian Deng. Multi-dimensional imaging data recovery via minimizing the partial sum of tubal nuclear norm. Journal of Computational and Applied Mathematics , 372:112680, 2020.

Hao Kong, Xingyu Xie, and Zhouchen Lin. t-schattenp norm for low-rank tensor recovery. 326 IEEE Journal of Selected Topics in Signal Processing , 12(6):1405-1419, 2018. 327

Chunsheng Liu, Hong Shan, and Chunlei Chen. Tensor p-shrinkage nuclear norm for low-rank 328 tensor completion. Neurocomputing , 387:255-267, 2020. 329

Yang Zhou and YiuMing Cheung. Bayesian low-tubal-rank robust tensor factorization with 330 multi-rank determination. IEEE Transactions on Pattern Analysis and Machine Intelligence , 331 43(1):62-76, 2019. 332

333

334

Jian Lou and YiuMing Cheung. Robust low-rank tensor minimization via a new tensor spectral k -support norm. IEEE Transactions on Image Processing , 29:2314-2327, 2019.

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

Hailin Wang, Feng Zhang, Jianjun Wang, Tingwen Huang, Jianwen Huang, and Xinling Liu. Generalized nonconvex approach for low-tubal-rank tensor recovery. IEEE Transactions on Neural Networks and Learning Systems , 33(8):3305-3319, 2021.

Yisi Luo, Xile Zhao, Taixiang Jiang, Yi Chang, Michael K Ng, and Chao Li. Self-supervised nonlinear transform-based tensor nuclear norm for multi-dimensional image recovery. IEEE Transactions on Image Processing , 31:3793-3808, 2022.

Hao Kong, Canyi Lu, and Zhouchen Lin. Tensor q-rank: new data dependent definition of tensor rank. Machine Learning , 110(7):1867-1900, 2021.

Tongle Wu, Bin Gao, Jicong Fan, Jize Xue, and Wai Lok Woo. Low-rank tensor completion based on self-adaptive learnable transforms. IEEE Transactions on Neural Networks and Learning Systems , 2022.

Wenhao Xu, Xile Zhao, and Michael Ng. A fast algorithm for cosine transform based tensor singular value decomposition. arXiv preprint arXiv:1902.03070 , 2019.

Baburaj Madathil and Sudhish N George. Dct based weighted adaptive multi-linear data completion and denoising. Neurocomputing , 318:120-136, 2018.

Canyi Lu, Xi Peng, and Yunchao Wei. Low-rank tensor completion with a new tensor nuclear norm induced by invertible linear transforms. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5996-6004, 2019.

Michael K Ng, Xiongjun Zhang, and Xile Zhao. Patched-tube unitary transform for robust tensor completion. Pattern Recognition , 100:107181, 2020.

Guangjing Song, Michael K Ng, and Xiongjun Zhang. Robust tensor completion using transformed tensor singular value decomposition. Numerical Linear Algebra with Applications , 27(3):e2299, 2020.

Taixiang Jiang, Michael K Ng, Xile Zhao, and Tingzhu Huang. Framelet representation of tensor nuclear norm for third-order tensor completion. IEEE Transactions on Image Processing , 29:7233-7244, 2020.

Jianli Wang, Tingzhu Huang, Xile Zhao, Taixiang Jiang, and Michael K Ng. Multi-dimensional visual data completion via low-rank tensor representation under coupled transform. IEEE Transactions on Image Processing , 30:3581-3596, 2021.

Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? Journal of the ACM (JACM) , 58(3):1-37, 2011.

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends R © in Machine learning , 3(1):1-122, 2011.

David L Donoho. De-noising by soft-thresholding. IEEE transactions on information theory , 41(3):613-627, 1995.

Jianfeng Cai, Emmanuel J Candès, and Zuowei Shen. A singular value thresholding algorithm 371 for matrix completion. SIAM Journal on optimization , 20(4):1956-1982, 2010. 372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

Tianyi Zhou and Dacheng Tao. Godec: Randomized low-rank & sparse matrix decomposition in noisy case. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011 , 2011.

Xiaowei Zhou, Can Yang, and Weichuan Yu. Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE transactions on pattern analysis and machine intelligence , 35(3):597-610, 2012.

Hongwei Yong, Deyu Meng, Wangmeng Zuo, and Lei Zhang. Robust online matrix factorization for dynamic background subtraction. IEEE transactions on pattern analysis and machine intelligence , 40(7):1726-1740, 2017.

Yinqiang Zheng, Guangcan Liu, Shigeki Sugimoto, Shuicheng Yan, and Masatoshi Okutomi. Practical low-rank matrix approximation under robust l 1-norm. In 2012 IEEE Conference on Computer Vision and Pattern Recognition , pages 1410-1417. IEEE, 2012.

Naiyan Wang, Tiansheng Yao, Jingdong Wang, and Dit-Yan Yeung. A probabilistic approach to robust matrix factorization. In Computer Vision-ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VII 12 , pages 126-139. Springer, 2012.

Jiangjun Peng, Yao Wang, Hongying Zhang, Jianjun Wang, and Deyu Meng. Exact decomposition of joint low rankness and local smoothness plus sparse matrices. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

Andong Wang, QiBin Zhao, Zhong Jin, Chao Li, and GuoXu Zhou. Robust tensor decomposition via orientation invariant tubal nuclear norms. Science China Technological Sciences , 65(6):13001317, 2022.

Peer reviews:
comments: SUMMARY:
This paper addresses the challenges associated with the tensor nuclear norm, which is a proxy for the low-rank property of tensors. Existing transformations for the tensor nuclear norm lack adaptability and theoretical guarantees due to fixed or non-invertible nonlinear approaches. This paper introduces a practical and data-adaptive framework and provides an exact recoverable theoretical guarantee. Experiments are included to support the theoretical guarantees.

STRENGTHS:
The paper contains solid theoretical results for a very challenging problem. It provides efficient algorithms for recovery that are explained well and clearly. The experiments include both real and synthetic data.

WEAKNESSES:
The paper is dense to read in some places, like in the slurry of definitions at the end of Section 2. This is likely due to space limitations, but reduces the readability of the paper. The main results include some pretty strict assumptions like incoherence and uniformly distributed support sets. Although some of these are also used in other results, it could be discussed further what kinds of tensors have these properties in practice and how sensitive the methods are to these.

QUESTIONS:
It might be good to elaborate in lines 46-52 about the importance of data-dependency, why it is better than non-adaptive approaches.

In that same section, in (3), it would help to specify what "good" in this context means, i.e. what metrics are important in performance

In Remark 1, the phrase "the higher the solution efficiency of ATNN can be obtained" is not clear, perhaps this can be re-phrased

Definition 7 reads awkwardly -- is the "skinny t-SVD" an assumption or part of the definition?

LIMITATIONS:
The authors discuss interesting future work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new approach for learning a data-adaptive and learnable column-orthogonal matrix (COM) transform for tensor nuclear norm (TNN) minimization. The authors show that the proposed transform can capture the low-rank structure of tensors more effectively than existing fixed or nonlinear transforms. They also provide theoretical guarantees for the exact recovery of tensors under the COM transform. The proposed method is applied to tensor completion and tensor robust principal component analysis tasks and achieves better performance and efficiency than several state-of-the-art methods.\n\nSTRENGTHS:\n1. The authors build up an adaptive tensor nuclear norm, and develop tensor completion robust tenor PCA model. \n2. To guarantee the theoretical correctness of the proposed models, the authors analyze two theorems to ensure the exact recovery capability.\n\nWEAKNESSES:\n1. The proof architecture is regular and simple and has been used in [1,2], thus the theoretical novelty of this paper is rather limited.\n2. The adaptive TNN model has been already proposed in many related works, but the authors missed some important related references, see the linear invertible TNN-based RPCA [1,2], the nonlinear transform TNN [3], and the subspace denoising strategy NGmeet [4]. Thus, a related works section should be considered to give detailed differences between these methods and the proposed ATNN.\n3. In the experimental section, the authors only compare the computation-cost methods and neglect the related adaptive TNN methods.\n\n[1] Lu C. Transforms based tensor robust PCA: Corrupted low-rank tensors recovery via convex optimization[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 1145-1152.\n\n[2] Lu C, Peng X, Wei Y. Low-rank tensor completion with a new tensor nuclear norm induced by invertible linear transforms[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 5996-6004.\n\n[3] Yisi Luo, Xile Zhao, Deyu Meng, and Taixiang Jiang, ‘‘HLRTF: Hierarchical Low-Rank Tensor Factorization for Inverse Problems in Multi-Dimensional Imaging,’’ IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.\n\n[4] He W, Yao Q, Li C, et al. Non-local meets global: An integrated paradigm for hyperspectral denoising[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 6868-6877.\n\nQUESTIONS:\n1. In Table I, why not compare the computation complexity of different methods? The speed comparison is unfair by the subjective evaluation\n2. What are the differences between the proposed adaptive TNN and SALTS?\n3. Please give more details about the differences between the proposed method and TRPCA-COM [1-2], including the model, and theoretical improvements.\n4. The authors state the superior computational efficiency of the proposed method compared with the other adaptive TNN, why not compare these methods in the experimental section? e.g., Q-rank, SALTS, etc.\n\n[1] Lu C. Transforms based tensor robust PCA: Corrupted low-rank tensors recovery via convex optimization[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 1145-1152.\n[2] Lu C, Peng X, Wei Y. Low-rank tensor completion with a new tensor nuclear norm induced by invertible linear transforms[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 5996-6004.\n\nLIMITATIONS:\nPlease see the above Weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

comments: SUMMARY:
This paper introduces a new method of learning a transformation that can make the tensor nuclear norm better capture the low-rank structure of tensor data. The method is fast, data-adaptive, and has a theoretical guarantee of being reversible. The paper shows that the proposed method works well in experiments.

STRENGTHS:
1. This paper presents a clear and reasonable idea that is well written and easy to understand. 

2. The paper proposes an adaptive transformation matrix, which has a better expressiveness than methods with fixed transformations. 

3. The paper proves that ATNN has exact recovery capability under certain conditions. 

4. The paper conducts extensive experiments to validate the theoretical results and demonstrate the effectiveness of ATNN.

WEAKNESSES:
1. The performance of ATNN is limited by the linear transformation. As shown in Table 3, S2NTNN achieves higher accuracy and faster speed than ATNN. Since the theoretical analysis of deep neural networks is challenging and remains an open problem, the theory of S2NTNN is harder to establish.

2. ATNN has an advantage over TCTV in terms of computational efficiency for color video completion task, but it cannot surpass TCTV in terms of overall model performance.

3. Typos: Eq.(2) $A^{(k)}\rightarrow A^{i}$

QUESTIONS:
How does ATNN handle the case of a third-order tensor with low-rank properties along all its three modes?

LIMITATIONS:
See Weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-07

comments: SUMMARY:
The authors propose a novel Adaptive TNN (ATNN) for tubal rank (t-SVD)-based low-rank tensor recovery and apply it to Tensor Robust PCA and Tensor Complete problems. Theoretical recovery guarantees have been established which is technically sound. However, tubal rank is known as an analog to matrix rank. Compared to CP rank and Tucker rank, the theory of tubal rank is relatively easy to work with. That said, the reported theoretical results are solid but no surprise. In my option, the proposed ATNN is an analog to matrix sketch, although fast, it is not equivalent to the standard t-SVD.I also think 'learnable' is a misleading term in this paper.

STRENGTHS:
Solid analysis reslut.

WEAKNESSES:
1. In my option, the proposed ATNN is an analog to matrix sketch, although fast, it is not equivalent to the standard t-SVD.Just like matrix sketch is not equivalent to the stardand SVD. I don't see this discussed in the paper.

2. When a matrix algorithm is learnable, something like 
      Deep convolutional robust PCA with application to ultrasound imaging, ICASSP 2019.
      Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection, NIPS 2021.
come to my mind. To me, 'learnable' is a bit misleading in the paper. Anyhow, the learnable feature of the ATNN is neither discussed enough nor numerically demonstrated through the paper.

3. The reported numerical results in Tables 3-5, the proposed approach doesn't show sufficient emperical advantage.

4. The notation is really messed up in some part of the paper. For example, in (12), \mathcal{E] should be \mathcal{S} (or the other way around). In (13), \mathcal{S} should be \mathcal{E}.

QUESTIONS:
See weakness section.

LIMITATIONS:
No negative societal impact was found.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-25