Recent work by Kilmer and Martin, [10] and Braman [2] provides a setting in which the familiar tools of linear algebra can be extended to better understand third-order tensors. Continuing along this vein, this paper investigates further implications including: 1) a bilinear operator on the matrices which is nearly an inner product and which leads to definitions for length of matrices, angle between two matrices and orthogonality of matrices and 2) the use of t-linear combinations to characterize the range and kernel of a mapping defined by a third-order tensor and the t-product and the quantification of the dimensions of those sets. These theoretical results lead to the study of orthogonal projections as well as an effective Gram-Schmidt process for producing an orthogonal basis of matrices. The theoretical framework also leads us to consider the notion of tensor polynomials and their relation to tensor eigentuples defined in [2] . Implications for extending basic algorithms such as the power method, QR iteration, and Krylov subspace methods are discussed. We conclude with two examples in image processing: using the orthogonal elements generated via a Golub-Kahan iterative bidiagonalization scheme for facial recognition and solving a regularized image deblurring problem. Key words. eigendecomposition, tensor decomposition, singular value decomposition, multidimensional arrays AMS subject classifications. 15A69, 65F30 1. Introduction. The term tensor, as used in the context of this paper, refers to a multi-dimensional array of numbers, sometimes called an n-way or n-mode array. If, for example, A ∈ R n1×n2×n3 then we say A is a third-order tensor where order is the number of ways or modes of the tensor. Thus, matrices and vectors are secondorder and first-order tensors, respectively. Third-order (and higher) tensors arise in a wide variety of application areas, including, but not limited to, chemometrics [24] , psychometrics [13] , and image and signal processing [5, 15, 23, 17, 8, 20, 29, 28, 30] . Various tensor decompositions such as CANDECOMP/PARAFAC (CP) [4, 7] , TUCKER [26] , and Higher-Order SVD [14] have been developed to facilitate the extension of linear algebra tools to this multilinear context. For a thorough review of tensor decompositions and their applications, see [11] . Recent work by Kilmer, Martin [10] and Braman [2] provides an alternative setting in which the familiar tools of linear algebra can be extended to better understand third-order tensors. Specifically, in [9] and [10] the authors define a multiplication operation which is closed on the set of third-order tensors. This multiplication allows tensor factorizations which are analogs of matrix factorizations such as SVD, QR and eigendecompostions. In addition, [2] defines a free module of matrices (or n × 1 × n tensors) over a commutative ring where the elements are vectors (or 1 × 1 × n tensors) and shows that every linear transformation upon that space can be represented by multiplication by a third-order tensor. Thus, the significant contribution of those papers is the development of a framework which allows new extensions of familiar matrix
Kilmer et al. REF generalize the inner product of two vectors, suggest a notion of the angle between two vectors with elements in R and define a notion of orthogonality for two vectors.
THIRD ORDER TENSORS AS OPERATORS ON MATRICES: A THEORETICAL AND COMPUTATIONAL FRAMEWORK WITH APPLICATIONS IN IMAGING ∗
SIAM J. Matrix Analysis Applications
SIAM J. Matrix Analysis Applications
Computer Science Mathematics