# Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification
## 2 Notation and Background Material  
A tensor is a multi-way extension of a matrix whose entries depend on 3 or more indices. For example, entries of a 3-way tensor (or tensor with 3 modes) \(\mathcal{X}\in\mathbb{R}^{n_{1}\times n_{2}\times n_{3}}\) depend on 3 indices, \(x_{ijk}\), where \(i=1,\ldots,n_{1}\), \(j=1,\ldots,n_{2}\), and \(k=1,\ldots,n_{3}\). We will limit our exposition to the case of three-way tensors, but everything extends in the obvious way to four or more modes (as well as to matrices, with only two modes). We refer the reader to [23; 24] for an extensive overview of tensors and their applications. Working with tensors, it is often convenient to break them up into vectors and matrices. Given a tensor \(\mathcal{X}\), fixing two of the indices, we obtain fibers (which are vectors) of the tensor in a given mode; fixing one of the indices, we obtain slices (which are matrices) of the tensor in a given mode. One can flatten (or matricize) a three-way tensor in three different ways, by juxtaposing slices in the chosen mode; \(X_{(n)}\) will denote the mode-\(n\) flattening of \(\mathcal{X}\). For example, the mode-1 flattening of \(\mathcal{X}\) is a matrix of size \(n_{1}\times n_{2}n_{3}\), which can be thought of as arranging mode-1 fibers as columns of an \(n_{1}\times n_{2}n_{3}\) matrix. We will be interested in two types of products. The \(n_{i}\)-mode matrix-tensor product is a way of multiplying an \(m\times n_{i}\) matrix \(A\) by a tensor \(\mathcal{X}\in\mathbb{R}^{n_{1}\times n_{2}\times n_{3}}\), and consists of multiplying each mode-\(i\) fiber of \(\mathcal{X}\) by \(A\). Equivalently, this amounts to multiplying \(A\) by the mode-\(i\) flattening \(X_{(i)}\). This product is denoted \(A\times_{i}\mathcal{X}\), and its result is another tensor, although of different size: for example, \(A\times_{1}\mathcal{X}\) is a tensor in \(\mathbb{R}^{m\times n_{2}\times n_{3}}\). Another product we are interested in is the outer product of vectors: if \(u,v,w\) are vectors of lengths \(n_{1},n_{2},n_{3}\), their outer product (or tensor product) \(u\circ v\circ w\) is a 3-way tensor \(\mathcal{X}\in\mathbb{R}^{n_{1}\times n_{2}\times n_{3}}\) with entries defined as \(x_{ijk}=u_{i}v_{j}w_{k}\). A tensor of the form \(u\circ v\circ w\) is said to have rank 1. A fundamental tool in studying matrices is the singular value decomposition (SVD). This decomposition does not generalize to tensors completely (we cannot retain both a diagonal core and orthogonal side matrices) but it generalizes in two main ways: the canonical polyadic (CP) decomposition and the Tucker decomposition (as well as many variants of these). A CP decomposition of a tensor \(\mathcal{X}\) is a generalization of SVD that keeps a diagonal core, and amounts to writing the tensor as a minimum-length sum of tensors of rank 1:  
\[\mathcal{X}=\sum_{i=1}^{r}\sigma_{i}a_{i}\circ b_{i}\circ c_{i}=[\sigma_{1}, \ldots,\sigma_{r};A,B,C]. \tag{1}\]  
Such a minimum \(r\) is called the rank of the tensor (which is a generalization of the rank of a matrix) and it is unfortunately NP-hard to compute [25; 26; 27]. This decomposition is often unique (up to permuting the indices and rescaling the vectors), and therefore the vectors \(a_{i},b_{i},c_{i}\) contain useful features to further study the tensor \(\mathcal{X}\), even though in practice, one only computes an approximation of this decomposition. The matrices \(A=[a_{1},\ldots,a_{r}]\), \(B=[b_{1},\ldots,b_{r}]\), and \(C=[c_{1},\ldots,c_{r}]\) are often called factor matrices of \(\mathcal{X}\). Another generalization of the SVD of a matrix is the higher-order singular value decomposition (HOSVD), which preserves the orthogonality of the factor matrices but does not have a diagonal core:  
\[\mathcal{X}=\sum_{i,j,k}g_{ijk}a_{i}\circ b_{j}\circ c_{k}=A\times_{1}B\times_ {2}C\times_{3}\mathcal{G}\]with \(A,B,C\) orthogonal.