# Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification
## 1 Introduction  
Data often has a higher-order structure, and leveraging this structure can significantly improve the results of the problem at hand. However, many machine learning methods do not naturally extend to tensors in a way that account for this tensorial structure. In the case of classification problems, for example, there has been tremendous interest in extending linear discriminant analysis (LDA) to the tensorial setting. LDA is a classical and versatile method for classification, but is typically not well suited for data in matrix or in tensor forms (higher-order matrices). These types of data, with a higher order structure, are very common nowadays and there has been a lot of interest in developing machine learning methods that can leverage this structure. Natural examples of data in matrix form are black-and-white images and multi-lead electrocardiogram (ECG) readings. Similarly, color images, videos, and data in matrix forms that vary with time are examples of third order tensors. Classifying matrix data with LDA (or other vector-based classical machine learning algorithms) requires the data to be vectorized first; this often leads to large vectors, but--perhaps more importantly--also causes the loss of important information such asspatial locality. Furthermore, vectorization ignores the multilinear structure of the data. For example, a matrix of rank one depends on fewer parameters than a full-rank matrix. On top of that, matrices from different classes might have different ranks, so using the multilinear structure might help in the classification task. These considerations apply even more strongly to higher-order data, and tensor techniques have been successfully applied in a variety of classification settings such as images (see [1]), electroencephalogram (EEG) data (see [2; 3]), and ECG signals (see [4; 5]); for a recent survey of applications to clinical informatics, see [6]. On the other hand, tensors of order 3 or more enjoy many good properties (e.g., unique decompositions), so that sometimes it is in fact advantageous to reshape data given in form of vectors or matrices into data of higher order [7]. For these reasons, there has been intensive work in recent years on extending LDA to multilinear discriminant analysis (MDA) to leverage the multilinear structure of the data. Using the Fisher criterion for LDA, given data points \(x_{1},\ldots,x_{m}\in\mathbb{R}^{n}\) belonging to two classes \(C_{1},C_{2}\) of size \(m_{1},m_{2}\), one seeks the optimal projection to a hyperplane that maximizes distances between different classes (measured by the between-class scatter matrix) and minimizes distances within the same class (measured by the within-class scatter matrix). If \(\mu_{1},\mu_{2}\) are the means for each class and \(\mu\) is the overall mean, one can define the scatter matrix for class \(i\) as the covariance matrix for that class,  
\[S_{i}=\frac{1}{m_{i}}\sum_{j\in C_{i}}(x_{j}-\mu_{i})(x_{j}-\mu_{i})^{t},\]  
the within class scatter matrix as \(S_{W}=S_{1}+S_{2}\), and the between class scatter matrix as  
\[S_{B}=m_{1}(\mu_{1}-\mu)(\mu_{1}-\mu)^{t}+m_{2}(\mu_{2}-\mu)(\mu_{2}-\mu)^{t}.\]  
The hyperplane \(f(x)=w^{t}x\) to project onto is then obtained by maximizing  
\[J(w)=\frac{w^{t}S_{B}w}{w^{t}S_{W}w}.\]  
In an attempt to extend [PERSON]'s approach, most MDA methods seek to project tensor data to a lower dimensional vector or tensor space by similarly maximizing distances between classes while simultaneously minimizing distances within the same class. Early work [8; 9; 10] considered the case of matrix data. After that, many methods have been proposed for tensorial data, usually introducing between-class and within-class scatter matrices for each mode. Different methods consider various functions to optimize (like the scatter-ratio criterion or the scatter-difference criterion from [PERSON] discriminant analysis), and propose different algorithms to optimize them.