# Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification
## 1 Introduction
Discriminant analysis with tensor representation (DATER) [11], generalized eigenvalue discriminant analysis with tensor representation (DATEReig) [12] and constrained multilinear discriminant analysis (CMDA) [1] seek to maximize the scatter-ratio criterion iteratively by alternating through each mode; direct general tensor discriminant analysis (DGTDA) [1] seeks to maximize the scatter difference directly. The methods proposed in [3], such as manifold Tucker discriminant analysis (ManTDA) and manifold PARAFAC discriminant analysis (ManPDA), seek to optimize simultaneously, in all modes, the projection matrices over a product of Stiefel manifolds with respect to the scatter-ratio criterion or its simplified trace ratio objective. However, the optimization problems considered so far are ill-behaved in general due to the existence of local optima. Because of this, several iterations with different random initializations are sometimes necessary to try to avoid such local optima. However, this adds to the computational cost of the method, while still having no guarantee the computed solution is a global optimum. Furthermore, the performance of these methods drastically depends on the choice of the dimensions of the space to project onto. For example, a tensor of size \(10\times 10\times 10\) can be projected onto a tensor of size \(p\times q\times r\) with \(1\leq p,q,r\leq 10\). This amounts to, in principle, 1000 possibilities. In practice, one usually chooses all dimensionsto be equal, but if the size of tensor varies greatly from one mode to the other, this option is not viable. Therefore these methods include the additional cost of determining the best dimensions to project onto. We will see that our proposed method does not share these issues, since instead of projecting onto a lower dimensional subspace we look for a change of coordinates in the given space. In this paper, we take a radically different approach: Kempf-Ness multilinear discriminant analysis (KNMDA). Starting from the interpretation of classic LDA as a nearest Mahalanobis distance classifier, we reformulate LDA by means of invariant theory, which allows us to extend it naturally to structured data in the form of a well-behaved optimization problem. In fact, our framework allows us to also reformulate quadratic discriminant analysis (QDA) by means of invariant theory, and to extend QDA to tensorial data for the first time. Invariant theory, originating in 19 th century and modernized by [PERSON], can be applied to many areas of computer science, such as coding theory [13] and computer vision [14; 15]. Some newer applications more relevant to this study are those related to scaling algorithms [16], matrix and tensor completion [17], and maximum likelihood estimation [18]. In invariant theory, one studies the linear action of groups on vector spaces and polynomials that are invariant under these group symmetries. A major tool of invariant theory is the Kempf-Ness theorem [19]. Roughly speaking, the theorem states that if an orbit has a point that is closest to the origin, then this point is essentially unique. This translates to a certain optimization problem having a unique optimal solution. This result of Kempf-Ness theory provides a natural extension of LDA to the higher order setting. LDA is related to the Mahalanobis distance: it can be interpreted as a \(k\)-means algorithm, where we classify a data point based on the smallest distance from the mean of the training data of each class, and the distance is given by the Mahalanobis one. In turn, the Mahalanobis distance of a point \(x\) from the mean of the training data \(x\) can be thought of as Euclidean distance after a suitable change of coordinates, namely \(\Sigma^{-1/2}(x-\bar{x})\), where \(\Sigma\) is the sample covariance matrix across the entire training dataset. Therefore, we can interpret LDA as a \(k\)-means algorithm in a suitable coordinate system.