# Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification
## 3 Algorithm  
Before we apply Kempf-Ness theory for tensor classification, let us discuss an easier situation where we are not using any tensor structure. We consider a sequence of vectors \(x_{1},\ldots,x_{m}\in\mathbb{R}^{n}\) and the action of a subgroup \(G\subseteq\mathrm{GL}_{n}\) by multiplication. We combine the vectors into a matrix \(X=[x_{1}\ x_{2}\ \cdots\ x_{m}]\). We will see that in the case of centered vector data, simultaneously minimizing the norm of all vectors with respect to the \(\mathrm{SL}_{n}\)-action coincides with considering the Mahalanobis distance (up to a constant). The proof of the next Proposition can be found in Appendix A. Proposition 1 (Critical points for \(\mathrm{SL}\)-action): _Let \(X\in\mathbb{R}^{n\times m}\) of rank \(n\), with \(n<m\). Suppose that \(X=USV^{t}\) is a (truncated) singular value decomposition with \(U,V\in\mathrm{SO}_{n}\) and \(S\) a diagonal matrix whose diagonal entries are the singular values \(\sigma_{1},\ldots,\sigma_{n}\). Let \(\bar{\sigma}=(\sigma_{1}\sigma_{2}\cdots\sigma_{n})^{1/n}\) be the geometric mean. If_  
\[D:= \begin{bmatrix}\frac{\bar{\sigma}}{\sigma_{1}}&\ddots&\\ &&\frac{\bar{\sigma}}{\sigma_{n}}\end{bmatrix} =\overline{\sigma}S^{-1}\]  
_and \(A:=DU^{t}\), then \(AX\) is a critical point for the norm function, and therefore an absolute minimum point._  
If the mean of the vectors \(x_{1},x_{2},\ldots,x_{m}\) is zero and \(X=[x_{1}\ x_{2}\ \cdots\ x_{m}]\in\mathbb{R}^{n\times m}\), then we have \(XX^{t}=m\Sigma\) where \(\Sigma\) is the covariance matrix. Therefore, we have  
\[A^{t}A=UD^{2}U^{t}=\overline{\sigma}^{2}US^{-2}U^{t}=\overline{\sigma}^{2}( XX^{t})^{-1}=\frac{\overline{\sigma}^{2}}{m}\Sigma^{-1}. \tag{2}\]The Mahalanobis distance between two vectors \(x\) and \(y\) is  
\[\sqrt{(x-y)^{t}\Sigma^{-1}(x-y)}=\tfrac{\sqrt{m}}{\varphi}\big{|}A(x-y)\big{|}.\]  
so that (up to a scalar) \(A\) gives us the Mahalanobis distance \(\Sigma^{-1}\). Multiplying with \(A\) can be thought of as a normalization, so that the covariance matrix of \(Ax_{1},Ax_{2},\ldots,Ax_{m}\) is the identity, up to a scalar. If \(X\) has a rank less than \(n\), we can first \"regularize\" \(X\) by replacing it with \([X\mid eI_{n}]\) for some chosen \(\epsilon>0\) to ensure that the rank is \(n\), and then apply the previous proposition. One can think of \(\epsilon\) as a regularization parameter that can be used even if the rank of the matrix is already \(n\). Experimental results (as described in Section 4) suggest, on the other hand, that the results are often not affected by the choice of \(\epsilon\). QDA is an algorithm for binary classification of vector data that uses the Mahalanobis distance as follows. Suppose that there are two classes. In the training data, we have vectors \(x_{1},x_{2},\ldots,x_{m_{1}}\in\mathbb{R}^{n}\) that belong to the first class and training data \(y_{1},y_{2},\ldots,y_{m_{2}}\in\mathbb{R}^{n}\) that belong to the second class. Let \(\overline{x}\) and \(\overline{y}\) be the means of the \(x\)'s and the \(y\)'s, respectively. Let \(\Sigma_{1}\) and \(\Sigma_{2}\) be the covariance matrices of the \(x\)'s and \(y\)'s. To classify a given vector \(z\), we compare the \(\Sigma_{1}\)-Mahalanobis distance of \(z\) to \(\overline{x}\) with the \(\Sigma_{2}\)-Mahalanobis distance of \(z\) to \(\overline{y}\). Our goal is to generalize this approach to tensor data, and replace the Mahalanobis distance with a distance that respects the tensor structure. Instead of the action of \(\mathrm{SL}_{n}\), we can also consider the action of \(\mathrm{T}_{n}\), the group of diagonal matrices with positive diagonal entries and determinant \(1\). We have the following result, the proof of which can be found in Appendix A. **Proposition 2** (Critical points for \(\mathrm{T}\)-action). : _Suppose that \(X\in\mathbb{R}^{n\times m}\) has no zero rows. Let \(\sigma_{1},\ldots,\sigma_{n}\) be the Euclidean lengths of the rows of \(X\), and let \(\tilde{\sigma}=(\sigma_{1}\sigma_{2}\cdots\sigma_{n})^{1/n}\) be the geometric mean. If_  
\[A\coloneqq