# Quadratic Multilinear Discriminant Analysis for Tensorial Data Classification
## 3 Algorithm
\begin{bmatrix}\frac{\tilde{\sigma}}{\sigma_{1}}&&\\ &&\frac{\tilde{\sigma}}{\sigma_{n}}\end{bmatrix} \]  
_then \(AX\) is a critical point for the norm function, and therefore an absolute minimum point._  
Suppose that the means of the rows of \(X=[x_{1}\;x_{2}\;\cdots\;x_{m}]\) are zero. Thus, all features of the vector data have mean \(0\). Multiplying with \(A\) has the effect that all the features (rows) are normalized to have the same standard deviation. If \(X\) has a zero row, we can first \"regularize\" \(X\) by replacing it with \([X\mid e\mathbf{1}^{t}]\) for some chosen \(\epsilon>0\), where \(\mathbf{1}=(1,\ldots,1)\) is a row vector with \(n\) entries equal to \(1\). Instead of vector data, we will now consider tensor data. The techniques using Kempf-Ness theory are valid for tensors of any order, but we will work with 3-way tensors. Suppose that \(\mathcal{X}_{1},\mathcal{X}_{2},\ldots,\mathcal{X}_{m}\in\mathbb{R}^{n_{1}x_{ 2}\times n_{3}}\) are tensors. We could normalize the data with an arbitrary change of coordinates in \(\mathrm{SL}_{n}\), where \(n=n_{1}n_{2}n_{3}\), but this could change the tensor structure of the tensor data. For example, the tensor rank is not preserved under arbitrary coordinate changes in \(\mathrm{SL}_{n}\). Instead, we will consider the action of the a group \(G=H_{1}\times H_{2}\times H_{3}\) on \(\mathbb{R}^{n_{1}x_{2}\times n_{3}}\) where for each \(i\), \(H_{i}\) is equal to \(\mathrm{SL}_{n_{i}}\) or \(\mathrm{T}_{n_{i}}\). An element \((A,B,C)\in G\) in the group acts on a tensor \(\mathcal{T}\in\mathbb{R}^{n_{1}x_{2}\times n_{3}}\) by multiplying \(A\), \(B\) and \(C\) with \(\mathcal{T}\) in the first, second and third mode, respectively:  
\[(A,B,C)\cdot\mathcal{T}=A\times_{1}B\times_{2}C\times_{3}\mathcal{T}.\]  
for \(G\) and \(\mathcal{T}\) fixed, the \(G\)-orbit \(G\cdot\mathcal{T}\) of \(\mathcal{T}\) is the set of tensors of the form \((A,B,C)\cdot\mathcal{T}\) as \(A\), \(B\) and \(C\) vary. Consider the binary classification problem with training data given by three-way tensors \(\{\mathcal{X}_{i}\}_{i=1}^{m_{1}}\) of size \(n_{1}\times n_{2}\times n_{3}\) in class \(1\), and \(\{\mathcal{Y}_{i}\}_{i=1}^{m_{2}}\) of the same size in class \(2\). Given a new test tensor \(\mathcal{Z}\) from one of the two classes above, our goal is to identify the class to which it belongs. Though we have restricted our discussion to a binary classification problem for simplicity, our algorithm extends naturally to cases with more than two classes. Working with Theorem 1, it can be hard in general to check the existence of a critical point, or to find one if it exists. The optimization problem we wish to solve is to find matrices \((A_{1},B_{1},C_{1})\in G\) such that  
\[\sum_{i=1}^{N_{1}}|(A_{1},B_{1},C_{1})\cdot(\mathcal{X}_{i}-\bar{\mathcal{X}}) |^{2}\]  
is minimal for the first class (where \(\bar{\mathcal{X}}\) is the mean of the tensors in the first class), and similarly matrices \((A_{2},B_{2},C_{2})\in G\) for the second class. This optimization problem can be formulated in terms of Kempf-Ness theory. Denote by \(\mathcal{T}\) the four-way tensor obtained by concatenating along the fourth mode all (centered) tensors \(\mathcal{X}_{i}\), and denote by \(K=H_{1}\times H_{2}\times H_{3}\times I_{N_{1}}\) the group acting as \(H_{1}\times H_{2}\times H_{3}\) on the first 3 modes, and trivially (via the identity matrix) on the fourth mode. This optimization problem can then be equivalently stated as finding the minimum of the norm over the orbit \(K\cdot\mathcal{T}\). By Theorem 1 (or, more precisely, its version for fourth order tensors) it then follows that if there is a critical point, our problem has a unique minimum. In general, we will not be able to determine whether a critical point exists, but experimentally it will turn out to be the case if we adopt a suitable regularization technique.