# Tensor-Based Algorithms for Image Classification
## 1 Introduction  
Tensor-based methods have become a powerful tool for scientific computing over the last years. In addition to many application areas, such as quantum mechanics and computational dynamics, where low-rank tensor approximations have been successfully applied, using tensor networks for supervised learning has gained a lot of attention recently. In particular, the canonical format and the tensor train format have been considered for quantum machine learning (There are different research directions in the field of quantum machine learning, here we understand it as using quantum computing capabilities for machine learning problems.) problems, see, e.g., [1; 2; 3]. A tensor-based algorithm for image classification using sweeping techniques inspired by the density matrix renormalization group (DMRG) [4] was proposed in [5; 6] and further discussed in [7; 8]. Interestingly, also researchers at Google are currently developing a tensor-based machine learning framework called \"TensorNetwork ([[http://github.com/google/TensorNetwork](http://github.com/google/TensorNetwork)]([http://github.com/google/TensorNetwork](http://github.com/google/TensorNetwork)))\" [9; 10]. The goal is to expedite the adoption of such methods by the machine learning community. Our goal is to show that recently developed methods for recovering the governing equations of dynamical systems can be generalized in such a way that they can also be used for supervised learning tasks, e.g., classification problems. To learn the governing equations from simulation or measurement data, regression methods such as sparse identification of nonlinear dynamics (SINDy) [11; 12] and its tensor-based reformulation multidimensional approximation of nonlinear dynamics (MANDY) [13] can be applied. The main challenge is often to choose the right function space from which the system representation is learned. Although SINDy and MANDy essentially select functions from a potentially large set of basis functions by applying regularized regression methods, other approaches allow nestedfunctions and typically result in nonlinear optimization problems, which are then frequently solved using (stochastic) gradient descent. By constructing a basis comprising tensor products of simple functions (e.g., functions depending only on one variable), extremely high-dimensional feature spaces can be generated. In this work, we explain how to compute the pseudoinverse required for solving the minimization problem directly in the tensor train (TT) format, i.e., we replace the iterative approach from [5; 6] by a direct computation of the least-squares solution and point out similarities with the aforementioned system identification methods. The reformulated algorithm can be regarded as a _kernelized_ variant of MANDy, where the kernel is based on tensor products. This is also related to quantum machine learning ideas: As pointed out in [14], the basic idea of quantum computing is similar to kernel methods in that computations are performed implicitly in otherwise intractably large Hilbert spaces. Although kernel methods were popular in the 1990s, the focus of the machine learning community has shifted to deep neural networks in recent years [14]. We will show that, for simple image classification tasks, kernels based on tensor products are competitive with neural networks. In addition to the kernel-based approach, we propose another DMRG-inspired method for the construction of TT decompositions of weight matrices containing the coefficients for the selected basis functions. Instead of computing pseudoinverses, a core-wise ridge regression [15] is applied to solve the minimization problem. Although the approach introduced in [5; 6] only involves tensor contractions corresponding to single images of the training data set, we use TT representations of transformed data tensors, see [13; 16], to include the entire training data set at once for constructing low-dimensional systems of linear equations.