# Tensor-Based Algorithms for Image Classification
## 4 Numerical Results  
We apply the tensor-based classification algorithms described in Sections 3.2 and 3.3 to both the MNIST and fashion MNIST data sets, choosing the basis defined in (14) and setting \(\alpha\approx 0.59\). This value was determined empirically for the MNIST data set, but also leads to better classification rates for the fashion MNIST set. Kernel-based MANDy as well as ARR are available in Scikit-TT ([[https://github.com/PGelss/scikit_tt](https://github.com/PGelss/scikit_tt)]([https://github.com/PGelss/scikit_tt](https://github.com/PGelss/scikit_tt))). The numerical experiments were performed on a Linux machine with 128 GB RAM and an Intel Xeon processor with a clock speed of 3 GHz and eight cores. For the first approach, using kernel-based MANDy, we do not apply any regularization techniques. For the ARR approach, we set the TT rank for each solution \(\Xi_{i}\), see Algorithms 2-10, and repeat the scheme five times. Here, we use regularization, i.e., truncated SVDs with a relative threshold of \(10^{-2}\) are applied to the minimization problems given in Algorithm 2 (Lines 8 and 13). The obtained classification rates for the reduced and full MNIST and fashion MNIST data are shown in Figure 5.  
Figure 5: Results for MNIST and fashion MNIST: **(a)** Classification rates for the reduced 14 \(\times\) 14 images. **(b)** Classification rates for full 28x28 images. Reducing the image size by averaging over groups of pixels improves the performance of the algorithm. Similarly to [5; 6], we first apply the classifiers to the reduced data sets, see Figure 5a. Using MANDy, we obtain classification rates of up to 98.75% for the MNIST and 88.82% for the fashion MNIST data set. Using the ARR approach, the classification rates are not monotonically increasing, which may simply be an effect of the alternating optimization scheme. The highest classification rates we obtain are 98.16% for the MNIST data and 87.55% for the fashion MNIST data. We typically obtain a 100% classification rate for the training data (as a consequence of the richness of the feature space). This is not necessarily a desired property as the learned model might not generalize well to new data, but seems to have no detrimental effects for the simple MNIST classification problem. As shown in Figure 5b, kernel-based MANDy can still be applied when considering the full data sets without reducing the image size. Here, we obtain classification rates of up to 97.24% for the MNIST and 88.37% for the fashion MNIST data set. That we obtain lower classification rates for the full images as compared to the reduced ones might be due to the fact that pixel-by-pixel comparisons of images are not expedient. The averaging effect caused by downscaling the images helps to detect coarser features. This is similar to the effect of convolutional kernels and pooling layers. In principle, ARR can also be used for the classification of the full data sets. So far, however, our numerical experiments produced only classification rates significantly lower than those obtained by applying MANDy (95.94% for the MNIST and 82.18% for fashion MNIST data set). This might be due to convergence issues caused by the kernel. The application to higher-order transformed data tensors and potential improvements of ARR will be part of our future research. Figure 5 also shows a comparison with tensorflow. We run the code provided as a classification tutorial (www.tensorflow.org/tutorials/keras/basic_classification) ten times and compute the average classification rate. The input layer of the network comprises 784 nodes (one for each pixel; for the reduced data sets, we thus have only 196 input nodes), followed by two dense layers with 128 and 10 nodes. The layer with 10 nodes is the output layer containing probabilities that a given image belongs to the class represented by the respective neuron.