# Frequency-Domain and Spatial-Domain MLMVN-Based Convolutional Neural Networks
## 5 Simulation Results and Discussion
### MLMVN as a CNN in the Frequency Domain
It was obtained by applying the inverse Fourier transform to the Fourier spectrum of an image where only Fourier coefficients corresponding to frequencies 1-5 were preserved. By taking the inverse Fourier transform from any component-wise product of the downsampled Fourier transform (of an image shown in Figure 12d used as a network input) and a weighting vector, we can obtain a respective convolved image. As this image is complex-valued in general, we extract its magnitude to visualize it. Two of such images are presented in Figure 12e,f. Analyzing these convolved images, we may conclude that frequency-domain convolutions resulting from the learning process likely extract structural components from images, breaking them down into respective structures. Each convolutional kernel in the frequency domain resulting from the learning process becomes responsible for extracting structural elements that appear to be useful for recognition due to the network's self-organization based on its learning process. It is also possible to conclude that these convolutions are similar to the  
Figure 11: Accuracy of MLMVN as a frequency-domain CNN for Fashion MNIST with strict batch learning. ones extracting high-level features in traditional CNNs and performed in the second and further convolutional layers (if any). Comparison of the Capabilities of CNNMVN and MLMVN as a Frequency-Domain CNN with Those of Other Networks  
As mentioned above, our goal was to test the \"pure\" image recognition capabilities of both networks used in this paper. Thus, we intentionally did not apply any kind of preprocessing to the images used for learning and testing. Additionally, in all our experiments, we intentionally used the simplest possible network topologies to test the \"pure\" image recognition capabilities of both networks. Several researchers have published works reporting formally better recognition rates for the same two image datasets used here. However, these results were obtained after various kinds of preprocessing were applied to the images and using networks with more complicated topologies (more convolutional layers, more kernels, and more hidden layers in the fully connected part of a respective CNN). For example, the best result for the MNIST dataset (99.87%) was reported in [78], but an incomparably more complicated network was used (an ensemble of three CNNs with 3 \(\times\) 3, 5 \(\times\) 5, and 7 \(\times\) 7 convolutional kernels) in conjunction with sophisticated data preprocessing (data augmentation consisting of rotation and translation). Data augmentation, training data expansion, and elastic distortions [79]  
Figure 12: Comparison of an original image (**a**) with the frequency-domain downsampled image (**d**), and results of spatial-domain convolutions performed by CNNMVN (**b**,**c**) compared with those of frequency-domain convolutions performed by MLMVN (**e**,**f**). Figure description: (**a**) Original image; (**b**) After convolution with the 7 th kernel; (**c**) After convolution with the 26 th kernel; (**d**) Original image after frequency-domain downsampling (frequencies 1–5 were used) and inverse Fourier transform; (**e**) After convolution with the 221 st kernel; (**f**) After convolution with the 366 th kernel. are typical preprocessing methods that improve results. We intentionally did not use these methods as our goal was to test the network's generalization and recognition capabilities as they are. To obtain a higher classification rate using a regular real-valued CNN without data preprocessing, a larger network with some more sophisticated learning techniques (like dropout and various kinds of optimization) should be used. This is shown, for example, in [80], where to reach 98.96% classification accuracy on the MNIST dataset, a CNN with two convolutional layers containing 64 kernels each along with modified learning algorithm has been used. We also want to mention the result reported for the MNIST dataset in [81] (98.4% recognition accuracy).