# Frequency-Domain and Spatial-Domain MLMVN-Based Convolutional Neural Networks
## 1 Introduction
Since MLMVN is based on more functional neurons, it often performs better, learns faster, and develops better generalization capabilities than a classical MLP with sigmoidal neurons [61; 38; 60]. Another advantage is its derivative-free error backpropagation algorithm, based on the generalization of the error-correction learning rule, which does not suffer from the famous local minima problem. This algorithm is based on the error-sharing principle, according to which the global error of the network is shared among all the neurons since all of them contribute to this error by their local errors. These advantages of MLMVN inspired us to build a complex-valued convolutional neural network based on MVN (CNNMVN). It was introduced in [62], and its modified learning algorithm was presented in [63]. CNNMVN is a spatial-domain convolutional neural network, and its topology is generally quite similar to that of regular CNNs. In this paper, we provide a detailed mathematical derivation of the error backpropagation learning algorithm for CNNMVN. It will be shown that CNNMVN is a further generalization of MLMVN, and its backward convolution is similar to the one in real-valued CNNs. We discuss the custom error normalization for CNNMVN that improves classification accuracy. Since this network is complex-valued, we also explore how it processes real-valued images and what features are extracted as a result of its learning process. Another group of convolutional neural networks developed in recent years consists of those that use the frequency domain. For example, in [64], a Fourier convolutional neural network where the convolution operation is performed in the frequency domain was presented. In [65], it was suggested to use the frequency domain (in the discrete cosine transform basis) to represent convolutions and reduce feature map redundancy. Various frequency-domain approaches to CNN design were also discussed and developed, for example, in [66; 67; 68; 69; 70; 71; 72]. A common approach in all these publications is the use of real-valued networks to process the frequency domain information. In this paper, we consider in detail an MLMVN-based convolutional neural network in the frequency domain. We introduced it in [73] and further develop it here, also examining its performance. According to the Convolution Theorem, the convolution of two signals in the spatial domain is equivalent to the element-wise multiplication of the respective frequency-domain representations (Fourier transforms) of these signals. Therefore, if the input to the network is an image or signal represented in the frequency domain, we can interpret the first hidden layer of MLMVN as a frequency-domain convolutional layer. Its neurons perform the convolution operation in the frequency domain, and in this regard, the neurons of this layer are employed as convolutional kernels. We also present here a detailed analysis of the performance of the spatial- and frequency-domain MVN-based convolutional neural networks and compare the feature maps obtained by these networks as a result of their learning. Our simulation results, presented here, have shown that these networks have high potential in image recognition and that both MLMVN in the frequency domain and CNNMVN in the spatial domain show results that are comparable with the best results for regular CNNs. This should help in understanding how more sophisticated networks based on the same approach, but suitable for solving more challenging problems, should be designed. The structure of the paper is as follows. In Section 2, we recall some fundamentals of MVN and MLMVN. In Section 3, we consider in detail the error backpropagation and error adjustment process for CNNMVN. Here, we discuss the main ideas and derivation for error sharing. We also consider the pooling layers. In Section 4, we focus on MLMVN as a frequency-domain CNN and the frequency pooling operation. In Section 5, we analyze the performance of both approaches and consider custom normalization for CNNMVN.