# Frequency-Domain and Spatial-Domain MLMVN-Based Convolutional Neural Networks
## 1 Introduction  
In the last decade, convolutional neural networks (CNNs) have revolutionized the field of computational intelligence, particularly in the realm of computer vision. Since their introduction in [1], they have achieved remarkable success in various applications, ranging from image classification and object detection to facial recognition and medical image analysis. The idea behind CNNs was introduced in [2, 3] and was inspired by the architecture of the mammalian primary visual cortex. A classical CNN is a feedforward neural network whose architecture can be divided into two parts. The first part consists of the convolutional and pooling layers, while the second part comprises fully connected layers. The second part is essentially a multi-layer perceptron (MLP). The role of convolutional layers is to extract the main features from an input image or signal; pooling layers are used for feature downsampling, and the fully connected layers perform the actual classification based on the features extracted by the convolutional layers. Each convolutional layer consists of neural kernels that perform convolutions of the input data with the weighting convolutional kernels resulting from the learning process. As a result, convolutional layers may extract useful shapes, highlight or smooth edges, etc. This is the main advantage of CNNs over fully connected neural networks. Numerous applications of CNNs in image recognition and other areas have proven their high efficiency. There are many publications devoted to CNNs and their applications, which include refs. [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33]. There are also many developments of CNNs, which include further development of their architecture, learning techniques, and applications. We can mention among others, for example, hierarchicalCNNs [34] and VGG networks (very deep convolutional neural networks with over a dozen convolutional layers) [35], new and interesting applications in mechanical engineering [36]. The CNN introduced in [1] is a network based on real-valued neurons, and its learning rule is based on stochastic gradient descent. However, there is another family of feedforward neural networks: complex-valued neural networks (CVNNs). These networks are based on neurons employing complex numbers as weights. This approach has several advantages [37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48] over its real-valued counterparts. CVNNs typically learn faster, can properly work with complex-valued data, and develop better generalization capabilities. There are also complex-valued and quaternion-valued convolutional neural networks (CV-CNNs and QV-CNNs, respectively). The first CV-CNN was presented in [49] and has been actively developed over the past years. Since then, CV-CNNs and QV-CNNs presented in [50; 51; 52; 53; 54; 55; 56; 57; 58; 59] have demonstrated successful applications in synthetic-aperture radar (SAR) image recognition and filtering, as well as in general-purpose image recognition. There are two types of complex-valued neurons used in CVNNs. The first type is a complex-valued neuron with a complex hyperbolic tangent activation function [37], whose functionality is similar to that of a respective real-valued neuron. The second type is a multi-valued neuron (MVN) [38]. A single MVN can learn non-linearly separable input/output mappings [38], making this neuron more functional compared to others. An MVN may have a discrete or continuous activation function, which projects a weighted sum onto the unit circle and depends on the argument (phase) of the weighted sum [38]. Its learning algorithm is based on the derivative-free error-correction learning rule. The most well-known MVN-based neural network is a multilayer neural network with multi-valued neurons (MLMVN), which is a multilayer fully connected feedforward neural network [60].