# Frequency-Domain and Spatial-Domain MLMVN-Based Convolutional Neural Networks
## 3 CNNMVN Learning Algorithm and Error Backpropagation  
### CNNMVN Feedforward Process  
CNNMVN is a spatial-domain convolutional neural network based on multi-valued neurons. Its topology is similar to regular CNNs: it consists of one or more complex-valued convolutional layers, each convolutional layer may be followed by a pooling layer, and the convolutional part is followed by the fully connected part, which is essentially MLMVN.  
This network operates on the same principle as any CNN. Initially, an image to be recognized passes through a convolutional layer, where kernels extract important features and form feature maps. Subsequently, a pooling layer downsamples these feature maps. Additional convolutional or pooling layers may follow, extracting more specific features from the preceding layer's feature maps. Finally, the feature maps from the last convolutional (or pooling) layer are flattened into a vector before proceeding to the fully connected part.  
The complex-valued convolutional layer based on MVN kernels was initially proposed in [62], with a subsequent modification presented in [63]. Interestingly, while the error backpropagation for the convolutional part in CNNMVN is quite similar to that of regular CNNs, it is derived based on considerations similar to those used for MLMVN. This algorithm was outlined in [63] without detailed elaboration, which we aim to comprehensively present here.  
Let us consider CNNMVN with \(G\) convolutional layers, one fully connected layer, and one output layer. Let each convolutional layer consist of \(H_{g}\) kernels, where \(g=\overline{1,G}\). Let the input image \(x\) (or a feature map in the case when the first convolutional layer is not considered) be of size \(a_{g}\times b_{g}\times d_{g}\), where \(a_{g}\) and \(b_{g}\) stand for image (or feature map) rows and columns and \(d_{g}\) stands for its depth, respectively. Then, to perform a convolution properly, each kernel should have the same depth as the input image (or a feature map), and therefore the size of the kernels should be \(r_{g}\times r_{g}\times d_{g}\), where \(r_{g}\) stands for kernel rows and columns and \(d_{g}\) is the kernel's depth. The forward convolution process operates as follows: kernels slide over the image (or feature map), performing element-wise multiplications of the respective image intensities (transformed into the complex number lying on the unit circle according to (1)) with the corresponding kernel weights, and then summing the resulting products. This process can be described as follows:  
\[c_{ij}^{hg}=\sum_{k=1}^{d_{p}}\sum_{u=1}^{r_{p}}\sum_{v=1}^{r_{p}}w_{kuv}^{hg} x_{i+u-1,j+v-1,k^{\prime}},\quad\begin{array}{l}i=1,\ldots,a_{p}-r_{p}+1,\\ j=1,\ldots,b_{p}-r_{p}+1,\end{array} \tag{17}\]  
where \(c_{ij}^{hg}\) is a convolved pixel at indexes \(i,j\) in an image or a feature map produced by the \(h^{th}\) kernel (\(h=\overline{1,H_{g}}\)) of the \(g^{th}\) convolutional layer, and \(w_{fgh}^{kp}\) are the weights of this kernel. After convolutions are performed in all the kernels of the \(g^{th}\) layer, we obtain \(H_{g}\) feature maps and should apply the activation function (4) to them to obtain the final feature maps. These final feature maps then become the inputs for the next layer, which can be either a convolutional layer, a pooling layer, or the fully connected part of CNNMVN if this is the \(G^{th}\) (the last) convolutional layer.