# Frequency-Domain and Spatial-Domain MLMVN-Based Convolutional Neural Networks
## 2 MVN and MLMVN Fundamentals  
### Mvn  
MLMVN is a fully connected feedforward neural network based on multi-valued neurons. It is essential that the learning algorithm for CNNMVN is based on a generalization of the learning algorithm for MLMVN. Let us briefly recall some important fundamentals about MVN and MLMVN.  
MVN is a complex-valued neuron. It has \(p\) inputs, \(p+1\) weights, and one output, like a regular neuron. Its inputs can be arbitrary complex numbers while its outputs are located on the unit circle. If the inputs are real-valued, we should project them onto the unit circle as follows:  
\[z_{j}=e^{i\varphi_{j}},\ \ \varphi_{j}=\frac{x_{j}-\min\{X\}}{\max\{X\}-\min\{X \}}\lambda, \tag{1}\]  
where \(x_{j}\) stands for a real-valued datapoint, \(z_{j}\) is a datapoint projected onto a unit circle, and \(\varphi_{j}\) is the argument (phase) of the complex-valued datapoint \(z_{j}\); \(\min\{X\}\) and \(\max\{X\}\) are respectively the minimum and maximum values over a real-valued dataset; and \(\lambda\) determines the pre-defined part of the unit circle on which the data are projected (a standard value is \(\lambda=3\pi/\ 2\)), which means that the projected data items have arguments from \(0\) to \(\lambda\).  
Let \(\big{(}x_{1},\ldots,x_{p}\big{)}\) be the neuron's inputs, and \(\big{(}w_{0},w_{1},\ldots,w_{p}\big{)}\) be the neuron's weights. Thus, the weighted sum of inputs is as follows  
\[z=w_{0}+w_{1}x_{1}+\cdots w_{p}x_{p}. \tag{2}\]  
The actual output of a neuron results from its activation function applied to its weighted sum. There are two types of MVN activation functions commonly used: discrete and continuous. The discrete \(k\)-valued activation function \(P(z)\) divides the unit circle into \(k\) sectors and maps the weighted sum \(z\) onto the \(k^{th}\) root of unity \(\Big{\{}1,\epsilon_{k},\epsilon_{k}^{2},\ldots,\epsilon_{k}^{k-1}\Big{\}}\), which is located on a lower border of the corresponding sector of the complex plane. Thus, this activation function depends on the argument (phase) of the weighted sum:  
\[P(z)=\epsilon_{k}^{j}=e^{i2\pi j/k},\ \ 2\pi j/k\leq\arg(z)<2\pi(j+1)/k. \tag{3}\]The continuous activation function projects the weighted sum \(z\) onto the unit circle by normalizing it and preserving its actual phase:  
\[P(z)=e^{i\cdot arg(z)}=\frac{z}{|z|}. \tag{4}\]  
The error \(\delta\) of MVN is determined as the following difference,  
\[\delta=D-Y, \tag{5}\]  
between its desired output \(D\) and its actual output \(Y\). To adjust the weights, the error-correction learning rule for the neuron is as follows:  
\[\widetilde{W}=W+\frac{C}{p+1}\delta X^{*}. \tag{6}\]  
The rule with a self-adaptive learning rate is  
\[\widetilde{W}=W+\frac{C}{(p+1)|z|}\delta X^{*}, \tag{7}\]  
where \(W\) and \(\widetilde{W}\) are the weighting vectors before and after correction, \"+\" denotes component-wise addition, \(C\) is the learning rate (in general it should be a complex number, but in all practical applications it is usually equal to 1), \(X^{*}\) is the vector of reciprocal inputs, \(p\) is the number of inputs, and \(|z|\) stands for the absolute value of the weighted sum before adjustment (this is a self-adaptive part of the learning rate, and it is significant in MLMVN learning [38; 60]).