# Frequency-Domain and Spatial-Domain MLMVN-Based Convolutional Neural Networks
## 5 Simulation Results and Discussion
### MLMVN as a CNN in the Frequency Domain  
To evaluate the capability of this approach, we used two topologies: with one and two hidden layers. In both cases, the first hidden layer was considered convolutional. Therefore, the topology was either \(\mathrm{I}\rightarrow\mathrm{H}\mathrm{I}\rightarrow\mathrm{O}\) or \(\mathrm{I}\rightarrow\mathrm{H}\mathrm{I}\rightarrow\mathrm{H}\mathrm{I} \rightarrow\mathrm{O}\), where \(\mathrm{I}\) stands for the neural network's input, \(\mathrm{H}\mathrm{I}\) and \(\mathrm{H}\mathrm{I}\) represent the number of neurons in the first and second hidden layer, respectively, and \(\mathrm{O}\) denotes the number of neurons in the output layer. In both cases, the output layer consisted of 10 neurons, each performing binary classification using the \"One vs. All\" approach. The \"winner\" neuron is determined by the closeness of its weighted sum to the desired output of that neuron, the same rule as was used in CNNMVN. The inputs to the network were the normalized Fourier coefficients of the normalized input images (their range was transformed to \([0,1]\)) obtained using frequency pooling. For the MNIST images, we generated three datasets with frequencies 1-5, 1-6, and 1-7, and for the Fashion MNIST images, we have chosen frequencies 1-7, 1-8, and 1-9. In general, the entire learning process in this approach is equivalent to regular MLMVN, except that the inputs are the Fourier coefficients of the respective frequencies. However, we also incorporated some improvements. The first improvement is the use of error correction for neuron weights using the self-adaptive learning rule (7). The second improvement is that we calculated the global error of the network (5) as the difference between the desired output and the actual weighted sum, which is equivalent to not using the activation function in the output layer. This approach was first introduced in [74] and allows for improved generalization capability in regular MLMVN. Therefore, the global error of the network is given by:  
\[\delta^{*}=D-z, \tag{35}\]  
where \(\delta^{*}\) is the global error, \(D\) is the desired output of the neuron, and \(z\) is the actual weighted sum of the output neuron. For our tests, we also applied the batch learning algorithm technique with a batch size of 20,000 samples. At each step, the neural network learned samples from the batch and then moved on to the next batch until all elements of the training dataset were used. A new epoch then started, and these steps were repeated. Additionally, we employed the soft margins technique with a threshold of \(\pi/12\). The learning process continued for 200 epochs. The experimental results for the MNIST dataset are presented in Table 3, and for the Fashion MNIST dataset in Table 4. They demonstrate the high potential of MLMVN as a CNN in the frequency domain and warrant further examination. Our attention was drawn to the fact that all results are nearly the same across different topologies and the \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Topology** & **Pool Type** & \
\begin{tabular}{c} **Type of Error** \\ **Normalization** \\ \end{tabular} &
\begin{tabular}{c} **Epoch When** \\ **Maximum** \\ **Accuracy Reached** \\ \end{tabular} & **Accuracy** \\ \hline
16C5-h128-010 & None & Custom & 8 & 86.91 \\
16C5-h256-010 & None & Custom & 16 & 86.99 \\
32C5-h128-010 & None & Custom & 13 & 87.7 \\
32C5-h256-010 & None & Custom & 16 & 88.03 \\ \hline
32C5-P2-h256-010 & Max. pool & Custom & 13 & 86.99 \\
32C5-P2-h256-010 & Average pool & Custom & 8 & 84.78 \\ \hline
32C5-h256-010 & None & Default & 20 & 69.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Classification results for CNNMVN for Fashion MNIST dataset. number of frequencies used. It is evident that in most cases, the larger the network, the faster it reaches its peak accuracy, after which further learning becomes redundant.