# Fitness Landscape Analysis of Product Unit Neural Networks
## 6 Empirical Procedure
### Fitness Landscape Measures and Sampling Parameters  
Since the weights of an NN can take on any real number, the boundaries of the fitness landscape are infinite. However, bounds have to be provided within which the FLA metrics can be applied. This study used two sets of bounds for all problems. The first set of bounds, i.e., \([-1,1]\), focuses on areas where optimization algorithms are most likely to explore [21]. Additionally, \([-1,1]\) bounds limit the hyper-volume of the landscape, which facilitates better coverage when sampling and, subsequently, more accurate estimates of the fitness landscape measures. Larger bounds were also used. It is important to note that the bounds of the landscape represent the order of the exponential terms in the PU. With larger bounds, the PUNN performs significantly higher-order transformations of the input signals. To investigate the non-linear nature of the PUNN, \([-3,3]\) was used for all classification problems. For the regression problems, \([-3,3]\) was used for functions \(f_{1},f_{2}\) and \(f_{3}\), \([-7,7]\) was used for \(f_{4}\), and \([-5,5]\) was used for \(f_{5}\).  
For adequate coverage of the search space, the number of independent PRWs and MRWs was set to the dimensionality of the loss surface. PRWs and MRWs started each walk at one of the corners of the landscape. To perform a walk from every corner would require performing \(2^{N}\) walks, which becomes computationally infeasible for high dimensions. Therefore, this study performed a walk at every \((\frac{2^{N}}{N})^{th}\) corner of the landscape as recommended by [PERSON] [20]. Each PRW performs 1000 steps and each MRW performs 2000 steps at step sizes of 1% of the domain. Uniform sampling makes use of \(500\times N\) independent samples for the \(FDC_{s}\) measure and 2000 samples for the \(DM\). For dispersion, the 10% best solutions were used as recommended by [20]. The values used for the threshold \(\epsilon\) for neutrality measures \(M_{1}\) and \(M_{2}\) are provided in Table 2.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset** & **Inputs** & **PUNN Hidden** & **SUNN Hidden** & **Outputs** & **PUNN Dimensionality** & **SUNN Dimensionality** \\ \hline
**XOR** & 2 & 1 & 2 & 1 & 4 & 9 \\
**Iris** & 4 & 2 & 4 & 3 & 17 & 35 \\
**Wine** & 13 & 2 & 10 & 3 & 35 & 173 \\
**Diabetes** & 8 & 1 & 8 & 1 & 10 & 81 \\ \(f\)**1** & 1 & 1 & 2 & 1 & 3 & 7 \\ \(f\)**2** & 1 & 2 & 3 & 1 & 5 & 10 \\ \(f\)**3** & 2 & 5 & 5 & 1 & 16 & 21 \\ \(f\)**4** & 2 & 3 & 8 & 1 & 10 & 33 \\ \(f\)**5** & 1 & 2 & 3 & 1 & 5 & 10 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Datasets and optimal network architecture.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Datasets:** & **XOR** & **Iris** & **Wine** & **Diabetes** & \(f_{1}\) & \(f_{2}\) & \(f_{3}\) & \(f_{4}\) & \(f_{5}\) \\ \hline
**Small bounds:** & 0.02 & 2 & 2 & 2 & 0.02 & 0.02 & 0.2 & 2 & 0.2 \\ \hline
**Large bounds:** & 2 & 20 & 2000 & 20 & 0.2 & 2 & 2 & 2000 & 2 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Threshold \(\epsilon\) values used for neutrality measures.