## 4 Benchmarking
### Discussion
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline
**Problem** & **\#** &
\begin{tabular}{c} **Best** \\ **Known** \\ \end{tabular} & **BestDif** & **AvgDiff** & **Time** & **BestDif** & **AvgDiff** & **Time** & **Dif** & **Time** \\ \hline sg3 dl052000 & 125 & 112 & 0 & 1.2 & 0.042 & 0 & 1.8 & 0.039 & 0.058 & *** \\ sg3 dl054000 & 125 & 114 & 0 & 1.333 & 0.043 & 0 & 2.133 & 0.039 & 0.158 & *** \\ sg3 dl056000 & 125 & 110 & 0 & 1.333 & 0.041 & 0 & 1.467 & 0.038 & 0.803 & *** \\ sg3 dl058000 & 125 & 108 & 0 & 1.333 & 0.043 & 0 & 1.6 & 0.039 & 0.312 & *** \\ sg3 dl051000 & 125 & 112 & 0 & 3.267 & 0.042 & 0 & 2.4 & 0.039 & 0.030 & *** \\ \hline sg3 dl02000 & 1000 & 900 & 20 & 28.867 & 404.241 & 18 & 30.200 & 350.943 & 0.237 & *** \\ sg3 dl04000 & 1000 & 896 & 18 & 28.667 & 431.393 & 20 & 28.733 & 351.443 & 0.928 & *** \\ sg3 dl106000 & 1000 & 886 & 24 & 31.267 & 359.999 & 28 & 34.467 & 322.743 & 0.004 & *** \\ sg3 dl108000 & 1000 & 880 & 16 & 26,867 & 380.613 & 20 & 28.600 & 311.638 & 0.058 & *** \\ sg3 dl1010000 & 1000 & 890 & 18 & 28.800 & 390.797 & 20 & 29.733 & 354.691 & 0.397 & *** \\ \hline \hline \end{tabular}  
*: Mann–Whitney test, the difference is significant if \(p\)-value \(<\) \(\alpha\), (\(\alpha=0.05\)). **: average computation time (second). ***: \(<\)0.001.  
\end{table} 
Table 4: Results for [PERSON] et al. (2017) problems. All of the tests involving standard QUBO and standard max-cut problems show that the B-VNS and VNS algorithms are good. The VNS and B-VNS algorithms achieved most of the best-known values or were very close to them. It should be noted that the referenced best-known values were obtained using specific modified algorithms or by setting larger parameter values. The tests show that the B-VNS algorithm performed best in all of the max-cut problems, regardless of problem size, and it was shown to be best suited to QUBO problems, with sizes less than 500. We tested the B-VNS and VNS algorithm to investigate their performances. However, there are limitations to this study. First, we used the most basic form of VNS algorithm. We did not use the advanced or hybrid form because we focused on the construction mechanism of the neighborhood structure. Many studies regarding the VNS algorithm have been carried out, previously, but reports on the use of alternative neighborhood structures to solve QUBO problems are very rare. Therefore, our study is useful for, fundamentally, developing the VNS algorithm. Even though we used small parameter settings, this was sufficient to observe the potential of both the B-VNS and VNS algorithms. [PERSON] reported that the VNS algorithm is sensitive to problem characteristics, so early iterations can be used to predict the potential of the algorithm [34]. The results in this paper were, also, inseparable from the simulation we used. One of the crucial factors in metaheuristic simulation is the random-number generator. We used the Fortran language and applied a dynamic random seed. Random seeds change over time. Each time a random number function is called, it will use a different seed, to avoid generating specific patterns of random numbers. It is worth considering that the programming approach applied in simulation codes may, also, impact the results. Our tests only involved small- to medium-sized problems. However, our experiments show that using binomial distribution can, potentially, enhance the VNS algorithm. Thus, experimenting on much larger cases becomes a challenge. Considering that hybrid models tend to be more powerful, the hybrid form of the B-VNS algorithm is worth studying. Considering the successful implementation of parallel programming of the VNS algorithm [48], the potential of the B-VNS algorithm can be further developed, by applying a suitable hybrid model.