# SBNNR: Small-Size Bat-Optimized KNN Regression
## 4 Proposed Method
### Proposed SBNNR Method  
The proposed SBNNR (Small-size Bat-Optimized Nearest Neighbor Regression) is explained in Algorithm 3. In this algorithm, the length of position vectors is set to \(n=m+2d+9\). All components are doubles in the range of \((0,1)\):  
* The First \(m\) components are used to select instances. * Components \(x_{i}^{m+1},\ldots,x_{i}^{m+d}\) are used to select features. * \(x_{i}^{m+d+1},\ldots,x_{i}^{m+2d}\) are coefficients of the weight function that is used in making final estimations. * \(\delta=x_{i}^{m+2d+1}\) is the instance selection threshold. * \(\mu=x_{i}^{m+2d+2}\) is used in determining \(k\) in the KNN algorithm. * GAN settings: \[GAN \begin{cases}LearningRate_{GAN}=x_{i}^{m+2d+3}\\ BatchSize_{GAN}=2^{[9\times x_{i}^{m+2d+4}]}\\ Layers_{GAN}=1+[x_{i}^{m+2d+5}\times 10]\end{cases} \] (9)
* DNN settings: \[DNN \begin{cases}LearningRate_{GAN}=x_{i}^{m+2d+6}\\ BatchSize_{GAN}=2^{[9\times x_{i}^{m+2d+7}]}\\ Layers_{GAN}=1+[x_{i}^{m+2d+8}\times 10]\\ DropoutRate_{GAN}=x_{i}^{m+2d+9}\times 0.5\end{cases} \] (10)  
All of these components or decision variables are used to \"translate\" the position of a bat into a KNN predictor, as mentioned in line 12 of Algorithm 3. The \"translate\" routine is defined in the pseudo-code of Algorithm 4. In other words, in this routine, a typical KNN estimator is created with selected features and instances, and \(k\) is determined by \(\mu\), with the weight function shown in line 14 of Algorithm 4. One other building block of the proposed method is the objective function shown by \(Performance(x)\) in Equation (12). The objective function is based on the \(R^{2}\) score (Equation (11)), which is the most widely used criterion for evaluating the performance success of regression problems. It indicates how well the trends of the estimated results track the trends of the observed data [36]. ```
1:Input: Training Dataset \(S\) with \(m\) data points and \(d\) input features. 2:Initialize dimension of Bats positions \(n=m+2d+2\)
3:Initialize population of bats with position \(x_{i}\), velocity \(v_{i}\) and pulse frequency \(f_{i}\)
4:Set initial values for: * maximum iterations \(T\) * pulse rates \(r_{i}\) * loudness \(A_{i}\)
5:for\(t\) in \(\{1,2,\ldots,T\}\)do
6:for each bat \(b_{i}\)do
7: Utilize Equations (1) and (2) to generate new solutions. 8:if\(rand>r_{i}\)then
9: Choose a solution amongst the best solutions list. 10: Create a local solution centered on optimal solution. 11:endif
12: Translate Position \(x_{i}\) to a predictor \(KNN_{i}\)
13: Generate a random number \(rand\) uniformly in \([0,1]\)
14:if\(rand<A_{i}\) and \(Performance(KNN_{i})<Performance(Global_{Best})\)then
15: Mark new solutions as accepted. 16: Increase \(r_{i}\) and decrease \(A_{i}\). 17:endif
18:endfor
19:endfor
20:Output: the model with best fitness
```  
**Algorithm 3** SBNNR Algorithm  
```
1:Input: Training Dataset \(S\) and position vector \(x\). 2:\(\delta=x_{(m+2d+1)}\) and \(\mu=x_{(m+2d+2)}\)
3:\(S_{selected}=\phi\) and \(S_{selected}=\phi\)
4:for\(j\in\{1,\ldots,m\}\)do
5:if\(x_{j}>\delta\)then
6: Add \(S_{j}\) to \(S_{selected}\). 7:endif
8:endfor
9:for\(j\in\{m+1,\ldots,m+d\}\)do
10:if\(x_{j}>\delta\)then
11: Add \(j^{th}\) feature to \(F_{selected}\). 12:endif
13:endfor
14:Define weight function \((\omega:S_{selected}\rightarrow(0,1))\) for \(j\)-th selected instance as: \(\omega(j)=\frac{x_{(m+d+j)}}{z_{i}\in S_{selected}}\)
15:\(k=\mu\times|S_{selected}|\)
16:Develop GAN and DNN models for generate synthetic data and feature extraction based on \(S_{selected}\), \(F_{selected}\) and Equations (9) and (10). 17:\(D^{\prime}=\) Dataset after applying GAN and DNN.