# SBNNR: Small-Size Bat-Optimized KNN Regression
## 1 Introduction  
Machine learning has rapidly become a ubiquitous tool across various scientific domains, offering significant capabilities for data analysis and modeling. Whenever experimental, laboratory, or observational datasets are available, machine learning is frequently the first choice for data-driven exploration and insight generation. The landscape of real-world data is marked by its diversity, encompassing a wide spectrum of feature types and varying data volumes. While some fields benefit from plentiful data, many others suffer from limited datasets due to the challenges of experimental data collection. In fact, it is not uncommon to encounter datasets with fewer than 100 samples, prompting researchers to explore machine learning approaches customized for small sample sizes. This \"small data\" domain spans various fields, such as biomedicine, material design, petroleum engineering, biodiesel research, and numerous other disciplines [1, 2]. Developing models under the constraints of small datasets poses unique challenges, primarily related to data distribution and gaps. In datasets with sufficient samples, data points are typically distributed across a continuous feature space. However, smaller datasets often present sporadic or unevenly distributed data points [3], which can lead to overfitting and low prediction accuracy [4]. Addressing these challenges requires specialized frameworks and algorithms designed to enhance model generality and accuracy, particularly for regression tasks, where the synthesis of artificial data points within a feature space must be coupled with appropriate target values [5]. In this research, we focus on regression problems constrained bysmall data sizes, typically comprising around 500 samples or fewer. We aim to develop a framework based on the KNN algorithm that will enhance the robustness and reliability of models when dealing with limited data. For feature selection, instance selection, and hyperparameter tuning, we applied the Bat Algorithm (BA) to optimize these components. To address data sparsity challenges, we employed Generative Adversarial Networks (GANs) to generate synthetic data, enhancing the dataset's diversity. Additionally, DNNs were utilized as a deep learning approach for feature extraction from both synthetic and real datasets, ensuring a comprehensive representation of features. The KNN algorithm was chosen as our base model due to its effectiveness in handling small datasets without requiring strict parametric assumptions. KNN's non-parametric nature allows it to directly adapt to the inherent structure of data, making it particularly valuable for limited samples, where traditional assumptions about data distribution could lead to biased models. Furthermore, KNN's simplicity supports straightforward optimization strategies for instance and feature selection, both of which are critical in small-data contexts. By combining KNN with synthetic data generation through GANs and feature extraction using DNNs, we further enhance its capacity to capture underlying patterns, reduce overfitting, and improve generalizability. This hybrid approach takes advantage of KNN's simplicity while addressing small-data limitations, creating a robust and reliable framework for real-world applications. In other words, we selected KNN over other common algorithms because of its simplicity and interpretability, which make it well-suited for small datasets by allowing predictions based directly on similarity without requiring a complex model structure. For example, Support Vector Regression (SVR) may require the selection of a kernel function and specific assumptions about data distribution, which can introduce bias in small datasets where patterns are less predictable. On the other hand, while algorithms such as Random Forest (RF) also seem to be a good choice, they are often prone to overfitting with limited data due to their complexity, requiring more samples to generalize effectively.