# Automatic Age and Gender Recognition Using Ensemble Learning
## 3 Dataset and Methods
### Ensemble Learning
#### 3.3.3 Base-Level Classifiers  
In classification based on ensemble learning, it is very important to determine the base-level classifiers that form the ensemble model. The basic idea in determining base-level classifiers is to provide rich diversity to the ensemble by using various types of machine learning algorithms. Therefore, it is recommended to include in the ensemble various models that make different assumptions in solving predictive modeling tasks, such as linear models, decision trees, support vector machines, and KNN. To provide the desired diversity in this study, a classifier pool consisting of KNN, SVM, RF, LR, and extra trees was selected as the base-level classifier. A brief description of these classifiers is given in the following subsection. KNN is one of the most commonly used supervised machine learning methods. The KNN method, which stands out with its ease of use and flexibility, has no training phase and is therefore also known as lazy learning [31]. The letter \(K\) in the name of the algorithm represents the number of neighbors included in the classification process and is one of the most important parameters affecting classification accuracy. In the first stage of the algorithm, to determine the class to which a test sample belongs, the distances between the test sample and all samples in the training dataset are calculated. Then, the \(K\) nearest neighbors are selected according to the calculated distances, and the class label of the test sample is determined according to the majority vote among the \(K\) nearest neighbors. There are different distance metrics such as Euclidean, Manhattan, and Minkowski used to determine the nearest neighbors. However, no value gives the best results in all datasets for both the distance metric and the number of nearest neighbors. For this reason, the optimum values of these parameters should be determined through a process called hyperparameter tuning. There are different methods used for hyperparameter tuning, such as manual tuning, random search, and grid search [32]. Of these, the grid search requires less experience and computational load and is therefore more popular. In the grid search, the aim is to determine the values that give the best results by scanning  
Figure 1: The schematic diagram of stacking ensemble learning. hyperparameter values within a certain range. The grid search method was used for the parameter optimization of all classifiers developed in this study. Support Vector Machine (SVM)  
SVM is one of the most powerful supervised machine learning methods used in both classification and regression problems. However, it is used more frequently for classification purposes. SVMs are mainly developed for binary classification, but they can also be extended to multi-class classification tasks using various methods [33]. The purpose of the SVM algorithm is to find the best possible decision boundary that separates data points belonging to different classes into their classes. While this boundary is a line in a two-dimensional feature space, it is called a hyperplane in high-dimensional space. There may be multiple hyperplanes that separate data points into their classes. The best of these is the hyperplane with the maximum margin; that is, the distance between classes is maximum, and SVM tries to determine it. The SVM algorithm was originally proposed to create linear classifiers [34]. However, with the use of an approach called the kernel trick, SVMs have been enabled to perform non-linear classification effectively. The kernel trick is a simple method based on projecting data that cannot be classified as linear into a higher dimensional space where they can be classified as linear [35]. There are various kernel functions used for this projection, such as polynomial kernel, Gaussian kernel, and sigmoid kernel. The kernel function is highly dependent on the dataset and the task, so it is usually determined as a result of an optimization process. In this research, the grid search was used to determine the kernel function.