# Automatic Age and Gender Recognition Using Ensemble Learning
## 3 Dataset and Methods
### Ensemble Learning
#### 3.3.3 Base-Level Classifiers
Logistic Regression (LR)  
LR is a supervised machine learning algorithm mainly used for classification tasks. LR aims to estimate the probability that a test sample belongs to a particular class. To do this, first a linear combination of the features of the samples is taken and then a non-linear sigmoidal function is applied to them [36]. Logistic regression and linear regression can sometimes be confused since they both have the term regression in their names. However, they are quite different from each other. Linear regression is a regression algorithm, and the output is continuous values. On the other hand, logistic regression is a classification algorithm, and its output is the probability that a sample belongs to a particular class. LR was originally developed for binary classification, but it can be straightforwardly extended to multiclass classification (it is called multinomial logistic regression). Random Forest (RF)  
RF is a decision tree-based ensemble learning approach widely used in both classification and regression problems. In simple terms, random forest can be described as a large group of decision trees (DTs) that work together to produce the best possible conclusions. Each of the individual trees in the RF is constructed using a random subset of the training dataset. This subset is created by randomly selecting samples from the training data with replacement, and this process is called bootstrapping. A similar randomness in the selection of bootstrap samples is also used in feature selection. At each node, a randomly selected subset of features is used instead of all features. These two randomness in the training phase help the RF model make better predictions by reducing the correlation between individual decision trees [31]. To classify a new sample, the input vector of the sample is applied to each DT in the forest, and each DT produces an output by evaluating a different part of the input vector. These results are then combined according to majority voting to determine the final prediction. Various parameters affect the accuracy of the RF classifier. Of these parameters, the number of decision trees in the forest and the number of features considered for splitting at each leaf node are the most critical [37]. In this research, these two parameters were optimized with the grid search method. Extra Tree (E-TREE)  
E-TREE [38] is a modified version of the random forest classifier. Like random forest, it builds a large number of decision trees and splits nodes using a random subset of features. The main difference between E-TREE and RF is the changes in the strategies used to ensure variability of the samples and covariates. The first difference arises in the selection of the training set of each decision tree. While random forest uses randomly selected bootstrap samples, extra trees uses the entire training set. Another difference between the two methods is related to the selection of features used in each decision tree. In RF, a random subset of features is selected for each tree, and the best feature for each node split is selected based on some mathematical criterion (typically the Gini index). On the other hand, E-TREE is more aggressive and randomly selects a threshold value for each feature to split the node. Thus, E-TREE becomes more random than RF because it eliminates the bias caused by the selection of the best feature.