## 2 Materials and Methods
### Identification Methods
#### 2.3.3 Feature Classification  
As mentioned in Section 1, the EN-based PI problem can be considered as a multi-class classification problem. To evaluate the performance of the four ensemble learning methods, that is, RF, SAMME, GBDT, and XGBoost, the total set of extracted features were divided into a training subset and a test subset. A statistical model was fitted to the training set and was then used to predict the corresponding perfume type of the samples in the test set. The principles of the four ensemble learning methods are outlined as follows:  
* RF  
As a typical bagging method, RF constructs multiple decision trees based on \(sd\) samples, which are drawn with replacement from the training set [15]. The number of decision trees constructed by RF is \(M\) (\(M>1\)). The base leaner in RF is the classification and regression tree (CART) classifier [13]. The CART classifier in RF uses the Gini index as the tree node splitting criterion. Each constructed CART classifier outputs a classification result. Then, the final classification result is determined according to the principle of minorities subordinating to the majorities. Note that the prototypical CART can be a regression or classification tree, which are called the CART regressor and the CART classifier, respectively [13]. The base learners of RF and SAMME are CART classifiers, whereas GBDT and XGBoost both employ CART regressors as the base learner. * SAMME  
SAMME, an improved form of AdaBoost [24], which sequentially fits copies of a CART classifier on the same dataset, and steers subsequent CARTs to focus more on incorrectly classified samples by increasing their weights [16]. The pseudo-code of SAMME is shown in Algorithm 2. SAMME shares the same structure of AdaBoost. The critical improvement is the term \(log(K-1)\) in line 5 of Algorithm 2, which directly influences the update of weights for the features. At the first iteration, the weights for all the \(n\) features are set to be the same. Then, as shown in line 6 of Algorithm 2, the weights of previously mis-classified samples are increased, whereas other weights are decreased. ```
1 Initialize the weight of feature \(x\): \(w\): \(w\) = 1/\(n\) for \(i\) = 1, 2,..., \(n\). 2 for\(m\) = 1 to \(M\)do
3 Construct a CART classifier \(T^{(m)}(x)\) based on the weighted training set \(\{w\): \(x_{i}\),..., \(w\): \(w_{i}\). 4 Calculate the error term: \(err^{(m)}=\frac{\sum_{i=1}^{n}w_{i}\left(\zeta_{i}\pi^{(m)}(x_{i})\right)}{ \sum_{x_{i}=1}^{n}w_{i}}\), where \(\mathds{1}(\cdot)\) is the indicator function that equals 1 if \(i\) - is true, and otherwise equals 0. 5 Compute \(a^{(m)}=\eta\cdot\left[log\frac{1-err^{(m)}}{err^{(m)}}+log(K-1)\right]\), where \(\eta\) is the learning rate parameter, \(K\) is the number of classes. 6 Update the weight \(w\) with \(w^{i}\cdot\exp\left(a^{(m)}\mathbb{I}\left(c_{i}\
eq T^{(m)}(x_{i})\right) \right)\) for \(i\) = 1, 2,..., \(n\). 7 Normalize \(w\), \(i\) = 1, 2,..., \(n\). 8 end
9 Using \(\alpha^{(m)}\) and \(T^{(m)}\), \(m\) = 1,..., \(M\) for classification: \(C(x)=arg\max_{k}\sum_{m=1}^{M}\alpha^{(m)}\cdot\mathds{1}\left(T^{(m)}(x)=k\right)\)
```  
**Algorithm 2** SAMME [16]. The pseudo-code of GBDT [17] is shown in Algorithm 3. For \(K\)-class (K > 2) classification problems, GBDT sequentially constructs \(K\times M\) CART regressors, where \(M\) is the number of iterations. In each iteration, an individual CART regressor is fitted for each class in expectation of minimizing a greedy loss function, which is equal to the negative gradients of the multiclass log loss [17]. As shown in line 3 of Algorithm 3, using the Sigmoid function, the continuous prediction value \(F_{k}(\mathbf{x}_{i})\) of the constructed CARTs are mapped to probabilities that \(\mathbf{x}_{i}\) belongs to the \(k\)-th class \(p_{k}(\mathbf{x}_{i})\). During the construction of CART regressors, the mean squared error with the improvement score by Friedman [17] was used as the splitting criteria to induce the trees and leaf nodes.