# Identifying the Primary Odor Perception Descriptors by Multi-Output Linear Regression Models
## 2 Materials and Methods
### POPDs and the Identification Frameworks
#### 2.2.2 Validation Models  
As mentioned above, validation models identify the predictable and undetermined odor perceptual descriptors from the undetermined ones obtained from the former disjunctive model. The perceptual ratings of the CCOPDs are feed into the validation model as the inputs, and the perceptual ratings of the undetermined odor perceptual descriptors are as the outputs. Therefore, this validation task is a multi-output regression problem in the machine learning community [47]. In order to solve this multi-output regression problem, an assembly model composed of multiple independent regression models can be used, where each model is applied to each output variable or a single regression model with all target variables as output is used. The former approach is adopted in this paper due to its flexibility in fitting the data. Due to this dataset's moderate size, the classical and stable linear regression models have implemented: Ridge regression, LASSO regression, and ElasticNet regression. Therefore, a separate linear regression function is employed to predict one undetermined odor perception in the assembly model.  
The hypothesis for a single linear regression task is given as follows:  
\[y\big{(}x,w\big{)}=w^{T}x+w_{0} \tag{13}\]  
where \(x\) is the \(K\)-dimensional vector of the ratings for the clustering-centers odor perception descriptors, \(x\in\mathbb{R}^{K}\), \(K\) is the number of clusters, \(w\) is the weight vector, and \(w_{0}\) is the bias. The hypothesis is shared among three linear regression models. The difference lies in the cost function, which causes the parameters of these models to be different. Using square error to optimize, the cost function of the Ridge regression model is expressed as:  
\[J(w)=\frac{1}{2m}\sum\limits_{i=1}^{m}\big{(}y(x_{i},w)-y_{i}\big{)}^{2}+\lambda \sum\limits_{j=1}^{K}w_{j}^{2} \tag{14}\]  
where \(y_{i}\) is the real label of the output, i.e., the perceptual ratings of a NCCOPD. \(m\) is the total number of training samples, and \(\lambda\) is a hyperparameter to balance the model complexity and avoid overfitting.  
The cost function of LASSO regression model is given as:  
\[J(w)=\frac{1}{2m}\sum\limits_{i=1}^{m}\big{(}y(x_{i},w)-y_{i}\big{)}^{2}+ \lambda^{\prime}\sum\limits_{j=1}^{K}|w_{j}| \tag{15}\]  
where \(\lambda^{\prime}\) is the hyperparameter for LASSO model, functioning the same as the hyperparameter of Ridge model.  
The cost function of ElasticNet model is expressed as:  
\[J(w)=\frac{1}{2m}\sum\limits_{i=1}^{m}\big{(}y(x_{i},w)-y_{i}\big{)}^{2}+ \lambda^{\prime\prime}\big{(}\rho\sum\limits_{j=1}^{K}w_{j}^{2}+(1-\rho)\sum \limits_{j=1}^{K}|w_{j}|\big{)} \tag{16}\]  
where \(\lambda^{\prime\prime}\) and \(\rho\) are the hyperparameters.  
The Pearson correlation coefficients between the predicted values and the actual labels are used as the performance metric for each model based on representative literature in odor perception prediction [44; 48; 49]. A quarter of the samples is used for testing. the other samples are used for training and validation, and ten-fold cross validation are implemented. \(\lambda\) for Ridge regression model is searched in set {0.0001, 0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10, 30, 100}. \(\lambda^{\prime}\) for LASSO regression model is searched in set{0.0001, 0.0003,0.001, 0.03, 1, 3,10, 30,100, 200,400}. For ElasticNet regression model, \(\lambda^{\prime\prime}\) is searched in set{0.0001, 0.0003,0.001, 0.003,0.01, 0.03, 0.1, 3,10, 30,100}, and \(\rho\) is searched in set {0.1, 0.2, 0.5, 0.7, 0.9}. The training and testing of all the linear regression models are implemented 30 times, then the predicting performances of the testing set in all the 30 runs are averaged as the final predicting performance.