## 3 Methodology
### Learning Process  
Various ML models were evaluated for their suitability in the task of KOA prediction. A brief description of these models is given below.  
We tested logistic regression [26] which is likely the most commonly used algorithm for solving classification problems. Logistic regression models the probabilities for classification problems with two possible outcomes. It's an extension of the linear regression model for classification problems. The interpretation of the weights in logistic regression differs from the interpretation of the weights in linear regression, since the outcome in logistic regression is a probability between 0 and 1. We also evaluated decision trees (DTs) [27] which are a non-parametric supervised learning method used for classification and regression. They are simple to understand and to interpret. DTs require little data preparation and perform well even if their assumptions are somewhat violated by the true model from which the data were generated.  
K-Nearest Neighbor (KNN) [28] as well as non-linear support vector machines (SVM) algorithms [29], which can deal with the overfitting problems that appear in high-dimensional  
Figure 6: Pseudocode for the implementation of the proposed feature selection (FS).  
spaces. In the classification setting, the KNN algorithm essentially boils down to forming a majority vote between the K most similar instances to a given \"unseen\" observation. Similarity is defined according to a distance metric between two data points. A popular one is the Euclidean distance method. Furthermore, SVMs are a set of supervised learning methods used for classification, regression and outlier's detection. They are effective in high dimensional spaces and still effective in cases where the number of dimensions is greater than the number of samples.  
The ensemble technique Random Forest (RF) [30] was also evaluated using DT models as weak learners. RF classifier creates a set of decision trees from randomly selected subsets of training set. It then aggregates the votes from different decision trees to decide the final class of the test object. XGboost [31] and naive Bayes [32] algorithms were also considered. XGboost model is a sum of CART (tree) learners which try to minimize the log loss objective and the scores at leaves. These scores are actually the weights that have a meaning as a sum across all the trees of the model. Furthermore, they are always adjusted in order to minimize the loss. Moreover, naive Bayes methods are a set of supervised learning algorithms based on applying [PERSON] theorem with the \"naive\" assumption of conditional independence between every pair of features given the value of the class variable. Naive Bayes learners and classifiers can be extremely fast. The decoupling of the class conditional feature distributions means that each distribution can be independently estimated as a one-dimensional distribution.  
Hyperparameter selection was implemented to optimize the performance of our models and to avoid overfitting and bias errors. Each model was optimized with respect to a number of preselected hyperparameters (Table 2). Specifically (i) 'gamma': [0, 0.4, 0.5, 0.6],'maximal depth': [1; 2; 3; 4; 5; 6; 7; 8],'minimum child and weight': [1; 3; 4; 5; 6; 8] were optimized for XGboost, (ii) 'criterion': ['gini', 'entropy'],'minimum samples leaf': [1; 2; 3],'minimum samples split': [3; 4; 5; 6; 7] and 'number of estimators': [10; 15; 20; 25; 30] for random forest, (iii)'maximal features': ['auto','sqrt', 'log2'],'minimum samples leafs': [1; 2; 3; 4; 5; 6; 7; 8; 9; 10; 11] and'minimum number of decision splits': [2; 3; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15] for decision trees, (iv) 'C': [0.001, 0.01, 0.1, 1, 2, 3, 4; 5; 6, 7; 8, 9; 10] and 'kernel': ['linear','sigmoid','rbf','poly'] for SVMs, (v) 'k-parameter': [5; 7; 9; 12; 14; 15; 16; 17] for KNN and (vi) 'penalty': ['11', 'l2'] and 'C': [100, 10, 1.0, 0.1, 0.01] for logistic regression.