## 4 Data Mining Methods for Graduates' Cultivation
### Factor Selection and Classification
#### 4.2.2 Classification Algorithms  
In order to verify the validity and rationality of the feature selection results, and improve the accuracy of our predictions, we then use a variety of machine learning methods to train classifiers using the 11 features selected by Lasso and OMP methods and all of the features of the original data, respectively. The classification prediction methods selected in this paper cover four categories: decision tree, regression, neural network, and statistical methods, including Support Vector Machines (SVM), Naive Bayes (NB), Logistic Regression (LR), Random Forest (RF), and Multilayer Perceptron (MLP). These classification methods are presented as follows:  
**Support Vector Machines** (SVM) was first proposed by [PERSON] and [PERSON] in 1995. It shows many unique advantages in solving small sample, nonlinear and high-dimensional pattern recognition, and can be applied to other machine learning problems, such as function fitting [37]. It is based on the VC dimension theory of statistical learning theory and the principle of structural risk minimization, based on the complexity of the model information (i.e., the accuracy of learning for specific training samples ) and learning ability (i.e., without error). Find the best compromise between the ability to identify any sample in order to obtain the best promotion. Essentially, the SVM algorithm is a class classifier, a hyperplane that separates different classes of samples in the sample space. This means that, given a number of labelled training samples, the SVM algorithm outputs an optimally separated hyperplane. **Naive Bayesian** is a classification method based on [PERSON] theorem and feature condition independent hypothesis [38]. The basic idea is to solve the items to be classified under the conditions of this occurrence. The probability, which is the largest, is considered to be the category to which the item to be classified belongs. **Logistic Regression**, although with regression label, logical regression is a classification algorithm that fits the data to a logit function (or logistic function) to predict the probability of an event occurring [39]. It starts with a linear regression with a continuous value that has practical significance, but linear regression has no way to accurately and robustly segment the problem of classification. Therefore, an algorithm such as logistic regression is designed, and its output is characterized. The probability that a sample belongs to a category. **Random Forest** is a classifier that contains multiple decision trees. It is an important integrated learning method based on Bagging, which can be used for classification and regression. Bagging's strategy is to select in samples from the sample set through resampling (Bootstrap has put back repeated samples); train a weak classifier for these n samples (can be ID3, C4.5, CART, SVM, LR). The method is repeated; the above two steps are repeated m times to obtain m weak classifiers; the data are placed in the m classifiers, and the prediction results of the data are determined according to the voting results of the m classifiers. The random forest is an improvement based on Bagging. The corresponding strategy is to select n samples from the sample set using Bootstrap sampling. Select k attributes from all the attributes, and then use the information gain and Gini index method to find the most. The good segmentation attribute establishes the CART decision tree (also SVM, LR, etc. ), where k controls the degree of randomness introduction, repeats the above process to establish m classifiers, uses these trees to form random forests, and obtains predictions by averaging results. **Multi-Layer Perceptron** (MLP) is a feed forward artificial neural network model that maps a set of input vectors to a set of output vectors [40]. It is an important artificial neural network. The network consists of an input layer composed of sensing units, one or more layers of computing nodes to form a hidden layer, and one layer of computing nodes to form an output layer.