## 3 Analysis Plan
### Ensemble Methods  
Machine learning has been successfully applied for predictive modeling in various fields. Especially, ensemble methods have gained considerable attention in recent years because ensemble  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & & **Dropout** & **Non-Dropout** \\ \hline \multirow{2}{*}{Predicted lables} & Dropout & 0 & 0 \\  & Non-dropout & 100 & 9900 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The hypothetical confusion matrix showing the case where the accuracy of a binary classifier can be deceptive for an imbalanced dataset. In this example, the proportions of dropouts and non-dropouts are imbalanced, i.e., 100 dropouts (1%) vs. 9900 non-dropouts (99%). Despite of the complete misclassifications of the dropouts, the accuracy is still \(0.99=\left(0+9900\right)/\left(0+0+100+9900\right)=9900/10000\) because the correct predictions of the non-dropout dominate the accuracy.  
methods can substantially improve the accuracy of predictions by combining multiple machine learning algorithms [26]. Each algorithm in the ensemble methods is usually called a base (or individual or component) learner. The ensemble of learners has outperformed a single learner in various tasks, such as the object detection [27], lung cancer identification [28], and fraud detection [29]. The superior performance of ensemble methods mainly comes from the better generalizability of the ensemble of learners.  
In ensemble methods, bootstrap aggregating (bagging) and boosting are the two popular paradigms. As the name indicates, bagging improves the accuracy of predictions by aggregating predictions from multiple learners. In bagging, the \(N\) base learners are trained in parallel on \(N\) bootstrap samples from the original dataset, and then the \(N\) predictions from the \(N\) base learners are combined using majority voting or other synthesizing methods. Because of its aggregating (or averaging) nature, bagging can reduce the variance of a model. The random forest is a popular machine learning algorithm that uses the bagging paradigm. The random forest is the collection of \(N\) decision trees, and combines \(N\) predictions from the \(N\) decision trees trained on the \(N\) bootstrap samples to make a final prediction.  
Boosting is another ensemble paradigm. The idea behind boosting is to produce a series of weak learners to make a strong learner. Unlike bagging in which learners are trained in parallel and the predictions are aggregated without preference to any learner, boosting sequentially trains learners, and the predictions are aggregated with heavier weights on better learners. For example, the adaptive boosting (AdaBoost) calls a weak or base learner repeatedly in a series of rounds \(t=1,...,T\) by weighting previously misclassified instances with higher weight, and then combines the \(T\) predictions from the \(T\) base learners with the weights of learners determined during the training process [30]. Notice that both the instances and learners have their distributions of weights which are updated across the rounds. The weight distribution for instances at round \(t\) makes the learner at round \(t\) focus on the instances that were misclassified at round \(t-1\), whereas the weight distribution for learners determines how to combine the \(T\) learners after \(T\) rounds. In addition to the variance, the boosting can also reduce the bias because of its adaptive nature. In sum, the bagging trains a set of learners in parallel to reduce the variance by exploiting the independence between learners, whereas the boosting sequentially trains a series of learners to reduce both the bias and variance by exploiting the dependence between learners.  
In total, we trained four classifiers using the big data samples of the 165,715 high school students from the NEIS database in South Korea: random forest, boosted decision tree, random forest with SMOTE, and boosted decision tree with SMOTE.