# University Student Dropout Prediction Using Pretrained Language Models
## 3 Methodology
### Modeling  
#### 3.2.1 Formulating Numerical and Categorical Variables into a Natural Language Format  
We formulated the numerical and categorical variables of freshmen in a natural language format to leverage the knowledge of the PLM for the student dropout prediction task. We denoted the demographic variables of freshmen as \(D=\{d_{i}|d_{i}\in\mathbb{R},i=1,2,\ldots,n_{d}\}\), where \(n_{d}\) denotes the number of demographic variables. To exploit the textual understanding power of PLMs, we formulated all these variables into a natural language format and created sentences. We created a sentence using the mapping function \(f_{d}(D)=D^{*}\), where \(D^{*}\) is a sequence consisting of the vocabulary of the PLMs. Similarly, we denoted the academic variables as \(A=\{a_{i}|a_{i}\in\mathbb{R},i=1,2,\ldots,n_{a}\}\), where \(n_{a}\) is the number of academic variables. We then formulated them into a sentence using \(f_{a}(A)=A^{*}\). In Table 3, we used various formats depending on the specificity of each feature. First, for the categorical type, we formulated a different sentence format based on the value of the feature, excluding gender. For instance, we formulated the birth year, such as \"2002\", as in the sentence \"Born \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Feature** & **Mean** & **SD** & **Median** & **Min** & **Max** \\ \hline Major GPA & 3.27 & 1.15 & 3.5 & 0.0 & 4.5 \\ Nonmajor GPA & 3.31 & 1.1 & 3.58 & 0.0 & 4.5 \\ Volunteee hours & 1.99 & 10.77 & 0 & 0 & 130 \\ Parental financial income * & 55.07 & 27.21 & 63 & 8 & 85 \\ Course evaluation score & 4.2 & 0.41 & 4.27 & 2.35 & 5.0 \\ \hline \hline \end{tabular}
* The parental financial income is calculated only with data from students who received a national scholarship.  
\end{table} 
Table 2: Statistics of numerical data. The parental financial income is given as a percentile. \begin{table}
\begin{tabular}{c c} \hline \hline
**Group** & **Feature** & **Natural Language Form** \\ \hline \hline \multicolumn{2}{c}{Birth year} & Born in 2002 \\ \hline Gender & Male \\ \hline \multicolumn{2}{c}{Nationality} & South Korean nationality \\ \hline \multicolumn{2}{c}{Graduated in 2021 from} \\ \multicolumn{2}{c}{Graduation year,} & Yeokgok High School, \\ \multicolumn{2}{c}{high school,} & a public high school in \\ \multicolumn{2}{c}{high school region} & Yeokgok-dong, Bucheon-si, \\ information & & Gyeonggi-do \\ \hline \multicolumn{2}{c}{Admission year,} & Admitted in 2021 \\ \multicolumn{2}{c}{admission type,} & with early admission \\ \multicolumn{2}{c}{detailed admission type} & —writing test admission \\ \hline \multicolumn{2}{c}{Major field} & Major field is \\ \multicolumn{2}{c}{natural sciences} \\ \hline \multicolumn{2}{c}{Parental financial income} & Below 53\%, similar \\ \multicolumn{2}{c}{to average students} \\ \hline \hline \multicolumn{2}{c}{Major GPA} & 4.17 GPA in major, \\ \multicolumn{2}{c}{excellent} \\ \hline \multicolumn{2}{c}{Nonmajor GPA} & 3.5 GPA in nonmajor, \\ \multicolumn{2}{c}{normal} \\ \hline \multicolumn{2}{c}{Volunteer hours} & No volunteer activity \\ \hline \multicolumn{2}{c}{Student Club} & Participated in a student club \\ \hline \multicolumn{2}{c}{Academic activity} & Did not participate \\ \multicolumn{2}{c}{in an academic activity} \\ \hline \multicolumn{2}{c}{Course evaluation score,} & Course evaluation : 3.93, negative, \\ \multicolumn{2}{c}{course evaluation comments} & too many assignments \\ \hline \hline \end{tabular}
\end{table} 
Table 3: An example of formulating each feature into a natural language format. in 2002\". In contrast, we used the gender as is, such as \"Male\" in the sentence. Additionally, for the categorical type indicating participation status, we formulated two types of sentences. For instance, we formulated the student club, such as \"yes\" which means the student participated in a student club, as \"Participated in a student club\", otherwise, we used \"Did not participate in student club\".