## 4 Model Introduction
### Optimization  
XGBOOST is a commonly used algorithm in machine learning that performs very well on most classification tasks. Unfortunately, it is not very good at the task of predicting students' postgraduation plans. However, applying the method we propose to XGBOOST can improve its effectiveness.  
Similarly to other boosting algorithms, XGBOOST is an iterative decision tree algorithm; its base learner is a classification and regression tree (CART) and constructs an integration model in a phase-iterative manner. The ACCBOX algorithm's iterative update process is shown in Algorithm 1.  
```
Input: the students' behavior dataset \(D\), the colleges' prototype dataset \(D^{\prime}\), the college clusters \(U\). Loss function: \[\min_{\phi}\frac{1}{2}\sum_{i=1}^{n}\|\mathbf{y}_{i}-\mathbf{y}_{i}\|_{2}^{2}+ \frac{\alpha}{2}\Omega\left(\phi\right)+\frac{\beta}{2}\sum_{i=1}^{n}\|\phi \left(\mathbf{x}_{i}\right)-\sum_{j=1}^{c}\mathbf{t}_{j}\cdot\mathbb{I}\left( \mathbf{x}_{i}\in\mathbf{U}_{j}\right)\|_{2}^{2}.\] Output: CART tree ensemble \(\phi(x,z)\)
1 Initialize the model: \[F_{0}(\mathbf{x},\mathbf{z})=wf_{0}(X,Z)\] (18)
2 k=1
3 while(k<=K) Calculate the residuals: \[g_{i}=\phi\left(\mathbf{x}_{i}\right)-y_{i}\] (19) \[\hat{g}_{i}=\phi\left(\mathbf{x}_{i}\right)-\sum_{j=1}^{c}\mathbf{t}_{j} \cdot\mathbb{I}\left(x_{i}\in U_{j}\right).\] (20)
5 Fit a CART tree \(f_{k}\) to the above three residuals. Find the optimal weight for \(f_{k}\) by minimizing the following loss function: \[w_{k}=\underset{w}{\arg\min}\frac{1}{2}\sum_{i=1}^{n}\|F_{k-1} \left(x_{i}\right)+wf_{k}\left(x_{i}\right)-\mathbf{y}_{i}\|_{2}^{2}\] (21) \[+\frac{\alpha}{2}\Omega\left(\phi\right)\] \[+\frac{\beta}{2}\sum_{i=1}^{n}\|F_{k-1}\left(x_{i}\right)+wf_{k} \left(x_{i}\right)-\sum_{j=1}^{c}\mathbf{t}_{j}\cdot\mathbb{I}\left(\mathbf{x }_{i}\in\mathbf{U}_{j}\right)\|_{2}^{2}.\] Update the model: \[F_{k}(x,z)=F_{k-1}(x,z)+\alpha w_{k}f_{k}(x,z),\] (22) where \(\alpha\) refers to the learning rate of our model k=k+1;
6 endwhile;
7 output \[\phi(x,z)=F_{K}(x,z).\] (23)
```  
**Algorithm 1**Iterative update of ACCBOX  
At each iteration, ACCBOX uses the integrated model obtained at that stage to calculate the residual of the model's predicted and true values. There are two parts in the model: the residual of the student's career choice and the predicted value, the residual between the prototypical cluster centers' data and the predicted value, as shown in Equations (19) and (20).  
To update the existing integration model, it is necessary to train a CART tree in each iteration to fit the above three residuals and add that tree to our integration model. To ensure that the addition of that tree can benefit our model, we need to continuously optimize the parameters of that tree. The optimal parameters of that tree can be obtained by minimizing the loss function as shown in Equation (21). As a result, we can obtain the optimal tree ensemble by K-round iteration, as shown in Equation (23).  
In summary, we have fully introduced our ACCBOX model through three steps: prototypical cluster center generation, model training, and optimization. The pseudocode of the model algorithm is shown in Algorithm 2.  
```
Input: \(\mathcal{D}=\{\mathbf{x}_{i},\mathbf{y}_{i}\}_{i=1}^{n}\) the students' behavior dataset \(\alpha,\beta:\) the regularization hyperparameters \(U:\) the college clusters \(\hat{x}:\) the student test set Output: \(\hat{y}:\) the prediction of student's career choice
1 divide \(x\) into different colleges' clusters;
2 calculate college cluster centers' prototypes \(\{z_{j}\}\in R^{c\times d}\) according to Equation (10);
3 calculate prototypes' career choice labels \(\{t_{j}\}\in[0,3]^{c\times 1}\) according to Equation (11);
4 train the best tree ensemble model structure according to Algorithm 1;
5 using the tree ensemble model to predict student's career choice \(\hat{y}\);
```  
**Algorithm 2**The Approach Cluster Centers Based On XGBOOST (ACCBOX) algorithm.