## 3 Proposed System
### Integration of TF-IDF and XGBoost for Educational Data Classification  
This study proposes a model that integrates Term Frequency-Inverse Document Frequency (TF-IDF) and eXtreme Gradient Boosting (XGBoost) to analyze and classify diagnostic assessment data of students. This combination is designed to effectively process textual data and provide personalized feedback to students with high accuracy. TF-IDF is a feature extraction technique that converts textual data into numerical vectors, making them suitable for machine learning models. This study leverages the following advantages of TF-IDF:  
1. Importance-based weighting: words that frequently appear in a specific document but are rare across other documents receive higher weights, allowing the model to effectively learn meaningful patterns from text data. 2. Efficient data processing: TF-IDF enables the extraction of key terms even from concise text inputs, improving data quality and optimizing model learning. 3. Sparse matrix representation: by transforming text into high-dimensional vectors while maintaining computational efficiency, TF-IDF is well-suited for large-scale data processing. The feature vectorization process using TF-IDF plays a crucial role in enabling the learning model to understand and analyze textual data patterns. XGBoost, a tree-based boosting algorithm, offers several advantages for text data analysis:  
1. High classification performance: it delivers strong classification accuracy even with limited data and achieves fast training speeds. 2. Overfitting prevention: techniques such as limiting tree depth and applying subsampling prevent excessive fitting to training data. 3. Feature importance evaluation: XGBoost allows the assessment of feature importance, providing insight into key data patterns and enhancing the interpretability of textual data. 4. Non-linear data processing: the algorithm effectively captures complex non-linear relationships, enabling accurate classification of various error types. To ensure stable learning performance, this study fine-tuned key hyperparameters of XGBoost. The following settings were applied. To perform multi-class classification, the loss function was set to log-loss by configuring eval_metric = 'logloss'. To regulate the weight adjustments at each training step, a learning rate of learning_rate = 0.1 was applied. Additionally, the number of trees was set to n_estimators = 300, ensuring a sufficient number of training iterations for model convergence. To prevent overfitting, the maximum depth of trees was restricted to max_depth = 4. Furthermore, subsampling was employed, setting subsample = 0.8 to use 80% of the data for training at each iteration. Likewise, to enhance model stability and diversity, the proportion of features used per tree was set to colsample_bytree = 0.8. For multi-class classification, the softmax function was applied by setting objective ='multi:softmax', ensuring that the model predicts discrete class labels rather than probability distributions. Additionally, the number of unique class labels in the dataset was explicitly defined using num_class, aligning the model structure with the classification task. This combination of hyperparameters effectively prevented overfitting while maintaining high classification performance. The model demonstrated reliable classification results for key labels such as \"Calculation Error\" and \"Concept Deficiency\", ensuring stability even in low-frequency categories. The proposed integration of TF-IDF and XGBoost is designed to maximize classification performance. This combination provides several key benefits:  
1. Efficient text processing: TF-IDF converts textual data into numerical vectors, ensuring optimal input representation for learning models. The sparse matrix format enhances computational efficiency. 2. Accurate classification: XGBoost utilizes TF-IDF-generated feature vectors to achieve high classification accuracy with fast training times. 3.