# Towards AI-Generated Essay Classification Using Numerical Text Representation
## 5 Experiments and Results
### Results of Classification  
To evaluate the effectiveness of the text representations, we used various classifiers, including:  
* Logistic Regression: a baseline model effective for linearly separable data;
* Random Forest: an ensemble method that combines multiple decision trees to improve robustness and accuracy;
* XGBoost and LightGBM: advanced boosting algorithms that iteratively improve model performance by focusing on misclassified instances. Experiments were conducted on a dataset comprising essays labeled as either AI generated or student written. The dataset was split into training and test sets to objectively evaluate model performance. The division ensured that 75% of the data was used for training and 25% for testing, providing a reliable assessment of each model's performance. Various metrics such as accuracy, precision, recall, and F1-score were used to assess the classifiers. The comprehensive methodology employed in this study aimed to provide a robust comparison of various text representation techniques, highlighting their strengths and weaknesses in the context of text classification. This approach contributes to a better understanding of how different methods can be leveraged to improve natural language processing tasks. Table 7 shows the accuracy comparison for each classifier, highlighting the highest results for each. For logistic regression, the highest accuracy was observed with TF-IDF weights, achieving 99.82%. Similarly high values were obtained with Bag of Words (99.05%) and parameterized TF-IDF weights (98.98%). For random forests, the best performance was with word embeddings created using the fastText model (Skip-gram) trained on the essay dataset, achieving an accuracy of 98.03%. LightGBM also performed well with these embeddings, reaching an accuracy of 98.83%. For XGBoost, the highest accuracy was noted for Bag of Words (98.96%), with closely matching accuracies for TF-IDF weights, parameterized TF-IDF weights, and fastText word embeddings (Skip-gram), at 98.78%, 98.78%, and 98.91%, respectively. To provide a more detailed comparison, additional metrics were calculated from the confusion matrix. Besides accuracy, precision, recall, specificity, F1-score, and MCC (Matthews correlation coefficient) were considered. The values of these indicators were calculated assuming class 1 (essays generated by large language models) as the positive class. This allows for a precise evaluation of how well the embeddings and classifiers perform in detecting this class. The results are presented in Table 8, with the best results for each numerical representation highlighted. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Embedding Technique** & **Logistic Regression** & **Random Forest** & **XGBoost** & **LightGBM** \\ \hline Bag of Words & 99.05 & 97.90 & **98.96** & 98.63 \\ TF-IDF & **99.82** & 97.78 & 98.78 & 98.55 \\ Parameterized TF-IDF & 98.98 & 97.72 & 98.78 & 98.51 \\ fastText (Skip-gram) & 97.80 & **98.03** & 98.91 & **98.83** \\ fastText (Continuous Bag of Words) & 97.87 & 96.59 & 98.17 & 97.98 \\ fastText (Pretrained Model) & 89.88 & 95.41 & 97.42 & 96.87 \\ Sentence Embeddings MiniLM & 95.73 & 95.06 & 96.47 & 96.13 \\ Sentence Embeddings distilBERT & 96.63 & 93.78 & 96.24 & 95.60 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Accuracy comparison for individual embeddings and classifiers. For Bag of Words, TF-IDF weights, parameterized TF-IDF weights, and fastText word embeddings (Skip-gram), accuracies for all classifiers were equal to or greater than 97.72%. For fastText embeddings and Continuous Bag of Words, accuracies were equal to or greater than 96.59%. The lowest performance was observed for embeddings created using the pre-trained fastText model with logistic regression--this was the only instance where accuracy fell below 90% (89.88%). For the same embeddings and other classifiers, accuracies did not exceed 97.42%. For MiniLM sentence embeddings, the highest accuracy was 96.47%, and the lowest was 95.06%.