# Advanced Default Risk Prediction in Small and Medium-Sized Enterprises Using Large Language Models
## 3 Dataset and Methods
### Methods
#### 3.2.2 Machine Learning Methods  
Logistic Regression, originating from the field of econometrics, stands as a powerful tool for handling classification problems, particularly excelling in binary classification scenarios [34]. Its evolution enables the machine learning domain to predict the probability of event occurrences using this model, thereby furnishing an effective mathematical approach for classification decisions. Logistic regression not only boasts a concise and comprehensible model form but also provides a probabilistic interpretation of the model's prediction results, which is a crucial reason for selecting this method in this study. Support Vector Machines  
Support Vector Machine (SVM) is a powerful supervised learning algorithm that is primarily utilized for classification prediction [35]. Its core concept involves finding the optimal separating hyperplane within the dataset; one that maximizes the margin between datapoints of different classes. By incorporating kernel tricks, SVM effectively handles linearly inseparable data. Given the presence of numerous features in the dataset, SVM can handle high-dimensional data and performs well even in cases of linearly inseparable or nonlinear data, enabling effective classification prediction. Neural networks, inspired by the structure and functionality of the human brain, represent a computational model [36]. This approach often excels in capturing patterns and regularities hidden within datasets with complex structures and numerous features. Given the multifaceted nature of non-financial enterprise data in this study's dataset, neural networks can effectively learn intricate relationships among these data and handle various types of feature data. The decision tree is a widely adopted machine learning algorithm utilized for classification and regression tasks [37]. Decision trees are renowned for their ease of comprehension and interpretation, while also exhibiting relative insensitivity to missing values and irrelevant features. Considering the possibility of irrelevant features or missing values within the dataset, decision trees can effectively handle such scenarios. Additionally, owing to their intuitive nature, they contribute to the interpretation and explanation of the data. Random Forests  
Random Forest [38] is an influential ensemble learning methodology designed to enhance prediction accuracy and resilience by amalgamating the forecast results of numerous decision trees. This approach notably enhances prediction accuracy and robustness, especially when confronted with noise and anomalies within the dataset. In the financial domain, the random forest finds utility in detecting corporate credit risks or the likelihood of default. As evidenced by several studies, this algorithm can scrutinize a company's historical financial data, credit records, and market activities to discern intricate patterns and relationships indicative of future default events, thereby ensuring superior accuracy and reliability compared to individual decision trees [39; 40]. XGBoost  
XGBoost is another efficient and powerful gradient boosting framework that is widely employed across various practical applications, acclaimed for its outstanding performance and speed. Its advantages lie in optimizing the computational efficiency and resource consumption of traditional gradient boosting algorithms while enhancing model accuracy. In the realm of enterprise credit risk prediction within the financial domain, XGBoost stands out as a potentially highly effective tool. Its capability to handle vast arrays of features enables it to probabilistically identify key factors influencing corporate credit ratings [41; 42]. LightGBM  
LightGBM (light gradient boosting machine) [43] is an efficient gradient boosting framework renowned for its superior speed, heightened efficiency, and reduced memory footprint.