# Application of Machine Learning Models for Baseball Outcome Prediction
## 2 Method
### Evaluated Machine Learning Model  
The models' performance was evaluated using six key metrics: accuracy, F1 score, sensitivity, specificity, positive predictive value, negative predictive value, and the area under the receiver operating characteristic curve. The AUC-ROC (area under the receiver operating characteristic curve) is a key metric for evaluating the overall performance of a classification model. The performance of the machine learning models in this study was evaluated using six key metrics, each providing unique insights into the model's predictive capabilities. The formulas for these metrics are as follows:  
Accuracy--Measures the overall correctness of the model's predictions:  
\[\text{Accuracy}=\frac{\text{{True Positive}}+\text{{True Negative}}}{\text{{Positive}}+\text{{Negative}}} \tag{1}\]  
F1 score--Represents the harmonic mean of precision and recall, balancing false positives and false negatives:  
\[\text{F1 Score}=2\ \times\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}} \tag{2}\]  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Model** & **Hyperparameters Used** \\ \hline Logistic Regression & penalty = ‘l2’, solver = ‘lbfgs’, max_iter = 1000 \\ Decision Tree & criterion = ‘gin’, max_depth = 5, min_samples_split = 10 \\ Random Forest & n\_estimators = 100, max_depth = 10, min_samples_split = 10, bootstrap = True \\ XGBoost & n\_estimators = 100, max_depth = 6, learning_rate = 0.1, subsample = 0.8, colsample_bytree = 0.8 \\ LightGBM & n\_estimators = 100, learning_rate = 0.1, num_leaves = 31, max_depth = \(-\) 1 \\ ANN & 1 hidden layer (64 units), ReLU activation, dropout = 0.3, optimizer = Adam, epochs = 50 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Hyperparameter settings for the six machine learning models used.  
Sensitivity (recall or true positive rate, TPR)--Measures the model's ability to correctly identify positive cases:  
\[\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{3}\]  
Specificity (true negative rate, TNR)--Measures the model's ability to correctly identify negative cases:  
\[\text{Specificity}=\ \frac{\text{TN}}{\text{TN}+\text{FP}} \tag{4}\]  
Positive predictive value (PPV)--Indicates the proportion of predicted positive cases that are actually positive:  
\[\text{PPV}=\ \frac{\text{TP}}{\text{TP}+\text{FP}} \tag{5}\]  
Negative predictive value (NPV)--Indicates the proportion of predicted negative cases that are actually negative:  
\[\text{NPV}=\ \frac{\text{TN}}{\text{TN}+\text{FN}} \tag{6}\]  
The area under the ROC curve (AUC-ROC) is a key metric for evaluating the discriminative performance of classification models. Values range from 0.5 (no discrimination) to 1.0 (perfect discrimination), with thresholds commonly interpreted as excellent (0.90-1.00), good (0.80-0.90), fair (0.70-0.80), poor (0.60-0.70), and failing (!0.60). AUC is especially informative in cases of class imbalance or varying misclassification costs.  
In sports analytics, particularly baseball prediction models, a higher AUC indicates stronger predictive accuracy in distinguishing between wins and losses. Models with AUC near 0.5 lack discriminative power and require further refinement. The best-performing model was further analyzed using SHAP values and feature importance plots to enhance interpretability. Statistical analyses were conducted using Python to identify key performance indicators that influence game outcomes, offering strategic insights to improve competitive performance.