## 3 Results
### Classification Results and Model Evaluation  
An early stopping method was applied to prevent overfitting, particularly for the C1 group, which comprised 68.4% of the dataset, as the model was trained on unbalanced data. Training was halted when the AUROC score did not improve for five consecutive epochs. The training parameters for our model are detailed in Table S2. After fine-tuning the pre-trained model, the evaluation showed strong performance, with an AUROC of 0.91, an accuracy of 0.91, and an F1 score of 0.91 (Table 2). For the groups without LOC (C1 and C2), the recall rates were 94.0% and the precision rates were 90.7%, respectively. Although the group with LOC (C3) only represented 3.4% of the total data, the model predicted 148 patients with LOC out of 234 observed cases, achieving a recall and precision rate of 63.3% (Figure 2). The results of AUROC's 5-fold cross validation on the test dataset are shown in Figure S1. The weighted-loss method was applied to prevent the model's performance from disproportionately favoring the non-LOC group, which makes up a large portion of the dataset. With a conventional loss function, the AUROC result did not align with the model's actual loss under the same conditions (Figure 3a). By contrast, the weighted loss model showed higher evaluation metrics than the conventional loss model (Figure 3). Specifically, training with the weighted loss function led to a 31.9% improvement in AUROC (from 0.69 to 0.91), as well as increases of 2.3% in accuracy and 4.6% in F1 score relative to the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Variable** & **Number** & **\%** \\ \hline Gender & & \\ Male & 13,371 & 57.4 \\ Female & 9937 & 42.6 \\ Age, years & & \\
0–19 & 8114 & 34.8 \\
20–39 & 6497 & 27.9 \\
40–59 & 5470 & 23.5 \\
60–79 & 2469 & 10.6 \\ \(\geq\)80 & 758 & 3.3 \\ LOC 1 status & & \\ C1 2 & 15,775 & 67.7 \\ C2 3  & 6240 & 26.8 \\ C3 4  & 780 & 3.4 \\ C99 5  & 513 & 2.2 \\ Disposition at emergency department & & \\ Discharge (to home) & 21,038 & 90.3 \\ Admission & 1993 & 8.6 \\ Transfer & 101 & 0.4 \\ Death & 55 & 0.2 \\ NA 6  & 121 & 0.5 \\ Death & & \\ Yes & 119 & 0.5 \\ No & 23,189 & 99.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Clinical characteristics of the participants. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Input Data** & **AUROC** & **Accuracy** & **Precision** & **Recall** & **F1** & **Weighted F1** \\ \hline Pre-/Post-diagnosis note & 0.916 & 0.910 & 0.735 & 0.737 & 0.735 & 0.911 \\ Pre-diagnosis note & 0.906 & 0.904 & 0.724 & 0.723 & 0.718 & 0.905 \\ Post-diagnosis note & 0.746 & 0.818 & 0.695 & 0.428 & 0.447 & 0.797 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance metrics of loss of consciousness classification according to input dataset. unweighted model. Notably, there was no significant difference in performance between the two loss functions on the training dataset (Figure 3b). Next, we analyzed the contribution of different types of text used in the model. Models trained with only pre-diagnosis notes showed a slightly lower AUROC score than models incorporating both pre- and post-diagnosis notes, though the difference was minimal (Figure 3a). Furthermore, the early stopping points for models using only pre-diagnosis notes occurred sooner than for models using both pre- and post-diagnosis notes,  
Figure 3: Evaluation of the performance of LOC classification among NLP-based models using different clinical diagnosis notes. (**a**) Model performance evaluated in the test dataset. The left panel showed a difference in classification performance according to types of clinical notes. The performance improvement for unbalanced data using the weighted loss function is shown in the right panel. The results of the analysis performed on the training dataset are displayed in (**b**). Points in the scatter plot indicate last updated model performance according to the early stop method. Figure 2: Multiclass confusion matrix for test dataset.