## 3 Results and Discussion  
In Table 2, we present the best scores achieved on the test set, along with their respective standard deviations (mean \(\pm\) std). These scores are calculated based on all seeds used for a given configuration. Furthermore, we provide the best learning rates (LRs) and number of epochs for each model. Since averaging the epochs or the learning rate would not be meaningful, the reported LR represents the best result obtained by averaging the performances of the five repetitions. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **Micro** & \multicolumn{3}{c}{**Macro**} & \multicolumn{3}{c}{**Parameters**} \\  & **F1** & **P** & **R** & **F1** & **LR** & **Epoch** & **Task** \\ \hline gpt-2 & 0.776 \(\pm\) 0.033 & 0.582 \(\pm\) 0.031 & 0.426 \(\pm\) 0.030 & 0.462 \(\pm\) 0.033 & 8 \(\times\) 10\({}^{-5}\) & 12 & TC \\ bert [27] & 0.768 \(\pm\) 0.020 & 0.582 \(\pm\) 0.044 & 0.488 \(\pm\) 0.029 & 0.494 \(\pm\) 0.028 & 5 \(\times\) 10\({}^{-5}\) & 12 & MLM \\ roberta [28] & 0.810 \(\pm\) 0.059 & **0.623 \(\pm\) 0.051** & 0.543 \(\pm\) 0.041 & 0.541 \(\pm\) 0.059 & 5 \(\times\) 10\({}^{-5}\) & 13 & MLM \\ bio-es [30] & 0.822 \(\pm\) 0.062 & 0.588 \(\pm\) 0.041 & **0.565 \(\pm\) 0.070** & **0.552 \(\pm\) 0.027** & 8 \(\times\) 10\({}^{-5}\) & 13 & MLM \\ bio-ehr-es [30] & **0.823 \(\pm\) 0.034** & 0.570 \(\pm\) 0.035 & 0.531 \(\pm\) 0.048 & 0.526 \(\pm\) 0.034 & 5 \(\times\) 10\({}^{-5}\) & 13 & MLM \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performances of different models (mean \(\pm\) std) for NER task in our dataset constructed from maternal EHRs, where P, R, and F1 represent precision, recall, and F1 score measures, while LR, TC, and MLM indicate learning rate, token classification, and mask language models. Bold values identify the highest results per column. To determine the best performances, we rank them using the macro F1 score. In most cases, the best macro F1 score results align with the other evaluation measures. The highest performers for each measure are highlighted in bold. Table 2 illustrates that the bio-ehr-es model achieves the highest micro F1 score, while the bio-es model performs best regarding macro F1 score. To delve deeper into the comparison of these models, Tables 3 and 4 present the recall (R), precision (P), and F1 score (F1) measures for each class, considering both exact and partial boundaries. Similarly to Table 2, the reported values represent the mean and standard deviation obtained from five repetitions using five different seeds. When comparing the bio-ehr-es and bio-es models, we observe that bio-es demonstrates higher sensitivity, while bio-ehr-es exhibits greater precision overall. For instance, regarding recall with partial boundaries, bio-es outperforms bio-ehr-es in seven out of eight classes, with the only exception being class T081, corresponding to quantitative concepts. Conversely, for precision with partial boundaries, bio-ehr-es surpasses bio-es in seven out of eight classes, with the exception of class T079, representing temporal concepts. Moreover, bio-es performs better in six out of eight classes in terms of F1 score. It should be noted that the class-level comparisons for partial and exact boundaries are consistent, meaning that the classes with the best performance for partial boundaries align with those for exact boundaries. In general, Tables 3 and 4 demonstrate that classes T184 (signs and symptoms) and C151844 (negations) exhibit superior performance in both the bsc-bio-ehr-es and bsc-bio-es models; the top performers for each measure are highlighted in bold type. Overall, the