# Weakly-Supervised Multilingual Medical NER for Symptom Extraction for Low-Resource Languages
## 3 Results  
This section presents the evaluation results of the eight fine-tuned multilingual NER models, each trained to extract medical entities (PROBLEM, TEST, TREATMENT) in different languages. The performance of each model was analyzed using the standard metrics, including precision, recall, and F1 score.  
Table 6 provides a comparative overview of the performance metrics for the eight language-specific NER models. The English model demonstrated the highest overall performance, with a precision of 80.85%, recall of 79.30%, and F1 score of 80.07%, alongside the lowest evaluation loss at 0.24. This suggests that the English model is both accurate and consistent in identifying entities, likely due to the abundance of high-quality English training data. The German and Spanish models also showed strong results, with F1 scores of 78.70% and 77.61%, respectively, and evaluation losses below 0.35, indicating reliable performance in these languages. Portuguese followed closely with an F1 score of 77.21%, showing its capability in medical NER despite slight variations in recall and precision.  
In contrast, the Greek model exhibited the lowest F1 score at 69.10% and a higher evaluation loss of 0.41, suggesting greater challenges in recognizing entities in Greek accurately. The models for Italian, Polish, and Slovenian fell within a mid-range performance level, with F1 scores between 75.56% and 75.72% and evaluation losses ranging from 0.34  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Language** & **Precision** & **Recall** & **F1 Score** & **Loss** & **BLEU Score** & **Runtime** \\ \hline En & 80.85\% & 79.30\% & **80.07\%** & 0.24 & N/A & 12 h 34 m \\ \hline De & 78.94\% & 78.46\% & **78.70\%** & 0.30 & 47.5 & 16 h 7 m \\ \hline El & 70.69\% & 67.58\% & **69.10\%** & 0.41 & 56.4 & 17 h 50 m \\ \hline Es & 77.14\% & 78.08\% & **77.61\%** & 0.33 & 54.9 & 14 h 23 m \\ \hline It & 75.91\% & 75.28\% & **75.60\%** & 0.34 & 53.9 & 14 h 57 m \\ \hline Pl & 75.52\% & 75.60\% & **75.56\%** & 0.40 & 47.5 & 23 h 17 m \\ \hline Pt & 77.25\% & 77.16\% & **77.21\%** & 0.34 & 50.4 & 16 h 35 m \\ \hline Sl & 75.78\% & 75.66\% & **75.72\%** & 0.37 & 33.9 & 14 h 17 m \\ \hline \hline \end{tabular}  
* Notes: English (En); German (De); Greek (El); Spanish (Es); Italian (It); Polish (Pl); Portuguese (Pt); Slovenian (SI); hours (h); minutes (m).  
\end{table}
Table 6: Overview of the models’ performances.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Metric** & **Formula** \\ \hline Precision (P) & P = TP/(TP + FP) \\ \hline Recall (R) & R = TP/(TP + FN) \\ \hline F1 Score (F1) & F1 = 2 \(\times\) (P \(\times\) R)/(P + R) \\ \hline \hline \end{tabular}  
* Notes: TP—True Positives; FP—False Positives; FN—False Negatives.  
\end{table}
Table 5: Evaluation metrics.  
to 0.40. These results indicate that while the models performed reasonably well, there may be language-specific nuances or variations in translation quality that affect performance.  
The Translation BLEU score column represents the BLEU score for translations from English to each respective language, measuring the quality of the machine translation used to generate training data in different languages.