## 2 Related Works
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**LLM/Reference** & **Language** & **Dataset** & **Domain** & **Tasks** & **Evaluation for Each Task** \\ \hline BioBERT [7] & English & Abstracts & & - REL & - F1 86.51\% \\  & English & PubMed, PMC & Medical & - NER & - F1 93.47\% \\  & & articles & & - QA & - F1 60.0\% \\ \hline ClinicalBERT [8] & English & MIMIC-III & Clinical/Medical & - Predict hospital readmission & - AUROC 71.4\% \\ \hline RobertaClinical [9] & Spanish & \begin{tabular}{c} Medical crawler, \\ Clinical cases, \\ mespan\_Medline, \\ PubMed \\ \end{tabular} & Clinical/Medical & - NER & - F1 88.21\% \\ \hline Med-BERT [10] & English and Chinese & Clinical Records & Clinical & - NER & - F1 93\% \\ \hline BlueBERT [11] & English & PubMed & \begin{tabular}{c} - Sentence similarity \\ - REL \\ - CLS \\ - NER \\ - NLI \\ \end{tabular} & - Pearson 84.8\% \\  & English & \begin{tabular}{c} PubMed \\ MIMIC-III \\ - NLI \\ \end{tabular} & Biomedical & - REL & - F1 74.4\% \\  & & & & - CLS & - F1 87.3\% \\  & & & & - NER & - F1 86.6\% \\  & & & & - NLI & - F1 84.9\% \\ \hline PubMedBERT [12] & English & PubMed & Biomedical & - NER & - NER & - NER \\  & & & & - REL & - macro-F1 77.24\% \\  & & & & - CLS & - macro-F1 82.32\% \\  & & & & - QA & - macro-F1 55.84\% \\ \hline SciBERT [13] & English & Semantic Scholar & \begin{tabular}{c} Computing, \\ Biomedical \\ \end{tabular} & \begin{tabular}{c} - NER \\ - CLS \\ - REL \\ - NLI \\ - POS \\ - NER \\ - NER \\ \end{tabular} & - F1 67.57\% \\ \hline ELECTRICIDAD [16] & Spanish & OSCAR 20 & General & - CELS & - NER & - F1 76.75\% \\  & & & & - CELS & - F1 70.98\% \\ \hline ROBERTA [17] & Spanish & \begin{tabular}{c} Wikipedia, \\ Wikinews \\ \end{tabular} & General & \begin{tabular}{c} - NER \\ - POS \\ - POS \\ \end{tabular} & - NER & - Accuracy 98.97\% \\ \hline BETO [19] & Spanish & \begin{tabular}{c} Wikipedia \\ OPUS19 \\ \end{tabular} & General & \begin{tabular}{c} - NLI \\ - NER \\ - POS \\ \end{tabular} & - Accuracy 82.01\% \\ \hline [PERSON] [20] & Spanish & mC4 & General & \begin{tabular}{c} NER \\ - POS \\ \end{tabular} & - F1 87.79\% \\ \hline [PERSON] [21] & French & \begin{tabular}{c} Wikipedia, \\ Books, \\ Common Craw \\ \end{tabular} & General & \begin{tabular}{c} - CLS \\ - PAWS-X \\ - WSD \\ - \end{tabular} & - Accuracy 93.4\% \\ \hline [PERSON] [22] & Dutch & \begin{tabular}{c} OSCAR, \\ Common Crawl, \\ SoNaR-500 \\ \end{tabular} & General & \
\begin{tabular}{c} - Sentiment analysis \\ - Zero-shot \\ - NER \\ - NER \\ \end{tabular} & - Accuracy 98.75\% \\ \hline [PERSON] [23] & Italian & TWITA & Social networks & - TensorFlow detection & - F1 60.90\% \\  & & & & - Sentiment analysis & - F1 72.23\% \\  & & & & - Subjectivity & - F1 79.06\% \\ \hline [PERSON] [14] & English &
\begin{tabular}{c} Surgical records from \\ the Korea University \\ Guro Hospital \\ \end{tabular} & Clinical & - ICD-9 Classification & - F1 74.15\% \\  & & & & - AUC 98.42\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of LLMs reported in the literature. The construction of large language models based on the Transformer architecture has been shown to facilitate the resolution of a range of NLP tasks. Nevertheless, to obtain adequate and dependable outcomes, it is essential to tailor the models to the language and the particular domain in which the tasks are to be addressed. This paper presents MedicoBERT, a language model capable of answering medical questions in Spanish. The model was trained using a hyperparameter optimization technique. MedicoBERT aims to address the limitations or gaps of existing models in the Spanish medical domain. These models, pre-trained on general corpora, lack the specialized medical vocabulary and terminology necessary for optimal performance in tasks such as identifying medical entities or extracting answers from medical texts in Spanish. In contrast, MedicoBERT is pre-trained on a substantial corpus of scientific texts in Spanish, comprising over 3 M and 1.8 B words.