## 5 Discussion  
The results of the MedicoBERT model generation are divided into two categories: model fitting through adaptive learning and fine-tuning to answer medical questions in Spanish. The latter category includes searching for the best hyperparameters through optimization methods. The adaptive learning results showed that training domain-specific tokenizers ensures an accurate transformation of text into numerical values. In addition, better recognition of specialized medical vocabulary and terminology was observed, allowing for more practical information extraction and, consequently, deeper and more meaningful analysis of medical literature. The pre-training of the base model on the medical literature has excellent potential for improving the performance and applicability of LLMs in specific domains. This was demonstrated by achieving a low perplexity value of 4.28, indicating a successful adaptation of the model to the medical domain in Spanish. It was also observed that the performance  
Figure 5: Calibration strategies for the base BERT and MedicoBERT models in the medical QA task. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Epoch** & **Batch** & **Learning Rate** & **Weight** & **Average F1** \\ \hline \(3.25\) & \(8\) & \(2\times 10^{-5}\) & \(0.015\) & \(61.72\) \\ \hline \hline \end{tabular}
\end{table} 
Table 8: MedicoBERT training results for the QA task with harmonic search hyperparameters. of language models in a given domain improves as the number of texts and words available in that domain increases. In the context of the fine-tuning results, our MedicoBERT model represents a significant advancement in adapting large language models to the medical domain in Spanish. In contrast to previous studies, our model is designed to answer medical questions. In contrast to the language models presented in [7; 8; 10; 11; 12; 13; 14], which focus on the medical or clinical domain in English, our model offers the advantage of being in Spanish, which makes it more accessible and valuable for Spanish-speaking health professionals. Similarly, it differs from other models in Spanish, such as [13; 16; 18; 19], which are tailored to non-specialized or general domains, as our model focuses specifically on the medical domain. It is noteworthy that although the authors of [9] presented a model adapted to the medical domain in Spanish, it was designed to solve the named entity recognition (NER) task. In contrast, our model addresses one of the most complex NLP tasks--question answering (QA)--where it achieved promising results, correctly answering 62% of questions. As evidenced by the literature, there is growing interest in large language models within the scientific community. However, studies focusing on Spanish and the medical domain must be expanded. This paper contributes to developing large language models explicitly designed for Spanish in the medical domain. The extant literature on Transformer-based models for the medical domain in Spanish has predominantly concentrated on natural language processing tasks, including entity recognition, sentence labeling, and classification. In contrast, our model is designed to provide accurate responses to questions. Although language models capable of answering questions in the medical domain have been developed in English, a research gap persists in Spanish. This study addresses this gap by proposing a model trained on a general question dataset (SQUAD) and a medical corpus designed and validated by domain experts (BioAsq). This combination enables the model to comprehend the general structure of questions and the nuances of medical language, facilitating more precise and pertinent responses. One of the key strengths of MedicoBERT is its comprehensive training on CoWeSe, the most extensive Spanish medical corpus currently available. This extensive training has enabled MedicoBERT to develop a deep semantic understanding of medical language, equipping it with a profound knowledge of diseases, treatments, and symptoms.