## 1 Introduction  
The medical literature is expanding at a rapid pace, as evidenced by the more than two million scientific papers published worldwide since the onset of the pandemic ([[https://reports.dimensions.ai/covid-19/](https://reports.dimensions.ai/covid-19/)]([https://reports.dimensions.ai/covid-19/](https://reports.dimensions.ai/covid-19/)), accessed on 4 July 2024) in the field of research related to the SARS-CoV-2 virus. Manually extracting pertinent information about medical advancements, such as treatments, diseases, or drugs, from the vast number of publications is a laborious task that requires significant time and human resources. In this context, the necessity for developing computational methods or tools that facilitate the rapid and efficient extraction of information from unstructured or semi-structured medical documents is becoming increasingly paramount.  
Several tools facilitate the analysis and processing of information contained in text in this format. These include technologies such as Word2 Vec [1], which generates vector representations of text for subsequent computational processing, and Long Short-Term Memory (LSTM) neural networks [2]. The latter are highly effective for tasks involving large amounts of text; however, their sequential processing makes them a relatively slow method. Nevertheless, with the introduction of the Transformer architecture [3], text processing has been revolutionized, allowing a more significant number of texts to be processed in less time due to its attention mechanism. This mechanism allows information to be processed in parallel, resulting in a significant reduction in processing time.  
The introduction of the Transformer architecture has facilitated the development of large language models (LLMs) based on deep learning [4], such as Bard, LLaMA, Bloom, BERT, and GPT. These models have provided the scientific community with indispensable tools for processing large amounts of text and solving complex natural language processing (NLP) tasks. Some of their most prominent applications include information extraction (IE), named entity recognition (NER), sentiment analysis, text classification, summary generation, part-of-speech (POS) tagging, translation, and question answering (QA). Nevertheless, most of these LLMs have been trained in English and on general knowledge.  
This paper examines the feasibility of adapting an LLM to a specific domain despite being pre-trained on general texts such as English Wikipedia and Google Books. The proposal is to utilize the BERT model base [5], which was pre-trained in English, and adapt it to Spanish, one of the world's five most spoken languages [6]. The objective is to specialize the model in Spanish medical terminology and expressions to perform NLP tasks, such as answering extractive questions formulated by medical researchers in Spanish, thus enabling it to extract information from the medical literature. In order to adapt the large language model to medical terminology, more than 3 million Spanish texts related to the medical domain were used. Furthermore, fine calibration of the hyperparameters was performed to optimize the model's training and enable it to efficiently answer factoid extractive questions in the Spanish medical domain. This task involved fine-tuning using more than 34 thousand medical questions. This paper presents as its main contribution MedicoBERT, a large language model adapted to the medical domain in Spanish. This model supports various research applications in natural language processing within this domain.  
The rest of this paper is organized as follows. In Section 2, we briefly review related works on pre-trained large language models. In Section 3, we describe the research development by describing the datasets used, the model training, the evaluation, and the calibration of the hyperparameters. Section 4 presents the results obtained, and Section 5 analyzes and discusses these results. Finally, Section 6 presents the conclusions and suggestions for future work.