## 7 Conclusions  
The significance of this research lies in several aspects. First, we presented the intent recognition results obtained from an exploratory analysis in which different deep learning (DL) models were evaluated on the Hispabot-Covid19 dataset. Particularly, we compared traditional recurrent neural networks (RNN) and state-of-the-art transformer models including standard bidirectional encoder representations from transformers (BERT) [18] and its BETO variant [47]. Our experiments demonstrate that transformer models yield best performance. Specifically, this result was achieved by simply fine-tuning the BETO model for the intent detection task.  
Second, we proposed and validated a simple but effective method for extending our pretrained BERT models with task- or domain-specific vocabulary. The method specifically accounts for term frequencies to rank and select specific words to be added at the word level. The performance of the extended model was significantly improved. This approach could be particularly attractive to ad-hoc and special purposes, or very specific domains with unique vocabularies where limited training data are available. In spite of the promising results and applications of our proposed method, a number of issues require further research. Particularly, additional research should be performed toward suggesting solutions for automatically finding or identifying the exact and optimal amount of new tokens to be added (i.e., the precise value that achieves a good balance between model complexity and available training data).  
Third, when using transformer-like models and testing different text preprocessing methods, preserving the raw structure of the texts by not removing anything while simply performing a very basic text normalization helps to achieve a better performance. This result allowed a better understanding of the actual importance and convenience of applying (or not) those popular text preprocessing techniques when using DL and/or embedding representations compared to that of traditional approaches based on different foundations.  
Conceptualization, F.F.-M., C.L.-J., R.K., D.G., Z.C. and J.M.M.; data curation, F.F.-M., D.G. and Z.C.; formal analysis, F.F.-M., C.L.-J., R.K., D.G., Z.C. and J.M.M.; funding acquisition, F.F.-M., D.G., Z.C. and J.M.M.; investigation, F.F.-M., C.L.-J., R.K., D.G., Z.C. and J.M.M.; methodology, F.F.-M., C.L.-J., R.K., D.G., Z.C. and J.M.M.; project administration, F.F.-M., D.G., Z.C. and J.M.M.; resources, F.F.-M., D.G., Z.C. and J.M.M.M.; software, F.F.-M. and C.L.-J.; supervision, F.F.-M., D.G., Z.C. and J.M.M.M.; validation, F.F.-M. and C.L.-J.; visualization, F.F.-M., C.L.-J., R.K. and J.M.M.M.; writing--original draft, [PERSON]-M.; writing--review & editing, [PERSON]-M., [PERSON], R.K., D.G., Z.C. and J.M.M. All authors have read and agreed to the published version of the manuscript.  
The work leading to these results was supported by the Spanish Ministry of Science and Innovation through the R&D&ij projects GOMINOLA (PID2020-118112 RB-C21 and PID2020-118112 RB-C22, funded by MCIN/AEI/10.13039/501100011033), CAVIAR (TEC2017-84593-C2-1-R, funded by MCIN/ AEI/10.13039/501100011033/FEDER \"Una manera de hacer Europa\"), and AMIC-PoC (PDC2021-120846-C42, funded by MCIN/AEI/10.13039/50110001033 and by \"the European Union \"NextGenerationEU/PRTR\"). This research also received funding from the European Union's Horizon2020 research and innovation program under grant agreement N\({}^{\circ}\) 823907 ([[http://menhir-project.eu](http://menhir-project.eu)]([http://menhir-project.eu](http://menhir-project.eu)), accessed on 2 February 2022). Furthermore, R.K.'s research was supported by the Spanish Ministry of Education (FPI grant PRE2018-083225).  
Restrictions apply to the availability of these data. Dataset is available from the authors with the permission of the Spanish Secretary of State for Digitalization and Artificial Intelligence (SEDIA).  
The authors declare no conflict of interest.  
References