Before revision: Bidirectional Encoder Representations from Transformers (BERT)models for medical specialties , such as BioBERT and clinicalBERT, have significantly improved in performing biomedical text mining tasks and have enabled extracting valuable information from biomedical literature ;
After revision: Pre-training large-scale neural language models on raw texts has made a significant contribution to improving transfer learning in natural language processing (NLP). With the introduction of transformer-based language models , such as BioBERT and clinicalBERT, have significantly improved in performing biomedical text mining tasks and have enabled extracting valuable information from biomedical literature ;