Before revision:  This proposed technique is an effective approach to develop localized medical BERT models and to enhance domain-specific models in the biomedical domain .
After revision:  Well-balanced pre-training by up-sampling instances derived from a corpus appropriate for the target task allows us to construct a high-performance BERT model .