Question: What in-domain text did they use?
Answer: PubMed+PMC
Alternative answer: PubMed+PMC (the data used for BioBERTv1.0) and/or CORD-19 (Covid-19 Open Research Dataset)