Question: What embeddings are used?
Answer:  simple lookup table embeddings learned from scratch, using high-performance contextual embeddings, which are ELMo BIBREF11, BERT BIBREF16 and ClinicalBERT BIBREF13
Alternative answer: ELMO BIBREF11, BERT BIBREF12 and ClinicalBERT BIBREF13