Before revision:  We  also present the SoftNER model that combines contextual information with domain specific knowledge using an attention network.
After revision:  We trained in-domain BERT representations (BERTOverflow) on 152 million sentences from StackOverflow, which lead to an absolute increase of +10 F-1 score over off-the-shelf BERT. We also present the SoftNER model that combines contextual information with domain specific knowledge using an attention network.