Question: What evidence do they present that the model attends to shallow context clues?
Answer: Using model gradients with respect to input features they presented that the most important model inputs are verbs associated with entities which shows that the model attends to shallow context clues