Before revision:  We propose two attention-based methods to analyze commonsense knowledge inside BERT , and the contribution of such knowledge for the model prediction.
After revision:  We propose two attention-based methods to analyze commonsense knowledge inside BERT , and the importance of such cues for the model prediction.