Before revision:  We also show that the "good" subnetworks vary considerably across GLUE tasks, opening up the possibilities to learn what knowledgeBERT actually uses at inference time .
After revision:  We also study the "good" subnetworks vary considerably across GLUE tasks, opening up the possibilities to learn what knowledgeBERT actually uses at inference time .