Question: What state-of-the-art compression techniques were used in the comparison?
Answer: baseline without knowledge distillation (termed NoKD), Patient Knowledge Distillation (PKD)
Alternative answer: NoKD, PKD, BERTBASE teacher model