Before revision:  This work considers distillation of BART and Pegasus , two state of the art  summarization models, on two datasets across a variety of student models.
After revision:  This work considers distillation of BART and Pegasus , two state of the art , pre-trained summarization models, on two datasets across a variety of student models.