Before revision:  We show that compared to the prior inference time reduction methods, PoWER-BERT offers  better trade-off between accuracy and inference time . Lastly, we demonstrate that our scheme can also be used in conjunction with ALBERT (a highly compressed version of BERT) and can attain up to 6.8x factor reduction in inference time with <1\% loss in accuracy  .
After revision:  We show that compared to the prior inference time reduction methods, PoWER-BERT offers better trade-off between accuracy and inference time compared to prior methods. We demonstrate that our scheme can also be used in conjunction with ALBERT (a highly compressed version of BERT) and can attain up to 6.8x factor reduction in inference time with <1\% loss in accuracy .