Before revision:  Through attention pruning, we find that about 90\% of the attention computation can be reduced for language modelling and about 50\% for machine translation and \%natural language inference prediction with BERT on GLUE tasks, while maintaining the quality of the results.
After revision:  Through attention pruning, we find that about 90\% of the attention computation for language modelling and about 50\% for machine translation and \%natural language inference prediction with BERT on GLUE tasks, while maintaining the quality of the results.