Before revision:  Experiments on  machine translation and language modeling tasks show that DeLighT matches  the performance of baseline Transformers with significantly fewer parameters.
After revision:  Experiments on machine translation and language modeling tasks show that DeLighT matches or improves the performance of baseline Transformers with significantly fewer parameters.