Before revision:  We provide evidence that this does not always need to be the case: with proper initialization and training techniques , the benefits of very deep transformers are shown to carry over to hard structural prediction tasks , even using small datasets .
After revision:  We provide evidence that this does not always need to be the case: with proper initialization and optimization , the benefits of very deep transformers are shown to carry over to hard structural prediction tasks , even using small datasets .