text: 
  
  TITLE
  New and better T5 checkpoints from scaling transformers paper
  
  COMMENTS
  24
  
  REACTIONS
  +1: 2
  -1: 0
  laugh: 0
  hooray: 0
  heart: 1
  rocket: 0
  eyes: 0
  
  BODY
  # 🌟 New model addition
  
  ## Model description
  This paper explores different ways of scaling T5:
  Scaling Efficiently: Insights from Pre-training and Finetuning Transformers
  https://arxiv.org/abs/2109.10686
  
  They found new checkpoints (DeepNarrow) that perform significantly better than the previous models on downstream applications.
  
  Here is a table from the paper that compares the old Base, Large, XL and XXL models to the new ones:
  
  <img width="551" alt="image" src="https://user-images.githubusercontent.com/37597043/152038969-de1fa56e-991d-493b-b94c-96cbc42d69be.png">
  
  The checkpoints were released today here:
  https://github.com/google-research/google-research/tree/master/scaling_transformers
  
  <!-- Important information -->
  
  ## Open source status
  
  * [x] the model implementation is available: the current T5 implementation in transformers
  * [x] the model weights are available: https://github.com/google-research/google-research/tree/master/scaling_transformers
  * [x] who are the authors: Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler
  