text: 
  
  TITLE
  Request: New LM Adapted checkpoints for T5
  
  COMMENTS
  0
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🌟 New LM Adapted checkpoints for T5
  
  ## Description
  
  Google released a new set of checkpoints for T5 v1.1. here:
  https://github.com/google-research/text-to-text-transfer-transformer/blob/master/released_checkpoints.md#t511
  
  Especially interesting for most people will be the checkpoints below, as finetuning T5 with a prompt or using T5 for conditional text generation is quite common and these checkpoints promise better performance. The default T5 v1.1 checkpoints have never seen sequences without sentinel tokens.
  
  ### LM-Adapted: t5.1.1.lm100k (copied from the readme)
  
  These "LM adapted" models are initialized from t5.1.1 (above) and train for an
  additional 100K steps on the LM objective discussed in the [T5 paper][paper].
  This adaptation improves the ability of the model to be used for [prompt
  tuning](https://arxiv.org/abs/2104.08691).
  
  * **t5.1.1.lm100k.small** (~77 million parameters): [gs://t5-data/pretrained_models/t5.1.1.lm100k.small](https://console.cloud.google.com/storage/browser/t5-data/pretrained_models/t5.1.1.lm100k.small/)
  * **t5.1.1.lm100k.base** (~250 million parameters): [gs://t5-data/pretrained_models/t5.1.1.lm100k.base](https://console.cloud.google.com/storage/browser/t5-data/pretrained_models/t5.1.1.lm100k.base/)
  * **t5.1.1.lm100k.large** (~800 million parameters): [gs://t5-data/pretrained_models/t5.1.1.lm100k.large](https://console.cloud.google.com/storage/browser/t5-data/pretrained_models/t5.1.1.lm100k.large/)
  * **t5.1.1.lm100k.xl** (~3 billion parameters): [gs://t5-data/pretrained_models/t5.1.1.lm100k.xl](https://console.cloud.google.com/storage/browser/t5-data/pretrained_models/t5.1.1.lm100k.xl/)
  * **t5.1.1.lm100k.xxl** (~11 billion parameters): [gs://t5-data/pretrained_models/t5.1.1.lm100k.xxl](https://console.cloud.google.com/storage/browser/t5-data/pretrained_models/t5.1.1.lm100k.xxl/)
  
  ## Open source status
  
  * [x] the model implementation is available: t5 v1.1. with geglu
  * [x] the model weights are available: see links above
  * [x] who are the authors: Brian Lester, Rami Al-Rfou, Noah Constant
  