Update README.md

README.md

@@ -11,16 +11,18 @@ inference: false
 
 MosaicBERT-Base is a new BERT architecture and training recipe optimized for fast pretraining. 
 MosaicBERT trains faster and achieves higher pretraining and finetuning accuracy when benchmarked against 
-Hugging Face's [bert-base-uncased](https://huggingface.co/bert-base-uncased).
+Hugging Face's [bert-base-uncased](https://huggingface.co/bert-base-uncased). It incorporates efficiency insights 
+from the past half a decade of transformers research, from RoBERTa to T5 and GPT.
 
-__This model was trained with [ALiBi](https://arxiv.org/abs/2108.12409) on a sequence length of 512 tokens.__
+__This particular model was trained with [ALiBi](https://arxiv.org/abs/2108.12409) on a sequence length of 512 tokens.__
 
 ALiBi allows a model trained with a sequence length n to easily extrapolate to sequence lengths >2n during finetuning. For more details, see [Train Short, Test Long: Attention with Linear
 Biases Enables Input Length Extrapolation (Press et al. 2022)](https://arxiv.org/abs/2108.12409)
 
-It is part of the **family of MosaicBERT-Base models**:
+It is part of the **family of MosaicBERT-Base models** trained using ALiBi on different sequence lengths:
 
 * [mosaic-bert-base](https://huggingface.co/mosaicml/mosaic-bert-base) (trained on a sequence length of 128 tokens)
+* [mosaic-bert-base-seqlen-256](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-256)
 * mosaic-bert-base-seqlen-512
 * [mosaic-bert-base-seqlen-1024](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-1024) 
 * [mosaic-bert-base-seqlen-2048](https://huggingface.co/mosaicml/mosaic-bert-base-seqlen-2048)