Update README.md

README.md

@@ -56,7 +56,84 @@ _CC-By-NC-SA-4.0_ (non-commercial use only)
 > This new version of MPT-7B is truly impressive and I look forward to seeing what innovative applications developers will create using these powerful tools.
 > Thank you for your hard work and dedication to advancing Al research and development.
 
+## How to Use
+
+This model is best used with the MosaicML [llm-foundry repository](https://github.com/mosaicml/llm-foundry) for training and finetuning.
+
+```python
+import transformers
+model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-7b-chat', trust_remote_code=True)
+```
+Note: This model requires that `trust_remote_code=True` be passed to the `from_pretrained` method. 
+This is because we use a custom `MPT` model architecture that is not yet part of the Hugging Face `transformers` package.
+`MPT` includes options for many training efficiency features such as [FlashAttention](https://arxiv.org/pdf/2205.14135.pdf), [ALiBi](https://arxiv.org/abs/2108.12409), [QK LayerNorm](https://arxiv.org/abs/2010.04245), and more.
+
+To use the optimized [triton implementation](https://github.com/openai/triton) of FlashAttention, you can load the model with `attn_impl='triton'` and move the model to `bfloat16`:
+```python
+config = transformers.AutoConfig.from_pretrained('mosaicml/mpt-7b-chat', trust_remote_code=True)
+config.attn_config['attn_impl'] = 'triton'
+
+model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-7b-chat', config=config, torch_dtype=torch.bfloat16, trust_remote_code=True)
+model.to(device='cuda:0')
+```
+
+Although the model was trained with a sequence length of 2048, ALiBi enables users to increase the maximum sequence length during finetuning and/or inference. For example:
+
+```python
+config = transformers.AutoConfig.from_pretrained('mosaicml/mpt-7b-chat', trust_remote_code=True)
+config.update({"max_seq_len": 4096})
+model = transformers.AutoModelForCausalLM.from_pretrained('mosaicml/mpt-7b-chat', config=config, trust_remote_code=True)
+```
+
+This model was trained with the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
+
+```python
+from transformers import AutoTokenizer
+tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
+
+## Model Description
+
+The architecture is a modification of a standard decoder-only transformer.
+
+The model has been modified from a standard transformer in the following ways:
+* It uses [FlashAttention](https://arxiv.org/pdf/2205.14135.pdf)
+* It uses [ALiBi (Attention with Linear Biases)](https://arxiv.org/abs/2108.12409) and does not use positional embeddings
+* It does not use biases
+
+
+| Hyperparameter | Value |
+|----------------|-------|
+|n_parameters | 6.7B |
+|n_layers | 32 |
+| n_heads | 32 |
+| d_model | 4096 |
+| vocab size | 50432 |
+| sequence length | 2048 |
+
+## Limitations and Biases
+
+_The following language is modified from [EleutherAI's GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b)_
+
+MPT-7B-Chat can produce factually incorrect output, and should not be relied on to produce factually accurate information.
+MPT-7B-CHat was trained on various public datasets.
+While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
 
 ## Acknowledgements
 
-This model was finetuned by Sam Havens and the MosaicML NLP team
+This model was finetuned by Sam Havens and the MosaicML NLP team
+
+## Citation
+
+Please cite this model using the following format:
+
+```
+@online{MosaicML2023Introducing,
+    author    = {MosaicML NLP Team},
+    title     = {Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs},
+    year      = {2023},
+    url       = {www.mosaicml.com/blog/mpt-7b},
+    note      = {Accessed: 2023-03-28}, % change this date
+    urldate   = {2023-03-28} % change this date
+}
+```
+