Text Generation
Transformers
PyTorch
mpt
Composer
MosaicML
llm-foundry
custom_code
text-generation-inference
jacobfulano commited on
Commit
8c92147
1 Parent(s): c271818

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -4
README.md CHANGED
@@ -11,7 +11,7 @@ tags:
11
  # MPT-7B-Instruct
12
 
13
  MPT-7B-Instruct is a model for short-form instruction following.
14
- It is built by finetuning [MPT-7B (Base)](https://huggingface.co/spaces/mosaicml/mpt-7b) on a [dataset](https://huggingface.co/datasets/sam-mosaic/dolly_hhrlhf) derived from the [Databricks Dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and the [Anthropic Helpful and Harmless (HH-RLHF)](https://huggingface.co/datasets/Anthropic/hh-rlhf) datasets.
15
  * License: _CC-By-SA-3.0_ (commercial use permitted)
16
  * [Online Demo](https://huggingface.co/spaces/mosaicml/mpt-7b-instruct)
17
 
@@ -99,10 +99,30 @@ For more details on the pretraining process, see [MPT-7B](https://huggingface.co
99
 
100
  The data was tokenized using the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
101
 
102
- ## Training Configuration
 
 
 
 
 
 
103
 
104
- This model was finetuned on 440 A100-40GBs for about half a day using the [MosaicML Platform](https://www.mosaicml.com/platform).
105
 
106
  ## Acknowledgements
107
 
108
- This model was finetuned by Sam Havens and the MosaicML NLP team
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  # MPT-7B-Instruct
12
 
13
  MPT-7B-Instruct is a model for short-form instruction following.
14
+ It is built by finetuning [MPT-7B](https://huggingface.co/spaces/mosaicml/mpt-7b) on a [dataset](https://huggingface.co/datasets/sam-mosaic/dolly_hhrlhf) derived from the [Databricks Dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) and the [Anthropic Helpful and Harmless (HH-RLHF)](https://huggingface.co/datasets/Anthropic/hh-rlhf) datasets.
15
  * License: _CC-By-SA-3.0_ (commercial use permitted)
16
  * [Online Demo](https://huggingface.co/spaces/mosaicml/mpt-7b-instruct)
17
 
 
99
 
100
  The data was tokenized using the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) tokenizer.
101
 
102
+ ## Limitations and Biases
103
+
104
+ _The following language is modified from [EleutherAI's GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b)_
105
+
106
+ MPT-7B-Chat can produce factually incorrect output, and should not be relied on to produce factually accurate information.
107
+ MPT-7B-CHat was trained on various public datasets.
108
+ While great efforts have been taken to clean the pretraining data, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
109
 
 
110
 
111
  ## Acknowledgements
112
 
113
+ This model was finetuned by Sam Havens and the MosaicML NLP team
114
+
115
+ ## Citation
116
+
117
+ Please cite this model using the following format:
118
+
119
+ ```
120
+ @online{MosaicML2023Introducing,
121
+ author = {MosaicML NLP Team},
122
+ title = {Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs},
123
+ year = {2023},
124
+ url = {www.mosaicml.com/blog/mpt-7b},
125
+ note = {Accessed: 2023-03-28}, % change this date
126
+ urldate = {2023-03-28} % change this date
127
+ }
128
+ ```