Update README.md

README.md

@@ -11,9 +11,16 @@ datasets:
 - togethercomputer/RedPajama-Data-1T
 - bigcode/the-stack
 - allenai/s2orc
+- lukasmoeller/sail_preprocessed
 inference: false
 ---
 
+# MPT-7B SAIL
+
+This is a fine-tuned variant of MPT-7B, trained on the SAIL dataset (https://arxiv.org/abs/2305.15225). The preprocessed version can be found here: https://huggingface.co/datasets/lukasmoeller/sail_preprocessed
+
+I may have forgotten to add EOD tokens at the end of the target, might retrain if anyone is interested.
+
 # MPT-7B
 
 MPT-7B is a decoder-style transformer pretrained from scratch on 1T tokens of English text and code.