Create README.md

README.md

@@ -0,0 +1,7 @@
+---
+datasets:
+- c4
+language:
+- en
+---
+CoreX models are Llama models in which the first X decoder layers are kept, and then the model is finetuned on 1 billion tokens from some dataset. Base model stems from Llama2-7b, medium from Llama2-13b, xl from Llama2-70b.