Create README.md

README.md

@@ -0,0 +1,14 @@
+---
+model_type: llama
+pipeline_tag: text-generation
+datasets:
+- togethercomputer/RedPajama-Data-1T-Sample
+tags:
+- llama
+---
+
+This is [Llama 2 13b](https://huggingface.co/meta-llama/Llama-2-13b-hf) with some additional attention heads from original-flavor Llama 33b frankensteined on.
+
+Fine-tuned on ~10M tokens from RedPajama to settle in the transplants a little.
+
+Not intended for use as-is - this model is meant to serve as a base for further tuning, hopefully with a greater capacity for learning than 13b.