File size: 510 Bytes
0a07df8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
---
model_type: llama
pipeline_tag: text-generation
datasets:
- togethercomputer/RedPajama-Data-1T-Sample
tags:
- llama
---
This is [Llama 2 13b](https://huggingface.co/meta-llama/Llama-2-13b-hf) with some additional attention heads from original-flavor Llama 33b frankensteined on.
Fine-tuned on ~10M tokens from RedPajama to settle in the transplants a little.
Not intended for use as-is - this model is meant to serve as a base for further tuning, hopefully with a greater capacity for learning than 13b. |