Frankenmodels
Collection
They're not supposed to be that size! Neat, right?
•
8 items
•
Updated
•
3
This is Llama 2 13b with some additional attention heads from original-flavor Llama 33b frankensteined on.
Fine-tuned on ~10M tokens from RedPajama to settle in the transplants a little.
Not intended for use as-is - this model is meant to serve as a base for further tuning, hopefully with a greater capacity for learning than 13b.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 46.85 |
| ARC (25-shot) | 58.53 |
| HellaSwag (10-shot) | 82.55 |
| MMLU (5-shot) | 54.68 |
| TruthfulQA (0-shot) | 39.84 |
| Winogrande (5-shot) | 76.32 |
| GSM8K (5-shot) | 9.93 |
| DROP (3-shot) | 6.08 |
- Downloads last month
- 1,093
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.