Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,8 @@ library_name: transformers
|
|
| 5 |
license: apache-2.0
|
| 6 |
language:
|
| 7 |
- en
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# Model Card: Nous-Yarn-Llama-2-70b-32k
|
|
@@ -38,14 +40,14 @@ pip install git+https://github.com/huggingface/transformers
|
|
| 38 |
Long context benchmarks:
|
| 39 |
| Model | Context Window | 1k PPL | 2k PPL | 4k PPL | 8k PPL | 16k PPL | 32k PPL |
|
| 40 |
|-------|---------------:|-------:|--------:|------:|-------:|--------:|--------:|
|
| 41 |
-
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k |
|
| 42 |
-
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k |
|
| 43 |
|
| 44 |
Short context benchmarks showing that quality degradation is minimal:
|
| 45 |
| Model | Context Window | ARC-c | Hellaswag | MMLU | Truthful QA |
|
| 46 |
|-------|---------------:|------:|----------:|-----:|------------:|
|
| 47 |
-
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k |
|
| 48 |
-
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k |
|
| 49 |
|
| 50 |
## Collaborators
|
| 51 |
|
|
|
|
| 5 |
license: apache-2.0
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
+
datasets:
|
| 9 |
+
- emozilla/yarn-train-tokenized-8k-llama
|
| 10 |
---
|
| 11 |
|
| 12 |
# Model Card: Nous-Yarn-Llama-2-70b-32k
|
|
|
|
| 40 |
Long context benchmarks:
|
| 41 |
| Model | Context Window | 1k PPL | 2k PPL | 4k PPL | 8k PPL | 16k PPL | 32k PPL |
|
| 42 |
|-------|---------------:|-------:|--------:|------:|-------:|--------:|--------:|
|
| 43 |
+
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k | 3.71 | 3.27 | 2.96 | - | - | - |
|
| 44 |
+
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k | 3.61 | 3.22 | 2.91 | 2.82 | 2.45 | 2.23 |
|
| 45 |
|
| 46 |
Short context benchmarks showing that quality degradation is minimal:
|
| 47 |
| Model | Context Window | ARC-c | Hellaswag | MMLU | Truthful QA |
|
| 48 |
|-------|---------------:|------:|----------:|-----:|------------:|
|
| 49 |
+
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k | 67.32 | 87.33 | 69.83 | 44.92 |
|
| 50 |
+
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k | 67.41 | - | 68.84 | - |
|
| 51 |
|
| 52 |
## Collaborators
|
| 53 |
|