Update README.md
Browse files
README.md
CHANGED
@@ -5,6 +5,8 @@ library_name: transformers
|
|
5 |
license: apache-2.0
|
6 |
language:
|
7 |
- en
|
|
|
|
|
8 |
---
|
9 |
|
10 |
# Model Card: Nous-Yarn-Llama-2-70b-32k
|
@@ -38,14 +40,14 @@ pip install git+https://github.com/huggingface/transformers
|
|
38 |
Long context benchmarks:
|
39 |
| Model | Context Window | 1k PPL | 2k PPL | 4k PPL | 8k PPL | 16k PPL | 32k PPL |
|
40 |
|-------|---------------:|-------:|--------:|------:|-------:|--------:|--------:|
|
41 |
-
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k |
|
42 |
-
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k |
|
43 |
|
44 |
Short context benchmarks showing that quality degradation is minimal:
|
45 |
| Model | Context Window | ARC-c | Hellaswag | MMLU | Truthful QA |
|
46 |
|-------|---------------:|------:|----------:|-----:|------------:|
|
47 |
-
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k |
|
48 |
-
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k |
|
49 |
|
50 |
## Collaborators
|
51 |
|
|
|
5 |
license: apache-2.0
|
6 |
language:
|
7 |
- en
|
8 |
+
datasets:
|
9 |
+
- emozilla/yarn-train-tokenized-8k-llama
|
10 |
---
|
11 |
|
12 |
# Model Card: Nous-Yarn-Llama-2-70b-32k
|
|
|
40 |
Long context benchmarks:
|
41 |
| Model | Context Window | 1k PPL | 2k PPL | 4k PPL | 8k PPL | 16k PPL | 32k PPL |
|
42 |
|-------|---------------:|-------:|--------:|------:|-------:|--------:|--------:|
|
43 |
+
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k | 3.71 | 3.27 | 2.96 | - | - | - |
|
44 |
+
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k | 3.61 | 3.22 | 2.91 | 2.82 | 2.45 | 2.23 |
|
45 |
|
46 |
Short context benchmarks showing that quality degradation is minimal:
|
47 |
| Model | Context Window | ARC-c | Hellaswag | MMLU | Truthful QA |
|
48 |
|-------|---------------:|------:|----------:|-----:|------------:|
|
49 |
+
| [Llama-2-70b-hf](meta-llama/Llama-2-70b-hf) | 4k | 67.32 | 87.33 | 69.83 | 44.92 |
|
50 |
+
| [Yarn-Llama-2-70b-32k](https://huggingface.co/NousResearch/Yarn-Llama-2-70b-32k) | 32k | 67.41 | - | 68.84 | - |
|
51 |
|
52 |
## Collaborators
|
53 |
|