Commit
•
3d00a29
1
Parent(s):
5272bef
Update README.md (#7)
Browse files- Update README.md (7836faa52bf013d1e50247872c3ccca71a805cc2)
Co-authored-by: Chaitanya Singhal <Chaitanya890@users.noreply.huggingface.co>
README.md
CHANGED
@@ -138,9 +138,25 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
|
|
138 |
|
139 |
```
|
140 |
|
141 |
-
|
142 |
|
|
|
143 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
144 |
| Model | # Params | Average | ARC (25-shot) | HellaSwag (10-shot) | Winogrande (5-shot) | TruthfulOA (0-shot) | MMLU (5-shot) |
|
145 |
|-----------------------------------|----------|---------|---------------|---------------------|---------------------|---------------------|---------------|
|
146 |
| aiplanet/buddhi-128k-chat-7b | 7B | 64.42 | 60.84 | 84 | 77.27 | 65.72 | 60.42 |
|
@@ -151,6 +167,7 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
|
|
151 |
| CallComply/openchat-3.5-0106-128k | 7B | 59.38 | 64.25 | 77.31 | 77.66 | 46.5 | 57.58 |
|
152 |
| CallComply/zephyr-7b-beta-128k | 7B | 54.45 | 58.28 | 81 | 74.74 | 46.1 | 53.57 |
|
153 |
|
|
|
154 |
|
155 |
## Get in Touch
|
156 |
|
|
|
138 |
|
139 |
```
|
140 |
|
141 |
+
# Benchmarks
|
142 |
|
143 |
+
### Long Context Benchmark
|
144 |
|
145 |
+
<strong>LongICLBench Banking77</strong>
|
146 |
+
<div>
|
147 |
+
|
148 |
+
| Model | 1R/2k | 2R/4K | 3R/7K | 4R/9K | 5R/14K |
|
149 |
+
|-----------------------------------------|-------|-------|-------|-------|--------|
|
150 |
+
| aiplanet/buddhi-128k-chat-7b | 47.8 | 60.8 | 57.8 | 62.4 | 57.2 |
|
151 |
+
| NousResearch/Yarn-Mistral-7b-128k | 31.6 | 68.6 | 68 | 47 | 65.6 |
|
152 |
+
| CallComply/zephyr-7b-beta-128k | 40.2 | 41.2 | 33.6 | 03 | 0 |
|
153 |
+
| Eric111/Yarn-Mistral-7b-128k-DPO | 28.6 | 62.8 | 58 | 41.6 | 59.8 |
|
154 |
+
|
155 |
+
</div>
|
156 |
+
|
157 |
+
<strong>Short Context Benchmark</strong>
|
158 |
+
<div>
|
159 |
+
|
160 |
| Model | # Params | Average | ARC (25-shot) | HellaSwag (10-shot) | Winogrande (5-shot) | TruthfulOA (0-shot) | MMLU (5-shot) |
|
161 |
|-----------------------------------|----------|---------|---------------|---------------------|---------------------|---------------------|---------------|
|
162 |
| aiplanet/buddhi-128k-chat-7b | 7B | 64.42 | 60.84 | 84 | 77.27 | 65.72 | 60.42 |
|
|
|
167 |
| CallComply/openchat-3.5-0106-128k | 7B | 59.38 | 64.25 | 77.31 | 77.66 | 46.5 | 57.58 |
|
168 |
| CallComply/zephyr-7b-beta-128k | 7B | 54.45 | 58.28 | 81 | 74.74 | 46.1 | 53.57 |
|
169 |
|
170 |
+
</div>
|
171 |
|
172 |
## Get in Touch
|
173 |
|