yukiontheiceberg
commited on
Commit
•
ed4ff73
1
Parent(s):
c5f8676
Update README.md
Browse files
README.md
CHANGED
@@ -11,14 +11,15 @@ tags:
|
|
11 |
|
12 |
# CrystalChat
|
13 |
|
14 |
-
We present CrystalChat, an instruction following model finetuned from [LLM360/CrystalCoder](https://huggingface.co/LLM360/CrystalCoder).
|
15 |
|
16 |
-
| Model | Trained Tokens | ARC | HellaSwag | MMLU (5-shot) | GSM8K | Winogrande(5-shot) | TruthfulQA |
|
17 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
18 |
-
|
|
19 |
-
|
|
20 |
-
| CodeLlama-7b-Instruct | 2.5T | 43.35 | 66.14 | 42.75 | 15.92 | 64.33 | 39.23 |
|
21 |
-
| Llama-2-7b-Chat | 2T | 53.07 | 78.39 | 48.42 | 18.88 | 73.09 | 45.30 |
|
|
|
22 |
|
23 |
## Model Description
|
24 |
|
|
|
11 |
|
12 |
# CrystalChat
|
13 |
|
14 |
+
We present CrystalChat, an instruction following model finetuned from [LLM360/CrystalCoder](https://huggingface.co/LLM360/CrystalCoder). Here's a comparison table for some popular chat models.
|
15 |
|
16 |
+
| Model | Trained Tokens | Avg. of Avg.| Language Avg. | Coding Avg. | ARC | HellaSwag | MMLU (5-shot) | GSM8K | Winogrande(5-shot) | TruthfulQA | HumanEval (pass@1) | MBPP (pass@1) |
|
17 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
18 |
+
| **CrystalChat 7B** | 1.4T | 44.96 | 53.29 | 36.62 | 51.71 | 76.12 | 53.22 | 28.05 | 70.64 | 47.29 | 34.12 | 39.11 |
|
19 |
+
| Mistral-7B-Instruct-v0.1 | - | 44.34 | 54.86 | 30.62 | 58.05 | 75.71 | 55.56 | 32.00 | 74.27 | 55.90 | 29.27 | 31.96 |
|
20 |
+
| CodeLlama-7b-Instruct | 2.5T | 40.91 | 45.29 | 36.52 | 43.35 | 66.14 | 42.75 | 15.92 | 64.33 | 39.23 | 34.12 | 38.91 |
|
21 |
+
| Llama-2-7b-Chat | 2T | 34.11 | 52.86 | 15.35 | 53.07 | 78.39 | 48.42 | 18.88 | 73.09 | 45.30 | 13.26 | 17.43 |
|
22 |
+
|AmberChat 7B|
|
23 |
|
24 |
## Model Description
|
25 |
|