eval for 1.2
Browse files
README.md
CHANGED
|
@@ -29,7 +29,15 @@ The current version is **1.2**; congrats to our team on the new release!
|
|
| 29 |
|
| 30 |
Previous versions remain available in the repository. New models will be released for everyone once our team's training and validation process is complete :)
|
| 31 |
|
| 32 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
Shining Valiant uses the same prompt format as Llama 2 Chat - feel free to use your existing prompts and scripts!
|
| 34 |
A few examples of different formats:
|
| 35 |
|
|
|
|
| 29 |
|
| 30 |
Previous versions remain available in the repository. New models will be released for everyone once our team's training and validation process is complete :)
|
| 31 |
|
| 32 |
+
## Evaluation
|
| 33 |
+
|
| 34 |
+
| Model | Avg | ARC | HS | MMLU | TQA |
|
| 35 |
+
|-----------------------|--------|-------|-------|--------|-------|
|
| 36 |
+
| **Shining Valiant 1.2** | 74.17 | 72.95 | 87.88 | 70.97 | 64.88 |
|
| 37 |
+
| Llama 2 | 67.35 | 67.32 | 87.33 | 69.83 | 44.92 |
|
| 38 |
+
| Llama 2 Chat | 66.80 | 64.59 | 85.88 | 63.91 | 52.80 |
|
| 39 |
+
|
| 40 |
+
## Prompting Guide
|
| 41 |
Shining Valiant uses the same prompt format as Llama 2 Chat - feel free to use your existing prompts and scripts!
|
| 42 |
A few examples of different formats:
|
| 43 |
|