Update README.md
Browse files
README.md
CHANGED
|
@@ -52,6 +52,21 @@ The chat template for Chikuma_10.7B - V2 is a modified version of ChatML, optimi
|
|
| 52 |
{asistant}<|im_end|>
|
| 53 |
```
|
| 54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
### Training Environment
|
| 56 |
- Hardware: Single A100 80GB GPU in a runpod, utilized for approximately 1.5 hours.
|
| 57 |
- Training Script: Accessible via [Google Colab Notebook](https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing). Special thanks to [mlabonne](https://huggingface.co/mlabonne) for providing the template.
|
|
|
|
| 52 |
{asistant}<|im_end|>
|
| 53 |
```
|
| 54 |
|
| 55 |
+
## Benchmarks
|
| 56 |
+
| Benchmark Name | Performance |
|
| 57 |
+
|------------------------|-------------|
|
| 58 |
+
| AGIEval | 42.77 |
|
| 59 |
+
| GPT4All | 73.81 |
|
| 60 |
+
| TruthfulQA | 58.83 |
|
| 61 |
+
| Bigbench | 44.83 |
|
| 62 |
+
| ARC | 66.38 |
|
| 63 |
+
| HellaSwag | 85 |
|
| 64 |
+
| Winogrande | 78.77 |
|
| 65 |
+
| GSM8K | 63.68 |
|
| 66 |
+
| **Average** | **63.66** |
|
| 67 |
+
|
| 68 |
+
Details can be found [here](https://gist.github.com/sethuiyer)
|
| 69 |
+
|
| 70 |
### Training Environment
|
| 71 |
- Hardware: Single A100 80GB GPU in a runpod, utilized for approximately 1.5 hours.
|
| 72 |
- Training Script: Accessible via [Google Colab Notebook](https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing). Special thanks to [mlabonne](https://huggingface.co/mlabonne) for providing the template.
|