---
language:
- en
license: cc-by-nc-4.0
tags:
- text-generation-inference
- transformers
- unsloth
- mistral
- trl
base_model: alnrg2arg/blockchainlabs_7B_merged_test2_4
datasets:
- Open-Orca/SlimOrca
---

Benchmark Scores

|    Tasks    |Version|Filter|n-shot| Metric |Value |   |Stderr|
|-------------|------:|------|-----:|--------|-----:|---|-----:|
|arc_challenge|      1|none  |     0|acc     |0.5247|±  |0.0146|
|             |       |none  |     0|acc_norm|0.5623|±  |0.0145|

|  Tasks  |Version|Filter|n-shot| Metric |Value |   |Stderr|
|---------|------:|------|-----:|--------|-----:|---|-----:|
|hellaswag|      1|none  |     0|acc     |0.6270|±  |0.0048|
|         |       |none  |     0|acc_norm|0.8228|±  |0.0038|

|      Groups      |Version|Filter|n-shot|Metric|Value |   |Stderr|
|------------------|-------|------|-----:|------|-----:|---|-----:|
|mmlu              |N/A    |none  |     0|acc   |0.6243|±  |0.1341|
| - humanities     |N/A    |none  |     0|acc   |0.5717|±  |0.1400|
| - other          |N/A    |none  |     0|acc   |0.7016|±  |0.1143|
| - social_sciences|N/A    |none  |     0|acc   |0.7342|±  |0.0753|
| - stem           |N/A    |none  |     0|acc   |0.5192|±  |0.1257|

|  Tasks   |Version|Filter|n-shot|Metric|Value |   |Stderr|
|----------|------:|------|-----:|------|-----:|---|-----:|
|winogrande|      1|none  |     0|acc   |0.7774|±  |0.0117|

|Tasks|Version|  Filter  |n-shot|  Metric   |Value |   |Stderr|
|-----|------:|----------|-----:|-----------|-----:|---|-----:|
|gsm8k|      2|get-answer|     5|exact_match|0.6732|±  |0.0129|

|    Tasks     |Version|Filter|n-shot|Metric|Value |   |Stderr|
|--------------|------:|------|-----:|------|-----:|---|-----:|
|truthfulqa_mc2|      2|none  |     0|acc   |0.4795|±  |0.0148|

Average 65.658