Update README.md
Browse files
README.md
CHANGED
@@ -23,15 +23,15 @@ Heidrun-Mistral-7B-chat is a chat-model based on [Heidrun-Mistral-7B-base](https
|
|
23 |
|
24 |
It is a new SOTA Danish open-source LLM and shows very strong performance in logic and reasoning tasks.
|
25 |
|
|
|
|
|
26 |
# Benchmarks
|
27 |
|
28 |
The following benchmarks have been tested using [ScandEval](https://github.com/ScandEval/ScandEval). Rankings don't include merged models and GPT4 and GPT3.5 ranks 1 and 2:
|
29 |
|
30 |
- **MMLU-da**: 35.66%+-0.85%/51.68+-0.63%, ranks 3rd
|
31 |
-
- **DANSK**: 50.
|
32 |
-
- **Hellaswag-da**: 29.18
|
33 |
-
|
34 |
-
Further evaluations will be tested.
|
35 |
|
36 |
# Datasets
|
37 |
This model is trained on Danish instruction datasets [danish-OpenHermes](Mabeck/danish-OpenHermes) and [skoleGPT](https://huggingface.co/datasets/kobprof/skolegpt-instruct), which have not been safeguarded or alligned.
|
|
|
23 |
|
24 |
It is a new SOTA Danish open-source LLM and shows very strong performance in logic and reasoning tasks.
|
25 |
|
26 |
+
Heidrun-7B ranks 1st among Danish open-sourced LLMs based on the [ScandEval](https://scandeval.com/mainland-scandinavian-nlg/) benchmark and shares a 1st place .
|
27 |
+
|
28 |
# Benchmarks
|
29 |
|
30 |
The following benchmarks have been tested using [ScandEval](https://github.com/ScandEval/ScandEval). Rankings don't include merged models and GPT4 and GPT3.5 ranks 1 and 2:
|
31 |
|
32 |
- **MMLU-da**: 35.66%+-0.85%/51.68+-0.63%, ranks 3rd
|
33 |
+
- **DANSK**: 50.80% ± 2.33% / 34.04% ± 1.76%, ranks 3rd=
|
34 |
+
- **Hellaswag-da**: 29.18% ± 0.99%/ 46.64% ± 0.76%, ranks 4th
|
|
|
|
|
35 |
|
36 |
# Datasets
|
37 |
This model is trained on Danish instruction datasets [danish-OpenHermes](Mabeck/danish-OpenHermes) and [skoleGPT](https://huggingface.co/datasets/kobprof/skolegpt-instruct), which have not been safeguarded or alligned.
|