|
--- |
|
language: |
|
- sv |
|
license: llama3.1 |
|
library_name: transformers |
|
tags: |
|
- unsloth |
|
datasets: |
|
- neph1/bellman-7b-finetune |
|
- neph1/codefeedback-swedish |
|
--- |
|
|
|
# Model Card for Bellman |
|
|
|
This version of bellman is finetuned from llama-3.1-instruct-8b. |
|
It's finetuned for prompt question answering, based on a dataset created from |
|
Swedish wikipedia, with a lot of Sweden-centric questions. |
|
New from previous versions is questions from a translated code-feedback dataset, as well as a number of stories. It's not great at generating stories, |
|
but better than previosly. |
|
|
|
Please note, the HuggingFace inference api is probably trying to load the adapter (lora) which isn't going to work. |
|
|
|
 |
|
|
|
## Model Details |
|
|
|
Training run on 240724: |
|
|
|
Step Training Loss Validation Loss<br> |
|
25 1.352200 1.034565<br> |
|
50 1.033600 1.009348<br> |
|
75 1.022400 0.996665<br> |
|
100 1.002900 0.988050<br> |
|
125 1.014600 0.981633<br> |
|
150 1.006300 0.975584<br> |
|
175 0.988800 0.970966<br> |
|
200 0.985300 0.967037<br> |
|
225 0.992400 0.964120<br> |
|
250 0.950000 0.962472<br> |
|
275 0.931000 0.960848<br> |
|
300 0.932000 0.958946 <-- picked checkpoint <br> |
|
|
|
### Model Description |
|
|
|
|
|
- **Developed by:** Me |
|
- **Funded by:** Me |
|
- **Model type:** Instruct |
|
- **Language(s) (NLP):** Swedish |
|
- **License:** llama-3.1 |
|
- **Finetuned from model:** Llama3.1 Instruct 8b |
|
|
|
## Model Card Contact |
|
|
|
rickard@mindemia.com |
