File size: 1,862 Bytes
f9780f3 ef0afe9 f9780f3 d5d6cd9 205a180 f9780f3 3ac8421 f9780f3 61e8de7 f9780f3 3ac8421 82d3a2b 160fd54 f9780f3 47aff15 61e8de7 47aff15 f9780f3 61e8de7 f9780f3 61e8de7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
---
language:
- sv
license: llama3
library_name: transformers
tags:
- unsloth
datasets:
- neph1/bellman-7b-finetune
- neph1/codefeedback-swedish
---
# Model Card for Bellman
This version of bellman is finetuned from llama-3-instruct-8b.
It is arguable whether it's better at Swedish, because llama-3 is really good. It's however finetuned for prompt question answering, based on a dataset created from
Swedish wikipedia, with a lot of Sweden-centric questions.
New from previous versions is questions from a translated code-feedback dataset, as well as a number of stories. It's not great at generating stories,
but better than previosly.
Please note, the HuggingFace inference api is probably trying to load the adapter (lora) which isn't going to work.
240609: I've uploaded a 4-bit GPTQ quant, but it's completely untested.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/653cd3049107029eb004f968/IDGX3d9lGe6yx-yHjsrav.png)
## Model Details
Training run on 240606:
Step Training Loss Validation Loss<br>
25 1.506400 1.164538<br>
50 1.128500 1.059316<br>
75 1.095100 1.040511<br>
100 1.068700 1.031033<br>
125 1.061300 1.024377<br>
150 1.035700 1.017490<br>
175 1.061200 1.012095<br>
200 1.031600 1.007867<br>
225 1.031900 1.002652<br>
250 0.958300 1.003817<br>
275 0.967900 1.000483<br>
300 0.950000 0.998807<br>
325 0.974300 0.996894<br>
350 0.960700 0.994098<br>
375 0.956000 0.991491<br>
400 0.940500 0.988697<br>
425 0.949100 0.987253<br>
450 0.940600 0.986425 <-- Picked checkpoint<br>
475 0.888300 0.994204<br>
500 0.881700 0.994897<br>
### Model Description
- **Developed by:** Me
- **Funded by:** Me
- **Model type:** Instruct
- **Language(s) (NLP):** Swedish
- **License:** llama-3
- **Finetuned from model:** Llama3 Instruct 8b
## Model Card Contact
rickard@mindemia.com |