File size: 1,943 Bytes
0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
license: apache-2.0
---
<style>
img{
user-select: none;
transition: all 0.2s ease;
border-radius: .5rem;
}
img:hover{
transform: rotate(2deg);
filter: invert(100%);
}
@import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap');
</style>
<div style="background-color: transparent; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;">
**September Update** - this is the official model used in dot, keep in mind, none of these models use specific prompts, you might need to fine-tune them to use them as chatbots.
#### benchmarks
zero-shot evaluations performed on current sota ~0.5b models against the best language model below 2b parameters.
| Parameters | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average |
| -----------|--------------------------------|-------|-------|-----------|--------|------------|---------|
| 0.5b | qwen2 |**0.4413**| 0.2892| 0.4905 | 0.6931 | 0.5699 | 0.4968 |
| 0.6b | mobilellm | - | 0.3580| 0.5590 | 0.7230 | 0.5860 | - |
| 0.5b | danube3 | 0.2481| 0.3618| 0.6046 | 0.7378 | 0.6101 | 0.5125 |
| 0.5b | palmer |0.2617|**0.3729**|**0.6288**|**0.7437**| **0.6227** |**0.5260**|
| 1.7b | smollm |0.2765|0.4626| 0.6574 | 0.7606 | 0.6093 | 0.5533 |
#### supporters
<a href="https://ko-fi.com/appvoid" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 34px !important; margin-top: -4px;width: 128px !important; filter: contrast(2) grayscale(100%) brightness(100%);" ></a>
</div> |