Description
This is an instruction following model (based on Mistral v0.1 Base) optimized for Russian language. It was trained using kolibrify on a multitude of instruction datasets.
The model uses ChatML template. Adding a system prompt will likely improve the model's performance on your task (experiment with it). I recommend using the model with LMStudio.
Currently in pre-alpha, later releases will include more details regarding training procedure and data mix.
Instruction following evals
The model was tested using the following benchmarks:
| Eval name | Strict Value | Loose Value |
|---|---|---|
| Avg. | 49.92 | 52.52 |
| ifeval-prompt-level | 49.72 | 52.31 |
| ifeval-instruction-level | 60.91 | 63.18 |
| ru-ifeval-prompt-level | 39.55 | 42.32 |
| ru-ifeval-instruction-level | 49.52 | 52.27 |
- Downloads last month
- 58