Smallstral, layer pruned Mistral
Task | Metric | Mistral-7B-v0.1 Value | Smallstral Value |
---|---|---|---|
hendrycksTestRu-high_school_mathematics | acc | 0.3037 | 0.2852 |
hendrycksTestRu-high_school_microeconomics | acc | 0.3950 | 0.3319 |
hendrycksTestRu-high_school_physics | acc | 0.3113 | 0.2649 |
hendrycksTestRu-high_school_psychology | acc | 0.5193 | 0.1945 |
hendrycksTestRu-high_school_statistics | acc | 0.3241 | 0.4028 |
hendrycksTestRu-high_school_us_history | acc | 0.5392 | 0.2451 |
hendrycksTestRu-high_school_world_history | acc | 0.5781 | 0.2321 |
hendrycksTestRu-human_aging | acc | 0.4843 | 0.3184 |
parus | acc | 0.5000 | 0.4875 |
rcb | acc | 0.3653 | 0.3288 |
rcb | f1_macro | 0.3313 | 0.1691 |
rwsd | acc | 0.4917 | 0.4901 |
- Downloads last month
- 16
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.