Instructions to use meirdick/router-expert-grade_science with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use meirdick/router-expert-grade_science with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507") model = PeftModel.from_pretrained(base_model, "meirdick/router-expert-grade_science") - Notebooks
- Google Colab
- Kaggle
router-expert-grade_science
LoRA expert for the grade_science target of the route-then-admit pool, trained on meirdick/router-experts-data data/experts/grade_science.jsonl.
- base:
Qwen/Qwen3-4B-Instruct-2507 - rank 16, alpha 32, dropout 0.05, modules q_proj, k_proj, v_proj, o_proj
- lr 0.0002, epochs 2, max_len 1024, token budget 8192 per batch
- one example per item per recipe (direct,cot_short), under the serving system prompt; 40% of the mc items re-lettered to 5 to 10 options
- loss on the assistant turn only; held-out 5% for the val loss
| field | value |
|---|---|
| target | grade_science |
| items | 1500 |
| recipes | direct,cot_short |
| kinds | {'mc': 1500} |
| mc_padded | 534 |
| examples | 3000 |
| encoded | 3000 |
| dropped_too_long | 0 |
| train_rows | 2850 |
| val_rows | 150 |
| train_batches | 51 |
| train_tokens | 403963 |
| supervised_tokens | 64914 |
| steps | 102 |
| train_loss | 0.8913410418100801 |
| val_loss_before | 4.000902233691571 |
| val_loss | 0.8184471968267087 |
| seconds | 185.67 |
| trainable_params | 11796480 |
- Downloads last month
- 16
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for meirdick/router-expert-grade_science
Base model
Qwen/Qwen3-4B-Instruct-2507