router-expert-grade_science

LoRA expert for the grade_science target of the route-then-admit pool, trained on meirdick/router-experts-data data/experts/grade_science.jsonl.

  • base: Qwen/Qwen3-4B-Instruct-2507
  • rank 16, alpha 32, dropout 0.05, modules q_proj, k_proj, v_proj, o_proj
  • lr 0.0002, epochs 2, max_len 1024, token budget 8192 per batch
  • one example per item per recipe (direct,cot_short), under the serving system prompt; 40% of the mc items re-lettered to 5 to 10 options
  • loss on the assistant turn only; held-out 5% for the val loss
field value
target grade_science
items 1500
recipes direct,cot_short
kinds {'mc': 1500}
mc_padded 534
examples 3000
encoded 3000
dropped_too_long 0
train_rows 2850
val_rows 150
train_batches 51
train_tokens 403963
supervised_tokens 64914
steps 102
train_loss 0.8913410418100801
val_loss_before 4.000902233691571
val_loss 0.8184471968267087
seconds 185.67
trainable_params 11796480
Downloads last month
16
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for meirdick/router-expert-grade_science

Adapter
(5707)
this model