router-expert-computer_science

LoRA expert for the computer_science target of the route-then-admit pool, trained on meirdick/router-experts-data data/experts/computer_science.jsonl.

  • base: Qwen/Qwen3-4B-Instruct-2507
  • rank 16, alpha 32, dropout 0.05, modules q_proj, k_proj, v_proj, o_proj
  • lr 0.0002, epochs 2, max_len 1024, token budget 8192 per batch
  • one example per item per recipe (direct,cot_short), under the serving system prompt; 40% of the mc items re-lettered to 5 to 10 options
  • loss on the assistant turn only; held-out 5% for the val loss
field value
target computer_science
items 1500
recipes direct,cot_short
kinds {'mc': 1500}
mc_padded 542
examples 3000
encoded 3000
dropped_too_long 0
train_rows 2850
val_rows 150
train_batches 55
train_tokens 436373
supervised_tokens 68648
steps 110
train_loss 1.0584621136838739
val_loss_before 4.3211695482959485
val_loss 0.9644685333069056
seconds 201.03
trainable_params 11796480
Downloads last month
18
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for meirdick/router-expert-computer_science

Adapter
(5707)
this model