Balab2021/Qwen3.8-27B-Cricket-Reasoning-LoRA

LoRA adapter for Qwen/Qwen3.8-27B fine-tuned on verified cricket reasoning traces (T20I ball-by-ball situations from Valarmathy/CricketData; run-rate, chase-rate and batting-milestone questions with exact answers). Use thinking mode (enable_thinking=True) with temperature 1.0, top_p 0.95, top_k 20.

Qwen3.8-27B cricket reasoning LoRA - run general_sa-cricket.sft-696652

• SFT: 4410 verified reasoning rows, 138 steps (2.00 epochs) in 20.2 min, final train loss 0.3229
• Eval: 600 held-out questions (unseen matches) x 4 samples, thinking mode, temperature 1.0 / top_p 0.95 / top_k 20, answers checked against exact values

question type	metric	base	fine-tuned	delta
run_rate	accuracy (avg over samples)	98.50%	99.00%	+0.50 pts
run_rate	pass@4	100.00%	100.00%	+0.00 pts
run_rate	mean output tokens	404	280	-124
run_rate	truncated (hit token limit)	0.12%	0.00%	-0.12 pts
run_rate	finished but no parsable answer	0.00%	0.00%	+0.00 pts
chase_rate	accuracy (avg over samples)	93.88%	92.25%	-1.62 pts
chase_rate	pass@4	100.00%	100.00%	+0.00 pts
chase_rate	mean output tokens	929	610	-320
chase_rate	truncated (hit token limit)	0.38%	0.00%	-0.38 pts
chase_rate	finished but no parsable answer	0.00%	0.00%	+0.00 pts
milestone	accuracy (avg over samples)	99.38%	99.88%	+0.50 pts
milestone	pass@4	100.00%	100.00%	+0.00 pts
milestone	mean output tokens	910	557	-353
milestone	truncated (hit token limit)	0.50%	0.12%	-0.38 pts
milestone	finished but no parsable answer	0.00%	0.00%	+0.00 pts
overall	accuracy (avg over samples)	97.25%	97.04%	-0.21 pts
overall	pass@4	100.00%	100.00%	+0.00 pts
overall	mean output tokens	748	482	-266
overall	truncated (hit token limit)	0.33%	0.04%	-0.29 pts
overall	finished but no parsable answer	0.00%	0.00%	+0.00 pts

Questions answered more reliably after fine-tuning: 36; less reliably: 39 (of 600; compares the fraction of correct samples per question).

Plots