e=mc² Lesson 10: Does it earn a lane?
Dynamic int8 quantization of the from-scratch pankh-decider-v2 and the Qwen2.5-0.5B LoRA decider from Lesson 09, CPU latency and size, a stdlib HTTP server speaking pankhllm's /v1/systemone decision shape, and an end-to-end evaluation through the pankhllm 0.1.1 binary.
Lesson: https://theaivibe.org/emc2/quantize-small-llm-int8-does-it-earn-a-lane
Verdict, on 172 teacher-written test questions, against pankhllm's own regression trained on the same data and measured through the same binary (36.6% decided at 92.1%), with pankhllm's known-words gate on: pankh-decider-v2 int8 decided 30.2% at 96.2% (1.10 MB, 1.73 ms p50 inside pankhllm); Qwen int8 decided 36.0% at 93.5% (1,000 MB, 36.9 ms). Neither holds 98% precision, so neither earns a lane yet; the gate does not change that.
Files: lesson_10.ipynb (unexecuted), RESULTS.md, pankh-decider-v2-int8/ (model_int8.pt, config.json, vocab.json), pankhllm-lane.yaml. Author: Prateek Singh. MIT.