trunpreonx/24B0978-Week02-Track2-40-Submission01

24B0978 — Week 02 — Track 2, 40% target — Submission 01

Compressed checkpoint: 3,450,632,192 bytes = 0.3564 of the bf16 reference (9,682,900,992 bytes), measured with count_total_size (numel * itemsize over the state dict), which eval_submission.py documents as the graded quantity.

Model: Qwen/Qwen3.5-4B. Domain: math.

Result

	math accuracy	retention vs bf16
this submission	0.8590	0.9140
bf16 reference	0.9399	1.0000

Per evaluation set:

set	accuracy	retention
gsm8k	0.8621	1.0000
math500	0.8684	0.9167
math_hard	0.7959	0.8478
mmlu_pro_math	0.9697	1.0000

Measured on our own frozen benchmark (four math sets, 16384 new tokens, greedy), not on the hidden set.

Where the checkpoint is

artifact.pt (3.45 GB) is in the HuggingFace repo, not here — GitHub caps files at 100 MB. code.py loads it from its own directory.

https://huggingface.co/trunpreonx/24B0978-Week02-Track2-40-Submission01

This recipe is post-training only, so code.py can rebuild the checkpoint from the base model. It ships the measured one anyway: GPTQ is not bit-reproducible across runs — two builds of this recipe from identical source differ in 602 of 834 stored tensors, because the Hessian is measured on freshly drawn calibration text and GPU reductions do not re-associate the same way twice. Shipping the artifact means the graded checkpoint is exactly the one the numbers above describe, and saves ~200 GPU-minutes of recompression.

Recipe

stage	modules	format
1	all attention and FFN projections (q/k/v/o_proj, in_proj_qkv, in_proj_z, out_proj, gate/up/down_proj)	RHT + GPTQ, 6-bit, group 128
2	embed_tokens (tied to lm_head)	RTN, 8-bit, group 128

Calibration: 128 sequences from the eval-shaped mixture (50% competition maths, 25% grade-school, 25% multiple choice), matching the format split of the four graded sets rather than free-form derivations alone.

A random Hadamard transform is applied per weight block before quantisation and inverted at restore, so GPTQ's error compensation runs in an incoherent basis where outliers are spread across channels instead of concentrated in a few.

Track 2 relevance: every stage is a uniform grouped-affine integer code with a fixed group size. Dequantisation is a scale-and-shift over contiguous groups and the Hadamard transform is a butterfly — both are standard fused-dequant GEMM kernel shapes, so nothing here relies on data-dependent lookup or irregular sparsity that would block a CUDA speedup.

Reproducing

python evaluation/eval_submission.py \
  --submission 24B0978/Week02/Track2_40/Submission01/code.py \
  --model Qwen/Qwen3.5-4B

artifact.pt must sit next to code.py.

Recipe (exact)

{
  "id": "gen4-h40-rht-gptq-6bit",
  "track": 2,
  "needs": {
    "gpus": 1,
    "gpu_mem_gb": 40,
    "est_minutes": 200,
    "disk_gb": 25
  },
  "stages": [
    {
      "op": "quant_rht_gptq",
      "params": {
        "bits": 6,
        "group": 128,
        "seed": 1
      },
      "select": {
        "layers": "all",
        "modules": [
          "down_proj",
          "gate_proj",
          "in_proj_qkv",
          "in_proj_z",
          "k_proj",
          "o_proj",
          "out_proj",
          "q_proj",
          "up_proj",
          "v_proj"
        ]
      }
    },
    {
      "op": "quant_rtn",
      "params": {
        "bits": 8,
        "group": 128
      },
      "select": {
        "layers": "all",
        "modules": [
          "embed_tokens"
        ]
      }
    }
  ],
  "train": {
    "mode": "calib",
    "calib_samples": 128,
    "calib_source": "mix"
  },
  "hypothesis": "The 40% head ranks by ACCURACY ALONE, so the best entry is the most accurate recipe still measuring <= 0.4 -- and our banked control wastes the head. It spends 6.3 bits on plain round-to-nearest (int8 everywhere, int4 on the FFN) and scores 0.863. RHT+GPTQ at 6 bits with an 8-bit embedding lands at 0.356, comfortably inside, while using the algorithm measured 23.7x better than RTN at matched bytes (job 279735) and the rotation measured to help most where compensation is already working. This should be close to lossless, and it is the cheapest remaining experiment that can improve our single best banked result.",
  "citations": []
}