sDarth/24b1064-Week08-Compression-10-Submission01

Week 08 Compression 10 — Submission 01

Planned recipe: fast-v3 distilled Qwen3.5-2B, GPTQ W4/G64, desc_act=false, reasoning-aware math calibration (HMMT excluded).

Status: checkpoint rebuilt and quantized, but NOT eligible for the 10% track. The original r16, two-epoch, 128-trace fast-v3 training recipe was recovered from commit a63f6f5. Merged training completed on all 128 traces. The packed checkpoint is preserved at artifacts/week08_fast_v3_2b_gptq_w4_g64/compressed. Calibration used exactly 100,000 tokens across 89 sequences. No new HMMT evaluation or generation was run. The packed checkpoint was uploaded to the model repo at the user's request, revision 1a88568388fca70068bd31253398dfd3a03e573c. Decompression and vLLM smoke validation completed during the Week08 audit. The short 4096-token smoke parsed 2/3 answers (one response hit the cap). At the evaluator's 32,000-token cap, the same three-problem compatibility smoke completed with 2/3 correct answers and 100% parse rate. The corrected official tensor-size fraction is 0.1163386364 (11.63%); the complete directory fraction is 0.1186605510 (11.87%). The uncorrected official helper reports about 8.60% because it skips the packed embedding .pt sidecar when safetensors are present. That incomplete measurement must not be used to claim 10% eligibility. An audit exposed the exact sidecar tensors as safetensors and measured all of them with the official helper: 0.9375149012 GiB of text tensors. W4/G64 on a 2B student is NOT automatically within the 10% track. Measure the packed checkpoint with the official evaluator before uploading it. The compressor rejects a tensor-size fraction >= 0.10 relative to the course's 8.0585 GiB Qwen3.5-4B text baseline.

Public model repository: sDarth/24b1064-Week08-Compression-10-Submission01.

Expected distilled parent: artifacts/distilled/qwen35_2b_fast_v3_r16/model.

Install the self-contained submission environment with uv sync --python 3.12.

uv run python compress.py \
  --model_name Qwen/Qwen3.5-4B \
  --checkpoint_path /marimo/CS6013/artifacts/distilled/qwen35_2b_fast_v3_r16/model \
  --calibration_path /marimo/CS6013/data/traces/qwen35_4b_distill_fast_v3.jsonl \
  --output_path /marimo/CS6013/artifacts/week08_fast_v3_2b_gptq_w4_g64/compressed

--model_name identifies the course reference model; the supplied checkpoint is the distilled 2B student. Never substitute the untrained base 2B model.

uv run python decompress.py \
  --model_name Qwen-3.5-4B \
  --checkpoint_path /marimo/CS6013/artifacts/week08_fast_v3_2b_gptq_w4_g64/compressed \
  --output_path /marimo/CS6013/artifacts/week08_fast_v3_2b_gptq_w4_g64/decompressed

Before submission: verify official size_frac < 0.10, validate decompression and Qwen3.5 processor metadata, and smoke-test the serialized output with vLLM. The original recipe also packs the shared embedding/head at W4/G128. The bundled decompressor is designed to restore that sidecar and apply Qwen3.5's composite-wrapper packaging for vLLM. The official evaluator CLI and vLLM loaded the reconstructed artifact successfully in the Week08 audit. The official size helper may skip .pt sidecars beside safetensors; count the tied embedding storage too, not only the decoder shards. Do not upload a size-failing or unvalidated model as a finished submission.