ChiraSahu/Llama-3.2-3B-Instruct-GPTQ-Int8

meta-llama/Llama-3.2-3B-Instruct -- GPTQ Int8

Self-quantized with GPTQModel for the "Phase-Aware Energy and Efficiency Profiling of Quantized LLM Inference on Edge GPUs" project (SPIT Mumbai). No pre-quantized, unmodified GPTQ-Int8 checkpoint existed for this exact base model at quantization time -- the only one found was a finetuned variant, which would have confounded a quantization-only comparison.

• Base model: meta-llama/Llama-3.2-3B-Instruct (unmodified instruct checkpoint, no finetuning)
• Method: GPTQ, bits=8, group_size=128
• Calibration: 512 raw documents from allenai/c4 (en, shard 1 of 1024), packed into 2048-token blocks (calibration_concat_size=2048) -- matches the original GPTQ paper's calibration convention
• Quantization time: 45.8 min on a Kaggle Tesla T4

Quantized for benchmarking, not tuned for downstream deployment quality.