OptGear/Opt.Gear-1B-qat-int4-per-channel-unquantized

Opt.Gear-1B-QAT

This repository contains the Quantization-Aware Training (QAT) checkpoint of Opt.Gear-1B: master weights adapted to the deployment INT4 grid, together with the learned channel-wise quantization scales.

The deployment format is W4A16 (per-channel symmetric INT4 weights, 16-bit activations). The checkpoint performs best with its learned QAT scales, and also re-quantizes stably with other INT4 PTQ methods (RTN, AWQ, GPTQ).

For the full-precision model, see Opt.Gear-1B.

Making a model small is not enough for real devices. Beyond parameter count, the memory occupied by weights, memory bandwidth, arithmetic precision, and the kernels supported by the accelerator determine final inference performance. The question is: how much of the original quality survives when the model is squeezed into 4 bits?

Opt.Gear-1B-QAT adapts the trained model directly to the deployment INT4 grid via Quantization-Aware Training. On six English/Korean benchmarks, the QAT model loses only ~5.8 point on average from the Base model — under the exact per-channel symmetric W4A16 grid used at deployment time.

We recommend using this model if your deployment framework lacks support for group-wise PTQ or if you prioritize inference speed over accuracy; otherwise, group-wise QAT weights are preferable for maximum accuracy.

QAT Highlights

• Group-size-32 symmetric INT4 weight quantization with a W4A16 deployment format
• Hessian-based scale initialization: channel-wise INT4 scales initialized by a layer-wise output reconstruction objective measured on calibration activations — not simple max-based statistics
• Learned Step Size Quantization (LSQ): weights and step sizes jointly optimized under the full language-modeling objective, with straight-through gradient estimation
• No quantizer mismatch: the same channel-wise INT4 grid is used during initialization, QAT, and final weight conversion, so the learned quantization structure survives deployment export
• Quantization-friendly weights: the checkpoint re-quantizes stably with RTN/AWQ/GPTQ, giving deployment pipelines freedom in quantizer choice

For more details, please refer to our tech report and blog post.

Quantization Configuration

Item	Setting
Weight precision	INT4
Activation precision	INT16
Quantization format	W4A16
Weight quantization	Symmetric (zero-point fixed at 0)
Scale initialization	Hessian-based output reconstruction
Scale optimization	Learned Step Size Quantization
QAT length	1,000 steps

The scale for each quantization group is initialized as

Δ* = argmin_Δ ‖ WX − Q_Δ(W)X ‖²_F

where W is the full-precision weight matrix, X contains the calibration inputs, and Q_Δ(W) denotes the weights quantized with scale Δ. This objective accounts for the relative importance of different input directions instead of treating all weight perturbations equally. Starting from this initialization, QAT jointly refines the weights and the LSQ step sizes under the global training objective, while the forward pass consistently emulates the final W4A16 deployment format.

Benchmark Results

Base, QAT, and PTQ results of Opt.Gear-1B and Gemma3-1B on six English/Korean benchmarks. RTN, AWQ, and GPTQ rows re-quantize the master weights of the QAT checkpoint with each PTQ method; the QAT row evaluates the final QAT quantizer including learned scales. All results are obtained using the LM Evaluation Harness.

Method	MMLU	HellaSwag	WinoGrande	KMMLU	KoBEST	HAERAE	Avg.	Δ
Opt.Gear-1B
Base	43.2	56.0	58.2	36.0	60.3	44.2	49.7	–
QAT	37.1	51.0	55.9	30.4	55.4	33.9	43.9	-5.8
RTN	35.7	51.3	56.5	30.8	55.0	33.0	43.7	-6.0
GPTQ	35.3	51.7	55.3	25.5	54.8	33.8	42.7	-7.0
Gemma3-1B (reference)
Base	39.8	60.0	58.7	30.7	59.5	35.3	47.3	–
QAT	35.8	51.4	55.0	26.9	56.2	31.2	42.8	-4.6
RTN	32.8	49.9	57.1	24.8	54.8	23.0	40.4	-6.9
GPTQ	31.9	48.1	55.2	24.9	53.1	26.6	40.0	-7.3

Re-quantization Stability

The QAT checkpoint performs best with its learned scales, and the QAT-trained weights themselves respond stably to other INT4 quantizers:

Model	Quantizer	Avg	Δ vs QAT
Opt.Gear-1B-QAT-per-channel	Learned QAT scale	43.9	–
	RTN	43.7	-0.2
	GPTQ	42.7	-1.2
Gemma3-1B-QAT	Learned QAT scale	42.8	–
	RTN	40.4	-2.4
	GPTQ	40.0	-2.8

Under identical RTN re-quantization, Opt.Gear-1B loses only 0.2 points on average, versus 2.4 points for Gemma3-1B. The Opt.Gear-1B-QAT checkpoint is not merely a set of learned scale parameters — the weights themselves have been adapted to be quantization-friendly, giving deployment pipelines freedom in quantizer choice.

Quickstart

This checkpoint is intended for INT4 deployment. For best quality, export with the included learned channel-wise scales (the same per-channel symmetric W4A16 grid used during QAT). Re-quantizing with RTN/AWQ/GPTQ is supported and remains stable, at a small cost in accuracy (see table above).

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "OptGear/Opt.Gear-1B-qat-int4-per-channel-unquantized" 

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [
    {"role": "user", "content": "온디바이스 AI가 왜 중요한지 설명해줘."},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Best Practices

1. Use the learned scales when possible: The learned QAT quantizer gives the best accuracy (Avg. 43.9). Switch to RTN/GPTQ only when your deployment toolchain requires it — the accuracy cost is small but non-zero.

2. Keep the grid consistent: The checkpoint was trained against a per-channel symmetric INT4 grid with 16-bit activations (W4A16). Exporting to a different group size or asymmetric grid discards the learned quantization structure.

3. Termination tokens: Same as the full-precision model — the Instruction model terminates with <EOT>, the Base model with <EOS>.

4. Model capabilities: Task capability follows Opt.Gear-1B — Korean-English bilingual understanding and instruction following; not intended for code generation. See the base model card for full benchmark results and architecture details.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{optgear2026,
    title  = {{Opt-Gear} Technical Report},
    author = {{Opt.Gear Team}},
    year   = {2026},
    url    = {https://arxiv.org/abs/2608.01034}
}

---

Correspondence: contact@opt-ai.kr · Hugging Face: huggingface.co/OptGear