dsuman/gemma3-12b-SQ-GPTQ

Gemma3-12B (W8A8): SmoothQuant + GPTQ

Description

Our submission applies SmoothQuant and GPTQ to the Gemma 3 12B model, producing a W8A8 (8-bit weight, 8-bit activation) compressed model. SmoothQuant is first applied to reduce activation outliers by migrating some of the quantization difficulty from activations onto the weights, rescaling so that activations become easier to quantize at the cost of slightly harder-to-quantize weights. The weights are then quantized using GPTQ and activations using RTN quantization.

1. Set up the environment

bash setup.sh

2. Run inference

bash run.sh \
  --lang-pair ces-deu \
  --batch-size 512 \
  --input input.txt \
  --output output.txt

Arguments:

Flag	Description
--lang-pair	Language pair to translate. This submission only supports ces-deu (Czech → German).
--batch-size	Number of sequences processed per batch (max_num_seqs in vLLM).
--input	Path to a plain-text file with one source sentence per line.
--output	Path to write translations to — one line per input line, in the same order.