Fully Quantized Transformer for Improved Translation
State-of-the-art neural machine translation methods employ massive amounts of parameters.Drastically reducing computational costs of such methods without affecting performance has been up to this point unsolved.In this work, we propose a quantization strategy tailored to the Transformer architecture.We evaluate our method on the WMT14 EN-FR and WMT14 EN-DE translation tasks and achieve state-of-the-art quantization results for the Transformer, obtaining no loss in BLEU scores compared to the non-quantized baseline.We further compress the Transformer by showing that, once the model is trained, a good portion of the nodes in the encoder can be removed without causing any loss in BLEU.
We fully quantize the Transformer to 8-bit and improve translation quality compared to the full precision model.An 8-bit quantization method to quantize the machine translation model Transformer, proposing to use uniform min-max quantization during inference and bucketing weigts before quantization to reduce quantization error.A method for reducing the required memory space by a quantization technique, focused on reducing it for Transformer architecture.