Before revision:  In this work, we propose a novel integer-only quantization scheme for Transformer based models that quantizes the entire inference process. In particular, we demonstrate how to approximate nonlinear operationsin Transformer architectures , e.g., GELU, Softmax, and Layer Normalization, with lightweight integer computations.
After revision:  In this work, we propose a novel integer-only quantization scheme for Transformer based models that quantizes the entire inference with integer-only arithmetic. Based on lightweight integer-only approximation methods for nonlinear operations , e.g., GELU, Softmax, and Layer Normalization, with lightweight integer computations.