Before revision:  In particular, we demonstrate how to approximate nonlinear operationsin Transformer architectures , e.g., GELU, Softmax, and Layer Normalization, with lightweight integer computations. We use those approximations in our method, I-BERT , with an end-to-end integer-only inference, and without any floating point calculation.
After revision:  In particular, we demonstrate how to approximate nonlinear operationsin Transformer architectures , e.g., GELU, Softmax, and Layer Normalization, I-BERT , with an end-to-end integer-only inference, and without any floating point calculation.