  Deep learning models typically use single-precision (FP32) floating point
data types for representing activations and weights, but a slew of recent
research work has shown that computations with reduced-precision data types
(FP16, 16-bit integers, 8-bit integers or even 4- or 2-bit integers) are enough
to achieve same accuracy as FP32 and are much more efficient. Therefore, we
designed fbgemm, a high-performance kernel library, from ground up to perform
high-performance quantized inference on current generation CPUs. fbgemm
achieves efficiency by fusing common quantization operations with a
high-performance gemm implementation and by shape- and size-specific kernel
code generation at runtime. The library has been deployed at Facebook, where it
delivers greater than 2x performance gains with respect to our current
production baseline.
