# Train Me If You Can: Decentralized Learning on the Deep Edge
## 2 Background
### Quantization  
The training of ML models typically performs over 32-bit floating-point data. However, the same precision is usually not required during inference [63]. Over the last several years, researchers have shown that mapping floating-point tensors to integer tensors, with lower bit-width, and performing computations on them can lead to results equivalent to the traditional approach [64]. This process, known as quantization, involves the encoding of (i) the sign, (ii) the integer part, and (iii) the fractional part of a float in a single integer value. Quantized values are typically represented in \(Qm.n\) format, where \(m\) specifies the number of bits for the integer part and \(n\) the number of bits for the fractional part. Quantization not only decreases the inference latency but also reduces the memory footprint of ANNs [65]. Consequently, researchers are studying the application of quantized ANNs to develop low-power and smart applications in a variety of fields, including image processing and segmentation, keyword spotting, and predictive maintenance [44]. Independently of the field of application, quantization must always attend to the ISA of the platform where the ANN will be deployed. This is fundamental to speed up memory access and math operations. **Quant. aware training vs. Post-training quant. ** As quantization reduces the number of bits to represent a value, saturation is a common problem that affects the accuracy of a quantized ANN. A common strategy to address this problem is quantization-aware training, in which the quantization error is considered as part of the loss returned by the loss function [66; 67; 68]. Nevertheless, this comes at the cost of increased overhead and latency in the training pass. In contrast, post-training quantization does not incur additional overhead during training time, as quantization is only performed after the training process. **Scaler and zero-point:** The quantization of a floating-point value always requires scaling and zero-point factors [66; 67]. Scalers are typically calculated as detailed in Equation (6) and require the previous definition of the quantization format (\(Qm.n\)) and the range of floating-point values to be quantized. The zero-point is usually calculated as detailed in Equation (7) and is always rounded to an integer value:  
\[scaler=\frac{x_{max}-x_{min}}{2^{n}-1} \tag{6}\]  
\[zero=-round(x_{min}*scaler)-2^{n}-1 \tag{7}\]  
**Quantization granularity:** The quantization granularity must consider two metrics: (i) impact on model accuracy and (ii) computational cost [66; 67]. Computing the scaling factor to each weight and activation leads to negligible accuracy loss. Nevertheless, this is unfeasible as each neuron requires a scaling operation during inference. Besides increasing the decision latency, it also tremendously increases the memory footprint of the quantized ANN, as scaling and zero factors need to be stored for each weight. The most common quantization strategies consider a layer-by-layer or filter-by-filter granularity [66; 67]. **Fixed bit-width vs. mixed bit-width:** In fixed bit-width, the number of bits to represent a given weight or activation is the same in the whole ANN [66; 68]. In contrast, in a mixed bit-width setting, the number of bits to represent weights and activations may vary between layers or filters, depending on the quantization granularity [66; 68]. **Static vs. dynamic quantization:** In static quantization, the quantization format is set before inference, using a representative dataset [66; 67]. In contrast, dynamic quantization computes the quantization format during the inference process for each input [66; 67]. Therefore, model parameters and activations are stored in low-precision bit-width, but the operations are performed in floating-point data. For each calculus, this approach requires the dequantization of the inputs and the quantization of the outputs.