# A Systematic Literature Review on Distributed Machine Learning in Edge Computing
## 4 Answering the RQs
### RQ2--Techniques and Strategies
#### 4.2.8 Model Compression  
Model compression is an efficient technique for deploying NN models on edge devices with limited resources by altering the network architecture itself in an attempt to reduce its parameters, and therefore its size. In this context, [PERSON] et al. [105] propose AutoML for Model Compression (AMC), using reinforcement learning and achieving model compression in a fully automated way. [PERSON] et al. [15] perform the compression by removing neurons with low contribution in a network, thus reducing the model size while maintaining its accuracy. [PERSON] et al. [46] seek deep compression with methods for parameter sharing and pruning, low-rank approximation, and knowledge transfer [106; 107]. Deep compression has also achieved remarkable results in SqueezeNet [108]. Bearing the domain of Transportation Cyber-Physical Systems (T-CPS) insight, [PERSON] et al. [32] introduce a lightweight deep learning model to support MEC applications in the field, aiming to reduce latency and improve context-awareness. It uses Factorization Convolutional (FC) layers, alternating with compression layers, which is named as lightweight CNN-FC by the authors. Experimental results indicate that lightweight CNN-FC significantly decreases the number of unnecessary parameters and model size while maintaining high accuracy compared to conventional CNN. The Semi-Parallel DNN (SPDNN) [109] combines a number of deep architectures to produce a final model that takes advantage of specialized layers of each architecture while being much smaller than the combined sizes of these networks. A traditional approach for compressing NN models is called pruning, which seeks to remove redundancies in over-parameterized networks [65], thus reducing the number of parameters without significant impact on the results. Different pruning techniques can be applied either in training [110; 111] or during the inference stage [64; 112]. Model compression can also be attained with quantization [113] by representing weights and activations of a NN with reduced precision. For instance, [PERSON] et al. [45] use data quantization to reduce the numerical representation of weights and activations from 32 to 16 bits, as well as modify the receptive field of max-pooling layers from \(3\times 3\) to \(2\times 2\), producing the same final results. The extreme use of quantization leads to Binary NNs (BNNs), where the activations and weights are reduced to binary representations, with massive reductions in resource usage and costs for edge computing. Nonetheless, binarization must be applied very carefully to not prompt drastic performance and scalability issues in complex tasks. [PERSON] et al. [55] propose assessing the degree of redundancy of each layer before applying binarization since its use on layers with a higher degree of redundancy will ultimately lead to lower performance loss, while layers with a negative degree of redundancy should be kept un-binarized. Several algorithms fail to counter the degradation caused by binarizing weights and activations completely. To confront this problem, Hybrid-Net [65] applies the Principal Component Analysis (PCA) in a reverse manner: instead of its traditional use as a dimensionality reduction technique, PCA is used here to identify layers that contribute to the most relevant transformations, based on their ability to expand data into higher-dimensional space, where it could be linearly separable. Then the bit precision in the inputs and weights of these significant layers increases. At the same time, the remaining layers are kept entirely binary, producing a successful mixed-precision network topology in the challenge of optimized and highly accurate quantized NNs with binary representations. [PERSON] et al. [49] explore the connection between Binary and Spiking Neural Networks (SNNs) to seek a reduction in computation time and a considerable reduction in model size, relying on competitive accuracies of SNNs in large-scale image recognition datasets, such as ImageNet.