# Optimization of Linear Quantization for General and Effective Low Bit-Width Network Compression
## 1 Introduction  
Due to their excellent data representation and feature learning ability, DNNs (deep neural networks) have achieved great success in a large number of tasks such as image classification, object detection, speech synthesis, and industrial and medical fields [1]. For example, [PERSON] et al. applied a DNN to speed control of three-phase induction [2] and blood diseases detection [3]. Their application scope has been extended to edge devices [4]. DNNs are high-complexity models that require huge energy consumption and memory cost. Due to factors such as security, privacy (e.g., medical data and industrial data), and latency (e.g., smart sensors, wearables, autopilot, and UAV tracking), inference of DNNs must be performed on edge devices, but the available computing resources are limited and subjected to strict power constraints [5]. Therefore, how to reduce the computation and storage while maintaining the accuracy of neural networks is a key issue in energy-sensitive applications. A fundamental way to decrease energy consumption and memory cost is to compress networks into lightweight models through pruning and quantization. In practice, quantization usually needs to be done after the models are first \"slimmed\" down by pruning to obtain an effective lightweight model because neural networks are often over-parameterized. In 2015, [PERSON] et al. proved that reasonable pruning would not affect the accuracy of the neural networks [4]. Pruning can reduce a large pre-trained model into a smaller model without retraining [6]. Figure 1 depicts the basic inference process of the neural network. Each neuron connects through synapses with multiple neurons on the upper layer and needs to perform multiple floating-point operations. However, a largepart of weights, i.e., the synapses, is redundant in the neural network. Not all operations will affect the reasoning ability of the network. [PERSON] et al. trained the model through structural learning with forgetting (SLF), eliminating weak connections between neurons during training and revealing the most influential injection molding parameters, while the performance of the model did not decline significantly [7]. Through pruning, the weight matrix can be converted into a sparse matrix and then many calculation tricks can be applied to improve the inference delay [8]. Pruning may result in a structured or unstructured network [6]. Unstructured pruning can prune any weight and can be considered fine-grained pruning. Compared with structured pruning, it does not generate irregular matrices while maintaining higher accuracy. This paper focuses on DNN weight quantization, which pays more attention to the accuracy of the model before and after pruning. Therefore, this paper will use fine-grained pruning to pre-process the model. After the model is pruned, it can be further \"slimmed\" by converting floating-point weights into low-precision values through quantization. The quantized neural network has been proven to achieve a significant inference acceleration without a major drop in accuracy [9]. There are several types of quantization schemes according to the different quantization bit-widths. For example, [PERSON] et al. proposed BNNs (binary neural networks) to replace floating-point weights with 1 bit [10]. [PERSON] et al. proposed to use XNOR-Net to quantify both activation and weight to 1 bit based on BNN [11]. [PERSON] et al. proposed TWNs (ternary weight networks) to quantize the weights into 2-bit weights [12]. In general, binary and ternary quantization schemes are associated with a specific network and have no generality. This paper endeavors to develop an effective quantization method that applies to general neural networks. The quantization methods of DNN can be classified into two types: quantization aware training and post-training quantization. Quantization-aware training is to fine-tune the pre-trained model employing backpropagation when the floating-point weights are converted into fixed-point numbers [13].