# Energy-Efficient Respiratory Anomaly Detection in Premature Newborn Infants
## 4 Model Quantization  
CNN models consume a considerable amount of energy due to their high computational complexity. The high energy consumption is due to the large model size involving several thousand parameters. It is therefore challenging to deploy the inference, i.e., a trained CNN model on battery-powered mobile devices, such as smartphones and wearable gadgets due to their limited energy budget. To address this high energy overhead, energy-efficient solutions have been proposed to reduce the model size and computational complexity. Some common approaches include the pruning of network weights [33] and low bit precision networks [34]. We focus on the latter techniques. Specifically, we implement both bit precision weights and activations to reduce model sizes and computational power requirements. To perform the training of a CNN with low-precision weights and activations, we use the following quantization function to achieve a k-bit quantization [35]. \[Z_{q}=Q(Z_{r})=\frac{1}{2^{k}-1}\operatorname{round}\Bigl{(}(2^{k}-1)Z_{r} \Bigr{)}, \tag{8}\] \begin{table}
\begin{tabular}{c l} \hline \hline Learning rate & 0.001 \\ \hline Batch size & 5 \\ \hline Optimizer & Adam \\ \hline Data shuffle & per epoch \\ \hline Maximum epochs & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of hyperparameters for the proposed 1 DCNN model. where \(Z_{r}\in[0,1]\) is the full precision value and \(Z_{q}\in[0,1]\) is the quantized value obtained using the k-bit quantization. The quantization of weights is given by  
\[w_{q}=Q\bigg{(}\frac{\mathtt{tanh}(w_{r})}{2\cdot\mathtt{max}(|\mathtt{tanh}(w_{ r})|)}+\frac{1}{2}\bigg{)}, \tag{9}\]  
where \(w_{r}\) is the original weight using full precision and \(w_{q}\) is the quantized value using k-bit quantization. The quantization of activations is given by  
\[x_{q}=Q(f(x)), \tag{10}\]  
where \(f(x)=\mathtt{clip}(x_{r},0,1)\) is the clip function bounding the activation function between 0 and 1. In this paper, we apply both bit precision techniques for both weights and activations using quantization with the QKeras library, which is a quantization extension to the Keras [36]. It enables a drop-in replacement of layers that are responsible for creating parameters and activation layers like the Conv 1D, Dense layers. It facilitates arithmetic calculations by creating a deeply quantized version of a Keras model. We tag the variables, weights, and biases created by the Keras implementation of the model and the output of arithmetic layers by quantized functions. Quantized functions are specified as layer parameters and then passed as a cumulative quantization and activation function, QActivation. The quantized bits quantizer used above performs mantissa quantization using the following equation. \[\mathtt{mantissa\ quantization}= \tag{11}\] \[2^{b-k+1}\cdot\mathtt{clip}\Big{(}\mathtt{round}(x_{r}\cdot 2^{k-b- 1}),-2^{k-1},2^{k-1}-1\Big{)}\]  
where \(x\) is the input given to the model, \(k\) is the number of bits for quantization, and \(b\) specifies how many bits of the bits are to the left of the decimal point. We conduct our experiment to perform quantization of our Conv1D model using 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, and 64 bits. We observe the performance accuracy increase with the increase in the quantization bits, with 2 bits achieving an 88.93% compared to using all the 64 bits achieving 97.15% (see the detailed results in Section 6.2). The QTools functionality is used to estimate the model energy consumption for the different bit-wise quantization implementations. It estimates a layer-wise energy consumption for memory access and MAC operations in a quantized model derived from QKeras. This is helpful when comparing the power consumption of more than one model running on the same device. The model size is calculated as the number of model parameters multiplied by the number of bits used in each scenario. We observe that when we increase the number of bits, the model size increases as well as the accuracy, but so does the consumption of energy (pJ).