# Lossless Compression of Sensor Signals Using an Untrained Multi-Channel Recurrent Neural Predictor
## 4 Experiments
### Results of Different Recurrent Units  
In this sub-section, the performance of the proposed MCRU as a probability predictor for sensor signals is evaluated and is compared to classic recurrent units, including LSTM and GRU. The two classic recurrent units were implemented based on the open-source code of tensorflow-compress. As a general-purpose compressor, the predictor resolution of tensorflow-compress is fixed to 8, and we stacked 2 recurrent layers for its overall RNN architecture in this experiment. For fair comparison, we set the channel number to 2 in the proposed MCRU. Other hyper-parameter settings can be found in Table 4. We use BPC, which is defined in Equation (3), as the performance metric. The results are reported in Table 5. We observed that all three recurrent units have stable performance over all signals. Specifically, the standard deviation over 10 runs with different random seeds for each model-signal pair within 0.005; thus, only median BPC is reported. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Sampling Rate** & **Resolution** & **Sampling Time** \\ \hline BLOND & 50 KSa/s & 16 bits & 120 s \\ ESC & 44.1 KSa/s & 16 bits & 105 s-190 s 1  \\ MPACT & 20 GSa/s & 16 bits & 0.25 ms \\ MPACT-14 bits & 20 GSa/s & 14 bits & 0.25 ms \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The detailed information of the sensor datasets used in this paper. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Name** & **Version** & **Method** \\ \hline Gzip & v1.5 & Dictionary coding + Huffman coding \\ BSC & v3.1 & Block-sorting compression \\ PAQ & 8L & Context mixing algorithm \\ CMIX & v18 & Context mixing algorithm + LSTM \\ tensorflow-compress (TC) & v3 & RNN based predictor + arithmetic coding \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Details of the compared compressors. As we can see from Table 5, the proposed MCRU outperforms the other two recurrent units consistently on different signals, except BLOND-5. This advantage is more obvious on signals that are more difficult to predict correctly. For example, compared to LSTM, MCRU has an average BPC advantage of **0.319** and **0.158** on ESC and MPACT signals, respectively. Furthermore, it can be clearly observed from Table 5 that tensorflow-compress suffers from severe performance decline on signals sampled from ADCs with a resolution that is not an integer multiple of 8. In such cases, re-grouping the bit sequence destroys the integrity of each signal point. On the other hand, MCRU is more robust, since its predictor resolution is exactly the same as the ADC resolution. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Hyper-Parameter** & **LSTM** & **GRU** & **MCRU** \\ \hline State size & 1024 & 1024 & 1024 \(\times\) 2 \\ Recurrent layers & 2 & 2 & 1 \\ Step length & 16 & 16 & 8 \\ Batch size & 256 & 256 & 256 \\ Channels \(C\) & - & - & 2 \\ Resolution \(R_{p}\) & 8 & 8 & \(R_{a}\) \\ \hline Optimizer & Adam & Adam & Adam \\ Start learning rate & 0.0005 & 0.0005 & 0.0005 \\ End learning rate & 0.0001 & 0.0001 & 0.0001 \\ Gradient clipping value & 5.0 & 5.0 & 5.0 \\ \hline Learnable parameters (\(\approx\)) & 15.2 M & 11.5 M & 10.0 M/8.1 M1 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Hyper-parameters used in the predictors of dynamic DRCs.