# Efficient FPGA Implementation of an ANN-Based Demapper Using Cross-Layer Analysis
## 6 Comparison
### FPGA Performance Compared to General-Purpose Processors  
To demonstrate the advantage of our FPGA design, we compare our implementation to the same network executed on general-purpose processors. As target platforms, we choose the high-performance GPU Nvidia RTX 2080, the embedded GPU Nvidia Jetson AGX and the embedded CPU ARM Cortex-A53. For a fair comparison, we implement only the decoder part of the AE on the general-purpose processors in a PyTorch environment. We compare throughput, energy efficiency and power consumption for inference and training. In contrast to our FPGA architecture, the throughput of those platforms increases dramatically with higher batch size; thus, we increase the batch size up until the device runs out of memory. It is to note that for the application of demapping communication symbols in low-latency communication, such high batch sizes are not practical, because the data enter the system sequentially and waiting for a complete batch would result in high latencies. However, for the sake of completeness and to give a baseline for other use cases, we also provide results for high batch sizes. Figure 13 shows the throughput of the high-performance GPU, embedded GPU, and embedded CPU compared to the power- and latency-optimized FPGA implementation. For a batch size of one, the throughput of the ARM Cortex-A53 is 495 samples/s for inference and 118 samples/s for backpropagation. The graphics processor units (GPUs) Jetson AGX and Nvidia RTX 2080 achieve 1450 and 4670 samples/s for inference as well as 282 and 978 samples/s for training. In contrast, the FPGA outperforms the high-performance GPU for very small batch sizes with \(1.23\cdot 10^{7}\) samples/s for inference and \(3.75\cdot 10^{6}\) samples/s for backpropagation by a factor of 2000 and 3800, respectively. The high parallelism of the RTX 2080 is only exploited with increasing batch size and leads to higher throughput than \begin{table}
\begin{tabular}{c c c c c} \hline  & **Batch** & **Power** & **Throughput** & **Efficiency** \\  & **Size** & **[W]** & **[GOPS]** & **[GOPS/W]** \\ \hline
[22] & 128 & 24.7 & 1220 & 38.13 \\
[23] & 128 & 12.4 & 276.7 & 22.3 \\
[29] & 16 & 14.24 & 86.12 & 6.05 \\
[30] & 40 & 50.5 & 479 & 9.49 \\
[31] & 1 & 0.67 & 4.39 & 6.55 \\ Ours & 1 & 0.568 & 5.28 & 9.656 \\ \hline \end{tabular}
\end{table} 
Table 1: Comparison to state-of-the-art FPGA training frameworks. the FPGA starting from batch size 10,000, which is far from being practical for low-latency communication systems. The power measurements are based on nvidia-smi for the graphics processor units (GPUs) and obtained with an external power meter for the CPU. Similar to the FPGA, the idle power is subtracted from the processing power to compute the dynamic power consumption. As shown in Figure 14, the FPGA consumes between 0.077 W and 0.568 W of power, whereas the high-performance GPU draws up to 259 W and the embedded GPU up to 9.8 W. Only the embedded CPU's power consumption lies in the same order of magnitude as the FPGA's, with 0.4 to 1.4 W.  
The dynamic energy consumption is calculated as the product of dynamic power and the average runtime of one sample. As illustrated in Figure 15, the FPGA outperforms all other platforms up to batch size 100,000, where only the Nvidia Jetson AGX is more energy efficient. For very small batch sizes, there is a huge gap between the Ultra96 and all other platforms. It consumes around 5000 times less energy than the embedded CPU. Those results show that general-purpose processors do not always provide an optimal platform for the inference and training of ANNs. In contrast to central processing units  
Figure 14: Power comparison. Figure 13: Throughput comparison. Figure 15: Energy comparison. (CPUs) and graphics processor units (GPUs), the flexibility of FPGAs enables a high level of specialization, because the DOP can be adapted to the constraints and requirements, which is exploited by our framework.