## 6 Governing Software
### Programming Languages and Frameworks
This raises the question of whether it is worth transitioning to C++, which can be a time-consuming solution and requires significant effort while leading to only marginal improvements in throughput. A practical alternative is the so-called (informally) C+, which is not an existing programming language, but rather an adaptation of C syntax (wherever possible) to C++ headers. While Python is very convenient for prototypical programming, it is not the best choice for fully exploiting next-generation server CPUs, which can feature up to 128 cores and require a large amount of memory. For such tasks, multi-thread programming in C/C++ with OpenMP [69] is a very effective solution in terms of both performance and simplicity (parallelizing a structured loop with OpenMP simply requires introducing a single additional instruction). This is also an appealing possibility for edge devices: Arm CPUs, such as those onboard the Nvidia SoM/SoC listed in Table 4, are multicore architectures that can greatly benefit from the parallel implementation of algorithms. This is especially relevant given that the CPU performance of SoM/SoCs is often less consistent than that of their GPU counterparts. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**Class** & **Device** & **AI (TOPS) 1** & **TDP (W) 1** & **RAM (GB) 1** & **Cost (G)** \\ \hline \multirow{4}{*}{GPU} & Nvidia 4060 & 353 & 160 & 16 (GDDR6) & 430 \\ \cline{2-6}  & Nvidia 4070 & 706 & 285 & 16 (GDDR6\(\times\)) & 800 \\ \cline{2-6}  & Nvidia 4080 & 836 & 320 & 16 (GDDR6\(\times\)) & 1050 \\ \cline{2-6}  & Nvidia 4090 & 1321 & 450 & 24 (GDDR6\(\times\)) & 1800 \\ \hline \multirow{4}{*}{SoM/SoC} & Nvidia Jetson Nano & 0.5 & 10 & 4 (LPDDR4) & 240 \\ \cline{2-6}  & Nvidia Jetson TX2 & 1.3 & 20 & 8 (LPDDR4) & 420 \\ \cline{1-1} \cline{2-6}  & Nvidia Jetson Xavier & 21 & 20 & 16 (LPDDR4\(\times\)) & 720 \\ \cline{1-1} \cline{2-6}  & Nvidia Jetson AGX Orin & 275 & 60 & 64 (LPDDR5) & 2250 \\ \cline{1-1} \cline{2-6}  & Nvidia Jetson Orin Nano & 40 & 15 & 8 (LPDDR5) & 550 \\ \hline TPU & Coral TPU & 4 & 2 & - & 80 \\ \hline FPGA & AMD Versal AI Edge VEX280 & 114 (\({}^{\text{\tiny{(a)}}}\)) & 75 & 12 & 6900 \\ \hline SoM & New CPUs Q4 2024 & up to 120 & 17-37 & 16/32 & - \\ \hline \hline \end{tabular}  
* AI (TOPS): maximum performance with AI tasks; TDP (W): thermal design power, maximum amount of heat that the device can dissipate under normal operating conditions; RAM (GB): amount and type of RAM memory; cost (€): approximate device cost as of 30 June 2024. \({}^{\text{\tiny{(a)}}}\) BFLOAT16, extrapolated.  
\end{table} 
Table 4: Comparison of GPUs, SoMs/SoCs, one ASIC commonly used for AI inference, and one reference FPGA. For each model, the highest-performing version has been considered, with the exception of the Nvidia Orin Nano. TOPS: tera operations per second. Data for the Nvidia GeForce, Nvidia Jetson, Coral USB and FPGA were taken from [70; 71; 72; 73]. Rugged versions are available at an increased cost. The last row refers to newly announced consumer CPUs (SoMs, System-on-a-Module) expected to arrive on the market soon, with the Intel Core Ultra 9 used as a term of comparison.