# A Novel Low-Complexity and Parallel Algorithm for DCT IV Transform and Its GPU Implementation
## 4 Implementation
### Overview of CUDA Used Architectures  
NVIDIA developed CUDA as a general-purpose parallel computing platform and as a programming model that can accelerate intensive applications that are executed on GPUs. CUDA cores are the most basic floating-point unit, part of an NVIDIA GPU, and are able to perform one operation per clock cycle. The CUDA cores are grouped inside a GPU on several Streaming Multiprocessors (SMs). All GPUs on which we tested the algorithm, existing on all the development systems, are CUDA capable and belong to two different architecture generations: Volta and Ampere. The CUDA cores for both architectures support mathematical operations on INT8, FP16, FP32, and FP64 precisions. Each NVIDIA architecture brings new improvements and new concepts regarding the previous architectures. For example, NVIDIA improved the design for the SM with new data path placement, improved instruction scheduler, enhancements on workload balancing, improvements to control logic partitioning, more efficient memory accesses, \begin{table}
\begin{tabular}{c c c} \hline \hline
**Development System:** & **Jetson AGX Xavier** & **Jetson Orin Nano** \\ \hline \multirow{8}{*}{GPU:} & Architecture & Volta & Ampere \\ \cline{2-3}  & SM Count & 8 & 8 \\ \cline{2-3}  & CUDA cores & 512 & 1024 \\ \cline{2-3}  & Tensor cores & 64 & 32 \\ \cline{2-3}  & L1 Cache/SM & 128 KB & 128 KB \\ \cline{2-3}  & L2 Cache & 512 KB & 256 KB \\ \cline{2-3}  & AI performance (INT8) & 32 TOPS & 40 TOPS \\ \cline{2-3}  & Floating point (FP32) & 1.41 TFLOPS & 1.28 TFLOPS \\ \cline{2-3}  & Min. Frequency & 114 MHz & 306 MHz \\ \cline{2-3}  & Max. Frequency & 2.265 GHz & 624 MHz \\ \hline \multirow{8}{*}{CPU:} & \multirow{2}{*}{Processor} & 8-core & 6-core \\  & NVIDIA Carmel & Arm Cortex-A78 AE \\ \cline{1-1}  & & ARM v8.2 & v8.2 \\ \cline{1-1} \cline{2-3}  & & 32 GB 256-bit, & 8 GB 128-bit, \\ \cline{1-1}  & Memory & LPDDR4x, & LPDDR5, \\ \cline{1-1}  & & 136.5 GB/s & 68 GB/s \\ \cline{1-1} \cline{2-3}  & CPU max. frequency & 1.2 GHz & 1.5 GHz \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The main features of the development systems used in this research. etc., in order to increase the efficiency of the previous architecture. Compared with the earlier architecture, the Volta architecture increases the number of CUDA cores per SM to 64, while the Ampere architecture supports 128 CUDA cores per SM. Based on the load of the GPU, their frequency can be varied between a minimum and a maximum limit, as shown in Table 2. Another factor differentiating NVIDIA system architectures lies in the L1 and L2 memory related to each SM--Table 2. However, along with the increased L2 capacity, the bandwidth of the L2 cache to the SMs is also different in different architectures. So, the GPU performance is mainly determined by the number of CUDA cores, the clock speed of each core, the architecture of the cores, and data throughput. Different algorithms (e.g., AI algorithms) require the use of numbers with fewer fraction bits (into the float representation) for obtaining a balance between the computationally expensive FP32 representation (single-precision floating-point format imposed by IEEE/54 standard [45]) and the reduced precision and range of the half-precision FP16 representation (also enforced by IEEE/54 standard). As a direct result, the Google company developed the BFloat16 format to address this problem. The Ampere architecture is the first one from the NVIDIA family of processors able to manipulate silicon numbers in BFloat16 precision, Table 3.  
One of the main differences between the different NVIDIA architectures, presented in Table 2, is the presence or absence of the Tensor Core units. These units were introduced for the first time in the Volta architecture and could matrix Fused Multiplication Addition and add computation very quickly. In this mode, two 4 \(\times\) 4 matrices are multiplied and added to a 4 \(\times\) 4 matrix.