# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 2 Background
### GPU Organization and Fault to Error Propagation  
Neural networks are computationally intensive algorithms often characterized by time-consuming and resource-demanding processes; consequently, hardware accelerators, such as GPUs, are generally employed to boost NN operations in parallel, reducing their execution latencies. In particular, the hierarchical structure of the GPU, as illustrated in Figure 2 and presented in [46], improves the execution of parallel tasks efficiently. The left side of Figure 2 represents the single-instruction multiple-thread (SIMT) programming model of the GPU. The program organizes the threads in grids and blocks. The blocks (B0, B1, B3) are collections of threads, and a grid (G0) is a collection of blocks. On the right hand side of Figure 2, the GPU hardware components are depicted: the GPU scheduler defines the policy for distributing the threads to the physical components. Each streaming multiprocessor (SM) executes to a thread block (TB), while all SMs share the GPU device memory. An SM includes local memories and register files to support the parallel thread execution. Subsequently, the SM schedules several warps into the scalar/streaming processor (SP) of fused multiply-accumulate (FMA) core to perform integer, floating-point, and trigonometric operations. The description of the hierarchical representation provides useful insights to better understand the error propagation induced by a physical defect throughout the GPU architecture. Despite the benefits in terms of the performance stemming from the execution of parallel programs, as discussed in Section 1, physical defects in the semiconductors of GPUs and their propagation are primary sources of faults. On the left hand side of Figure 3, the propagation of the error induced by physical defect to the application level is depicted. It illustrates the varying levels of impact the defect can have, affecting both hardware and software. This supports the intuition behind the NBER error model that we used for our evaluations, which describes the errors occurring in the output of the fused multiply-accumulate cores during the execution of matrix multiplication algorithms. Physical defects can occur in a system due to manufacturing flaws or external factors that disrupt the GPU's operation, either temporarily or permanently. Specifically, these defects may trigger and propagate as hardware faults (for example, corrupting the logic state on a gate) across the microarchitecture during the execution of an application, leading to system failures or errors. Figure 2: Hierarchical GPU organization [46]. In more detail, corrupted logic states can significantly impact the execution of assembly instructions, leading to incorrect implementations of algorithms, such as the matrix multiplication algorithm. As shown on the right-hand side of Figure 3, the algorithm divides inputs, feature maps, and weight arrays into smaller submatrices known as tiles, which are then distributed among the GPU's parallel cores. The computation is carried out through several TB computations, which are further divided into smaller tiles at the warp level within the GPU's SMs. This tiling method requires each thread in a warp to compute up to four small matrix multiplications between tiles, each of size \(4\times 4\)[47]. When a single SM has a faulty FMA core, errors can propagate to multiple threads within a warp, resulting in data corruption at the output of the tile. Consequently, if more than one tile is processed by the faulty SM, a similar error pattern will likely affect the results of the algorithm. However, the structural organization of the hardware or the implicit nature of the code in the application may mask some of these effects, meaning they can go unnoticed if they do not produce a visible corruption in the application's outputs or the system itself.