# Comparing Application-Level Hardening Techniques for Neural Networks on GPUs
## 3 Proposed Evaluation Methodology
### Error Models
#### 3.1.1 Weight Single Bit-Flip  
This error model is widely used to validate the effectiveness of SW HTs [14] by simulating faults in the memory shared among different threads of the same SM. As pointed out in [65], WSBFs FI can provide preliminary hints for the reliability assessment in front of measures, such as error coverage and error resilience. The number of faults to be injected is computed by resorting to the statistical method introduced by [66] and detailed in Equation (1).  
\[n=\frac{N}{1+e^{2}\times\frac{N-1}{t^{2}\times p\times(1-p)}} \tag{1}\]  
Starting from an initial population size \(N\), the resulting total number of faults \(n\) satisfies an error margin \(e\), the cut-off point \(t\) corresponding to a confidence level, and the probability \(p\) of randomly picking a specific value of the features describing the fault. When weights are corrupted, the characteristics of a fault are: _kernel_\(K\), _channel_\(Ch\), _row_\(R\), _column_\(Col\), and _bitmask_\(b\), which collectively define the exact position of the bit to flip in the parameters tensor to perturb.  
(\(K\),\(Ch\),\(R\),\(Col\),\(b\)) is a vector of integers, and one of the assumptions behind Equation (1) requires that  
* K \(\sim\) Uniform[0,batch_size],
* Ch \(\sim\) Uniform[0,tensor_depth],
* R\(\sim\) Uniform[0,tensor_width],
* Col \(\sim\) Uniform[0,tensor_height],
* b \(\sim\) Uniform[LSB,MSB].  
Data (i.e., weights and entries of the intermediate FMs) are represented in \(FP32\) format and, according to the IEEE Standard for Floating-Point Arithmetic (IEEE 754), a generic FP32 encoded number \(x\), in binary scientific notation, can be written as shown in Equation (2).  
\[x=(-1)^{\text{sign}}\times 2^{\text{exponent}-127}\times(1.\text{fraction}) \tag{2}\]  
where \(sign=1\) if \(x\) is negative, and \(sign=0\) if \(x\) is positive, then _exponent_ is an 8-bit unsigned integer with a bias of 127, and in the end, the fraction represents the mantissa of the \(x\) binary encoding. As we can see from the equation, a bit-flip in the mantissa does not affect the real representation of \(x\), while a bit-flip performed on the exponent would induce a high error. To support this statement, several works [31; 67] have evaluated the effect of bit-flips on the binary encoding of the weights/neurons of the NN exploring the bit-wise impact on error percentage as well. Consequently, due to the negligible induced error by performing a bit-flip on the mantissa bits, as for \(b\), we set the least significant bit (LSB) to 19 and the most significant bit (MSB) to 31, so as to consider the sign bit, the exponent bits, and the mantissa bits from 23 to 19. However, for what concerns the upper bounds of the ranges from which the components of (\(K\),\(Ch\),\(R\),\(Col\)) are extracted, _batch_size_, _tensor_depth_, _tensor_width_, and _tensor_height_ represent the dimensions of the parameter tensors to corrupt in case the injection occurs in a convolutional layer.  
After extracting a sufficient number of faults and generating a fault list, we can use PytorchFI to perturb the weight parameters at run-time. At the end of each FI campaign, a specific evaluation process is performed, classifying faults into different categories according to their effects.