# Applying Reinforcement Learning to Protect Deep Neural Networks from Soft Errors
## 4 Case Study
### Ablation Study of the Proposed Method  
The bits highlighted by the bit masks could be protected by different practical schemes such as the BCH code and Hamming code. We utilize a similar comparative analysis as in [50,51] by selecting the Hamming Code for the ECC scheme to protect the bits within the weights. We use the MSB method and conventional Hamming code, which protects significant and arbitrary bits from soft errors, as a benchmark to compare with the proposed method. To further analyze the protection efficiency of the proposed framework, we also design a comparative scheme to protect \(k\) bits (referred to Top-k), where faults most frequently occur. The number of protected positions, \(k\), is the same as those selected by the DRL-based agents shown in Figure 4c,d. We primarily assess the proportion between information and redundant bits to present the effectiveness of these methods. Next, we introduce a SDC ratio to underscore the protection efficiency. Finally, we measure the performance of the DNNs under various soft error rates by adopting different protection schemes. All these evaluations are based on the following configurations: \(r_{e}^{i}\) follows the normal distribution, \(r_{b}\), with 10% and 20% to reveal that the SDC, which could be visible, affected the performance of the DNNs. We evaluate the efficiency of the protection schemes by modifying the metrics from [50; 51], where they measure the proportion between the information and redundant bits. \[\eta=\frac{k_{pro}\cdot(n_{i}-k_{i})}{k_{i}} \tag{11}\]  
where \((n_{i},k_{i})\) refer to a linear code with \(n_{i}\) bits and \(k_{i}\) information bits for a weight in the DNNs; \(k_{pro}\) refers to the protected bits in the information bits. Equation (11) indicates the redundancy rate to protect the same bits in a weight by using different schemes. With the same measurement in [50; 51], the average redundancy rate for the proposed method \(\eta_{mlp}=50.05\%\), and \(\eta_{resnet}=51.25\%\), while \(\eta_{msb}=56.24\%\), and \(\eta_{hamming}=75.00\%\). Since the Top-k scheme protects the same number of bits as the proposed methods, the average redundancy rate is the same as \(\eta_{mlp},\eta_{resnet}\). Due to the inherent robustness and probabilistic nature of the DNNs, comparing bit-by-bit could be invalid to reflect the degradation of the overall performance [4; 5; 6]. We propose a metric defined in [5; 44] as Equation (12) to further highlight and evaluate the protection schemes by normalizing the SDC ratio under the specific fault probability \(p_{e}\). A smaller SDC ratio indicates that a more robust DNN resists soft errors. \[p(SDC|p_{e})=\frac{\frac{\sum_{i=1}^{N_{t}}(\mathcal{P}_{g}-\mathcal{P}_{i})}{ N_{t}}}{\mathcal{P}_{g}}=\frac{\mathcal{P}_{g}-\tilde{\mathcal{P}}_{N_{t}}^{p_{e}}} {\mathcal{P}_{g}} \tag{12}\]  
where \(\mathcal{P}_{g}\) refers to the performance of the DNNs when they are error-free (golden run). \(N_{t}\) refers to the times of running DNNs with different protection schemes under the fault injection. We configure \(N_{t}=500\) across these experiments. \(\tilde{\mathcal{P}}_{N_{t}}^{p_{e}}\) is the same as Equation (2), referring to the average performance of the DNNs protected by different methods in \(N_{t}\) tests. Figure 7: Overall results by adopting transfer learning in different layers. The protected bit position is shown in the top of the bar. In Table 1, we list the SDC ratio which is computed by Equation (12). Although we observe that the Hamming code scheme outperforms other baseline solutions, it suffers a higher average redundancy rate than other baseline methods. On the other hand, the MSB and Top-k schemes demonstrate varying protection efficiency across different layers and DNN structures. Specifically, we observe that the performance of the MSB scheme tends to be better than that of the Top-k scheme in less sensitive layers of ResNet-18.