# Applying Reinforcement Learning to Protect Deep Neural Networks from Soft Errors
## 4 Case Study
### Ablation Study of the Proposed Method
This is because Top-k protects only a few bits, the same number as the DRL-based agent shown in Figure 7, which is fewer than the number protected by the MSB scheme. Beyond these baseline methods, the proposed method significantly decreases the SDC ratio under different BERs, improving the robustness of the DNNs. Compared to baseline methods that only protect a few crucial or random bits, we conclude that this improvement stems from the reward design of the RL-based agent, which effectively explores vulnerable bits within weights. In particular, the proposed method shows better performance than the Top-k scheme, which protects the same number of bits, indicating a higher protection efficiency of the proposed method. For MLP or ResNet-18 at \(r_{b}=20\%\), the average performance of the proposed method decreases from 98.9% to 97.67% and from 89.60% to 80.80% when protecting the critical layers, including Layers 1 and 2 of the MLP and Layers 1, 7, 8, and 9 of the ResNet-18. These results still significantly outperform those of the baseline schemes. Meanwhile, the SDC ratios significantly decrease to 1.24 within MLP and to 7.54 within ResNet-18. In addition, our method performs superior results in ResNet-18, implying that it is more suitable for neural networks with deeper layers. To further evaluate the propose methods, we illustrate the overall performance of the DNNs using both the proposed method and the baseline methods across the varying soft error rates in Figure 8. Although these methods demonstrate similar performance in the MLP with \(r_{b}=10\%\), as depicted in Figure 8a, the proposed method outperforms them as \(r_{b}\) increases. In particular, the proposed methods demonstrate significant improvement compared to the baseline methods across ResNet-18 in Figure 8c,d, suggesting that the proposed framework is more efficient and effective as the DNNs adopt a more complex model. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \multirow{2}{*}{**DNN Type**} & \multirow{2}{*}{\(r_{b}\)} & \multicolumn{6}{c}{**Average Performance**} & \multicolumn{6}{c}{**SDC Ratio**} \\  & & & \multicolumn{3}{c}{**Protecting in Critical Layers (\%)**} & \multicolumn{6}{c}{**Across the DNN**} \\ \cline{3-11}  & & **Our** & **Without** & **Hamming** & **Code** & **MSB** & **Top-k** & **Our** & **Without** & **Hamming** & **Code** & **MSB** & **Top-k** \\ \hline \multirow{3}{*}{MLP} & 10 & **98.30** & 94.32 & 98.26 & 98.16 & 96.83 & **0.47** & 2.92 & 0.50 & 0.62 & 1.40 \\ \cline{2-11}  & 20 & **97.67** & 70.55 & 93.91 & 90.35 & 86.61 & **1.24** & 17.30 & 3.50 & 5.64 & 8.15 \\ \hline \multirow{3}{*}{ResNet-18} & 10 & **83.60** & 62.98 & 70.29 & 65.89 & 76.63 & **5.55** & 14.63 & 11.17 & 12.93 & 9.91 \\ \cline{1-1} \cline{2-11}  & 20 & **80.80** & 32.91 & 70.98 & 66.61 & 71.11 & **7.57** & 31.99 & 10.92 & 18.03 & 9.93 \\ \hline \end{tabular}
\end{table} 
Table 1: Performance evaluation between different protection schemes. The golden run performances \(\mathcal{P}_{g}\) of MLP and ResNet-18 are 98.90% and 89.6%.