# Applying Reinforcement Learning to Protect Deep Neural Networks from Soft Errors
## 2 Related Work
### Protecting DNNs from Soft Errors  
Robustness has been considered as a key concept in the design of DNN protection. Previous research [4; 5; 6; 32; 33] refers to the robustness of DNNs as their capability to resist faults and maintain performance under various disturbances. The corresponding protection strategies for DNN systems can be broadly categorized as follows [33]:  
**Passive strategies** are characterized by a system that works without interfering with the faults. Such approaches usually design redundant modules (e.g., triple module redundancy) to filter the faulty values by voting for the major one [34; 35]. Considering the extensive parameters of DNNs, such a redundancy design could significantly increase the hardware dimensions. Some research selects critical neurons with redundant modules for fault tolerance [25; 35]. However, such approaches can become inefficient due to the challenges of identifying and addressing critical neurons in large-scale DNNs. Moreover, it is normally a challenging task to protect all potential critical neurons from various errors [4; 5]. Adding noise into the training data enhances the robustness of DNNs by avoiding overfitting [36]. Building on this training-based enhancement, recent studies also address the protection of DNNs against soft errors by introducing fault injections during training to improve robustness [37]. For example, a soft error library is developed to inject into the training dataset in [38]. However, these optimization-based approaches typically assume that faults occur in the input data rather than within the DNNs themselves (e.g., weights or biases of activation functions). This assumption may underestimate the impact of soft errors as DNNs inherently have the ability to filter certain unexpected inputs (e.g., values with extremely large magnitudes) through their activation functions. As an improvement, some works add regularization terms in loss functions to resist potential soft errors inside the DNNs [39]. These modifications incorporating regularization terms only provide a generic improvements of soft error resilience in DNNs. However, such a regularization may actually demand extensive iterations to converge the training process. **Active strategies** refer to the introduction of a specific mechanism that dynamically manages resources to mitigate the impacts of faults by addressing their locations [40]. One key effort is to detect errors in the DNN architectures. Some methods employ and optimize the checksum mechanisms for detecting errors [41] in the DNN accelerator. However, such an optimization requires additional computational overhead to implement the design [42]. Other research proposes to detect errors based on data-driven approaches (e.g., machine learning techniques [43; 44]). Such approaches usually need a complete comprehension of fault behaviors, requiring massive training data. Another effort in supporting active fault tolerance is to protect computational units within DNNs. Specifically, these efforts can be categorized as follows: (1) Layer-wise protection refers to analysis of layer-wise features from DNNs. For example, a median feature selection scheme [45] is proposed to filter the impact of bit errors from each layer of the DNNs. Similarly, some work [2; 46] reveals resiliency and correct errors by analyzing data distributions extracted from the intermediate layers of the DNNs. Although this approach improves the robustness by synthesizing layer-wise median features across DNN,such a fine-grained fault-tolerant module needs sophisticated data analysis techniques to extract the features from each layer, which limits its application on the large-scale DNNs. (2) Neuron-level protection refers to techniques that make DNNs recoverable from errors with dynamic constraints on the output value [24; 47]. For example, an output of activation functioning larger than a predefined threshold could dramatically affect the performance of the DNNs [24]. Therefore, this work proposes a clipped function to constrain the maximum output.