# Applying Reinforcement Learning to Protect Deep Neural Networks from Soft Errors
## 2 Related Work
### Injecting Faults Within DNNs  
Fault injection is a common approach to revealing the system-wide impacts of specific faults for requirements engineering, system and component design, and verification and validation [21]. Based on the characteristics of faults of concern, fault injection techniques for DNNs can be categorized into (1) functional fault injection, (2) technical fault injection, and (3) physical fault injection.  
**Functional fault injection** involves the use of Platform-independent Models (PiMs) to simulate and analyze the functional impacts of faults [19; 22]. This technique is typically applied during early design phases for revealing the sensitivity of DNN [17; 23] applications regarding faults commonly represented as bit-flips, stuck-at-0/1, and Gaussian noise with certain prior distributions. The injection affects either the structure or the parameters of specific DNN models [2; 5; 15; 23; 24; 25], with the scope of injection ranging from the network layers to specific neurons and parameters. By enabling an analysis of fault behaviors early in system development, functional fault injection constitutes a basis for identifying additional requirements on DNN robustness, resilience, and protection. While injecting functional faults with high fidelity into applications can often become very challenging, current studies have shown that a systematic approach to fault modeling and fault injections can still yield effectiveness similar to more detailed hardware simulation [26]. Many current approaches [2; 5; 15; 24; 25] treat the injections of faults as individual and uncorrected tasks, highlighting a significant area for improvement.  
**Technical fault injection** is a technique used to assess the robustness and reliability of DNN using Platform-specific Models (PsMs) or detailed hardware-dependent solutions. It is focused on the fault behaviors of DNNs when implemented with hardware resources like CPU and AI accelerators [19; 27; 28; 29]. The injections can be applied to the full DNN systems by targeting the microarchitectures, including Instruction Set Architectures (ISAs) and Register Transfer Level (RTL) properties, commonly through advanced virtual simulations (e.g., gem5 [30]). The aim is to effectively capture the actual system's responses to hardware faults [26; 27]. This type of injection is important for the detailed technical design of DNN systems, where the specification of hardware performance, robustness, and resilience are of particular importance.  
**Physical fault injection** relies on gaining physical access or close proximity to the target DNN systems in their actual operational environments to intentionally introduce physical faults [22]. As an example of physical fault injection, prior studies [6; 27] arrange the hardware (e.g., CPU and General Purpose--GPU) to be exposed to neutron beams. The results conclude that while the data corruption rate remains similar, physical fault injection exhibits a significantly higher system failure rate compared to one of the other fault injection methods. Beyond the synthetic simulations, this type of fault injection (e.g., the beam test) often requires high-cost equipment, such as radiation sources, to induce data corruption in memory. Moreover, beam tests typically offer a statistical perspective on the impact of fault models across the devices, such as Failure In Time (FIT), which does not allow faults to be measured at the level of individual memory cells or mapped to specific parameters of DNNs [31]. Therefore, this method is tailored for reliability analysis during the final technical development phase, such as with physical microchips or products.