# Fine-Grained Fault Sensitivity Analysis of Vision Transformers Under Soft Errors  
[PERSON]  
1 Laboratory of Digital IC and Space Application, School of Microelectronics, Xidian University, Xi'an 710071, China; [EMAIL_ADDRESS] (J.H.); [EMAIL_ADDRESS] (Y.X) 1  
[PERSON]  
2 Shenzhen Institute of Technology, Xidian University, Shenzhen 518000, China 2  
[PERSON]  
3 Guangzhou Institute of Technology, Xidian University, Guangzhou 510555, China; [EMAIL_ADDRESS] (Y.L.); [EMAIL_ADDRESS] (C.X.) 3  
[PERSON]  
3 Guangzhou Institute of Technology, Xidian University, Guangzhou 510555, China; [EMAIL_ADDRESS] (C.X.) 3  
[PERSON]  
1 Laboratory of Digital IC and Space Application, School of Microelectronics, Xidian University, Xi'an 710071, China; [EMAIL_ADDRESS] (J.H.); [EMAIL_ADDRESS] (Y.X) 1  
Footnote 1: [[https://doi.org/10.3390/electronics14122418](https://doi.org/10.3390/electronics14122418)]([https://doi.org/10.3390/electronics14122418](https://doi.org/10.3390/electronics14122418))  
Footnote 2: [[https://www.xidian.edu.cn/](https://www.xidian.edu.cn/)]([https://www.xidian.edu.cn/](https://www.xidian.edu.cn/))  
high computational complexity [18] and an increased susceptibility to hardware-induced errors [19]. (2) With continuous progress in integrated circuit technologies, the probability of soft error rises steadily in Very Large-Scale Integration (VLSI) [20; 21].  
The reliability of convolutional neural networks (CNNs) has been extensively researched, ranging from architecture-level sensitivity [22], data-type implementation [23], and model compression strategies [24], to hardware-level resilience designs [25; 26]. Recent works on ViT robustness have primarily focused on adversarial defense [27], privacy preservation [28; 29], and quantization/pruning techniques [30; 31]. However, studies specifically targeting the fault resilience of ViTs under hardware errors are still in their infancy, with existing works limited to basic simulation and lacking fine-grained analysis [32; 33]. To bridge this gap, we performed a systematic and fine-grained fault sensitivity analysis on quantized ViT models under realistic deployment conditions.  
The following summarizes our main contributions:  
* We proposed a fault resilience evaluation framework tailored to ViTs, incorporating realistic deployment settings via Int8 quantization.
* We performed a fine-grained fault sensitivity analysis across four key dimensions--model-wise, layer-wise, type-wise, and head-wise--under bit-flip fault injection scenarios.
* From a model-wise perspective, we observed that architectural parameters such as model size and patch size substantially influence error resilience, with BER thresholds varying by several orders of magnitude across models.
* At the layer-wise level, we identified the first transformer encoder layer as particularly susceptible to soft errors, owing to its foundational role in hierarchical feature extraction. Middle and later layers' MLP sub-blocks also emerge as critical vulnerability points due to their dominant computational load and contribution to representation learning.
* In the type-wise and head-wise analyses, we revealed heterogeneous vulnerability patterns across Q, K, and V projection matrices and attention heads. Notably, we integrated the mean attention distance (MAD) metric to interpret why certain heads are more fault-prone--an interpretability perspective not addressed in previous works.