# Fine-Grained Fault Sensitivity Analysis of Vision Transformers Under Soft Errors
## 2 ViT Model and Fault Resilience Evaluation Platform
### Fault Resilience Evaluation Platform  
#### 2.2.1 Soft Errors  
Inspired by ViT's remarkable accuracy, many studies have attempted to employ ViT in safety-critical missions like automotive, space, and industry to extract useful information from complex raw data [34]. Hence, the dependability of deployed accelerator-based  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model Variant** & **Component Name** & **Weight Category** & **Weight Dimension** \\ \hline \multirow{8}{*}{ViT-B/16} & \multirow{4}{*}{patch embedding} & cls\_token & \(1\times 1\times 768\) \\  & & pos\_embed & \(1\times 577\times 768\) \\  & & patch\_embed & \(768\times 3\times 16\times 16\) \\  & & attn.qkv & \(2304\times 768\) \\  & \multirow{2}{*}{encoder layer1\(\sim\)12} & attn.proj & \(768\times 768\) \\  & & mlp.fc1 & \(3072\times 768\) \\  & & mlp.fc2 & \(768\times 3072\) \\  & \multirow{2}{*}{classification head} & head & \(768\times 1000\) \\ \hline \multirow{8}{*}{ViT-B/32} & \multirow{4}{*}{patch embedding} & cls\_token & \(1\times 1\times 768\) \\  & & pos\_embed & \(1\times 145\times 768\) \\  & & patch\_embed & \(768\times 3\times 32\times 32\) \\  & & attn.qkv & \(2304\times 768\) \\  & \multirow{2}{*}{encoder layer1\(\sim\)12} & attn.proj & \(768\times 768\) \\  & & mlp.fc1 & \(3072\times 768\) \\  & & mlp.fc2 & \(768\times 3072\) \\  & \multirow{2}{*}{classification head} & head & \(768\times 1000\) \\ \hline \multirow{8}{*}{ViT-L/16} & \multirow{4}{*}{encoder layer1\(\sim\)24} & cls\_token & \(1\times 1\times 1024\) \\  & & pos\_embed & \(1\times 577\times 1024\) \\  & & patch\_embed & \(1024\times 3\times 16\times 16\) \\  & & attn.qkv & \(3072\times 1024\) \\  & & attn.proj & \(1024\times 1024\) \\  & \multirow{2}{*}{encoder layer1\(\sim\)24} & mlp.fc1 & \(4096\times 1024\) \\  & & mlp.fc2 & \(1024\times 4096\) \\  & & head & \(1024\times 1000\) \\ \hline \multirow{8}{*}{ViT-L/32} & \multirow{4}{*}{patch embedding} & cls\_token & \(1\times 1\times 1024\) \\  & & pos\_embed & \(1\times 145\times 1024\) \\ \cline{1-1}  & & patch\_embed & \(1024\times 3\times 32\times 32\) \\ \cline{1-1}  & & attn.qkv & \(3072\times 1024\) \\ \cline{1-1}  & & attn.proj & \(1024\times 1024\) \\ \cline{1-1}  & & mlp.fc1 & \(4096\times 1024\) \\ \cline{1-1}  & & mlp.fc2 & \(1024\times 4096\) \\ \cline{1-1}  & & head & \(1024\times 1000\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Dimensions of different weight matrices in different ViT models.  
systems (e.g., GPU or DNN accelerator) is of paramount importance. Their failure can result in catastrophic consequences [35]. Moreover, with the continuous scaling of CMOS technology feature size and the lowering voltage of large-scale VLSI designs, soft errors have become inevitable. Soft error is a type of transient error that temporarily corrupts data without causing permanent damage to the hardware. It is typically caused by external interface, such as high-energy neutron or alpha particle strikes. Soft errors usually affect only one bit of data, flipping it from 1 to 0 or from 0 to 1 in SRAM, registers, or flip-flops, so the bit-flip error was utilized to model soft errors in fault simulation [20; 21]. We will utilize soft errors or bit-flip errors interchangeably throughout this article. The soft errors can corrupt the data stored in memory and propagate along with the hardware data flow, spread to more operations, and generate incorrect computing results, which may induce considerable prediction accuracy loss of deep learning models.