# On the Safe Deployment of Matrix Multiplication in Massively Parallel Safety-Related Systems
###### Abstract  
Deep learning technology has enabled the development of increasingly complex safety-related autonomous systems using high-performance computers, such as graphics processing units (GPUs), which provide the required high computing performance for the execution of parallel computing algorithms, such as matrix-matrix multiplications (a central computing element of deep learning software libraries). However, the safety certification of parallel computing software algorithms and GPU-based safety-related systems is a challenge to be addressed. For example, achieving the required fault-tolerance and diagnostic coverage for random hardware errors. This paper contributes with a safe matrix-matrix multiplication software implementation for GPUs with random hardware error-detection capabilities (permanent, transient) that can be used with different architectural patterns for fault-tolerance, and which serves as a foundation for the implementation of safe deep learning libraries for GPUs. The proposed contribution is complementary and can be combined with other techniques, such as algorithm-based fault tolerance. In particular, (i) we provide the high-performance matrix multiplication CUTLASS library with a catalog of diagnostic mechanisms to detect random hardware errors down to the arithmetic operation level; and (ii) we measure the performance impact incurred by the adoption of these mechanisms and their achievable diagnostic coverage with a set of representative matrix dimensions. To that end, we implement these algebraic operations, targeting CUDA cores with single instructions and multiple-thread math instructions in an NVIDIA Xavier NX GPU. safety; reliability; CNN; matrix multiplication; GPU; fault detection Article  
On the Safe Deployment of Matrix Multiplication in Massively Parallel Safety-Related Systems. Parallel Safety-Related Systems  
[PERSON], [PERSON], [PERSON], [PERSON], [PERSON], [PERSON], F.  
Besides, Figure 1 illustrates the possible consequences of a single error on a CNN-based object detector. We classify an image (\"9e75b2a9-98437b5b.jpg\" from \"10K Image\" package) from Berkeley DeepDrive dataset [4] with YOLOv3 [5] based on a pre-trained model with the COCO dataset (\"yolov3.cfg\" and \"coco.data\" extracted from the configuration folder ([[https://github.com/[PERSON]/darknet/tree/darknet_yolo_v3](https://github.com/[PERSON]/darknet/tree/darknet_yolo_v3)]([https://github.com/AlexeyAB/darknet/tree/darknet_yolo_v3](https://github.com/AlexeyAB/darknet/tree/darknet_yolo_v3)), accessed on 12 March 2022). Activation weights from [[https://pjreddie.com/media/files/yolov3.weights](https://pjreddie.com/media/files/yolov3.weights)]([https://pjreddie.com/media/files/yolov3.weights](https://pjreddie.com/media/files/yolov3.weights)), accessed on 12 March 2022) [6]. In the absence of errors, the CNN correctly detects 35 classes, such as people or traffic lights, among others (Figure 1a). However, a single-bit error injection in a weight of the first layer of the CNN leads to detecting 11 classes not present in the image, such as three wine glasses or four dining tables (Figure 1b), but no people or traffic lights. In contrast, if the error is injected at bit 705, it does not impact the CNN inference, becoming a latent error if further diagnostics are not in place. These errors are of particular concern in safety-related systems, where misclassifications can have catastrophic consequences. Current solutions focus on selectively hardening the most vulnerable parts [7] or guaranteeing consistency among inputs and outputs matrices [8; 9], but not on arithmetic operations or all combinatorial (e.g., muxes, bypasses) and sequential elements (e.g., latches) between memories and arithmetic units. In the context of commercial off-the-shelf GPUs, implementation-specific tests [10] or software-based self tests (SBSTs) [11] of individual components are not viable since they require fine-grain knowledge of the GPU component design and implementation.