# On the Safe Deployment of Matrix Multiplication in Massively Parallel Safety-Related Systems
## 4 Evaluation
### Performance Impact  
In order to derive the relative performance impact, we divide the execution time of the matrix-matrix multiplication, including the diagnostic techniques, by the execution time of the original matrix-matrix multiplication, with identical matrix dimensions and compiler optimization. This performance impact is relative to single-matrices execution. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Square**} & \multicolumn{4}{c}{**Unbalanced**} \\ \hline
**Experiments** & **N** & **N** & **M** & **N** & **K** & **Name** \\ \hline Performance & 80 & 80 & 18 & 230,400 & 64 & L91 \\ Impact & 160 & 160 & & & & \\  & 320 & 320 & & & & \\ \hline Diagnostic & 20 & 20 & 32 & 29 & 144 & L1 \\ Coverage & 40 & 40 & 8 & 900 & 8 & L2 \\  & 80 & 80 & 15 & 225 & 48 & L3 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Matrix dimensions employed in the experiments. However, performance impact varies at system level depending on the safety architectural pattern in which the safety designer includes the diagnostic mechanism(s). Thus, the impact on the periodic diagnostic pattern described in Section 3.4 is calculated by dividing the single'safe matrix-matrix multiplication' execution by the DTI, not depending only on the performance impact of a single'safe matrix-matrix multiplication' execution but also on the number of iterations of the matrix-matrix multiplication in the DTI. According to the implementation of redundant patterns, the performance impact can either: (i) be multiplied by a factor of two (in cases where the matrix-matrix multiplication is re-executed on the same hardware) or (ii) maintain the same value as that observed for single MMMs (double hardware cost). This subsection provides the performance impact relative to single-matrices execution. From these results and according to the implemented safety architectural pattern, safety designers can compute the final performance impact in their systems. Initially, we perform timing experiments by disabling compiler optimizations to avoid additional safety challenges brought by optimizations (compiler option -O0). Throughout this paper, we use compiler optimizations to refer to both host and device compilers. In this way, we perform the performance impact experiments with the same optimizations for both. Figure 5 shows how the performance impact caused by the inclusion of both individual or a combination of checksums is rather small (between 1\(\times\) and 1.25\(\times\) for all matrix dimensions). In reference [14], the authors show that increasing matrix dimension sizes lead to decreasing performance impacts for sequential and AVX-based implementations. Furthermore, we also show significant differences across different checksums. Moreover, in Figure 5 we observe a very similar impact for different matrix dimensions in our GPU-based implementation, as well as across different checksums. This behavior relates to the fact that execution time is dominated by memory accesses. Hence, arithmetic operations due to the different checksums have a very limited impact that barely changes when varying the matrix dimensions or the specific checksum used. Since real-time safety applications may need optimizations to achieve the required performance, we evaluate the performance impact with a higher compiler optimization too (compiler optimization option -O3). In Figure 6, we can observe that the performance impact increases across all checksums, but particularly for the Fletcher and CRC if imple  
Figure 5: Performance impact with -O0 optimization. mented in the internal loop. Such impact further exacerbates when increasing the matrix dimensions, as opposed to the case of sequential and AVX-based implementations. CRC implementation involves four accesses to the shared memory to protect each word in the loop. In GPUs, memory latency is crucial from a performance point of view. That explains the performance impact produced by the CRC implementation, reaching an impact of up to \(\approx\)100\(\times\) the original implementation.