# Adaptive Hybrid Storage Format for Sparse Matrix-Vector Multiplication on Multi-Core SIMD CPUs
## 6 Evaluation  
### Experimental Setup  
**Hardware Platforms.** We use three multi-core CPUs: a 72-core Intel Knight Landing (KNL) multi-core processor, a 128-core AMD EPYC CPU, and a 64-core Phytium ARMv8 CPU. The KNL processor has a peak performance of 37 flops for double-precision operations [27]. Each KNL core has an AVX512 unit supporting four threads and running at 1.3 GHz. Each core has a private L1 data and L1 instruction cache of 32 KB. The AMD EPYC CPU features a peak performance of 3.5 Tflops double-precision operation [28,29]. The AMD CPU used has two sockets, each having 64 AVX2 cores and 256 MB L3. Each EPYC core supports two hardware threads running at 3.35 GHz, and a private 32 KB L1 data and a private 64 KB L1 instruction cache. Phytium 2000+ has 64 high-performance ARMv8 compatible xiaomi cores running at 2.2 GHz. The entire chip offers a peak performance of 563.2 Gflops for double-precision operations. The 64 hardware cores are organized into 8 panels, where each panel connects a memory control unit. Each core has a private L1 cache of 32 KB for data and instructions, and every four cores share a 2 MB L2 cache [30].  
\begin{table}
\begin{tabular}{p{85.4 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Features** & **Description** & **Importance** \\ \hline \(n\_rows\) & number of rows & 33.71\% \\ \(n\_cols\) & number of columns & \\ variation & matrix regularity & 3.96\% \\ \(nmz\_frac\) & percentage of non-zero elements & 42.72\% \\ \(nmz\_tot\) & total number of non-zero elements & \\ \(K_{histogram}\) & K is obtained by a histogram algorithm & \\ \(K_{average}\) & average number of non-zero elements per row & 2.10\% \\ nnz\_min & minimum number of non-zero elements per row & \\ \(nmz\_max\) & maximum number of non-zero elements per row & \\ nnz\_std & standard derivation of non-zero elements per row & 17.51\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Features used in our predictive model.  
**Systems Software.** Our platforms run a customized Linux operating system with a kernel v3.10.0 on KNL, v4.18.0 on AMD EPYC, and v4.19.46 on Phytium 2000+. We use Intel icc v17.0.4 on KNL, gcc v8.3.1 on AMD EPYC, gcc v9.3.0 on Phytium 2000+, with the default \"-O3\" option. We use the OpenMP threading model with 272 threads on KNL, 256 threads on EPYC and 64 threads on Phytium.  
**Datasets.** Our experiments use 956 square matrices from the SuiteSparse matrix collection [23]. The number of nonzero elements of the matrices ranges from 100 K to 200 M. The dataset includes both regular and irregular matrices, covering domains from scientific computing to social networks.  
**Native Format Settings.** In the experiments, we use double precision to store the matrices and vectors. On KNL for CSRS, \(\omega\) = 8, \(\sigma\) = 12, and for SELL-C-\(\sigma\), C = 8. On AMD EPYC for CSR5, \(\omega\) = 4, \(\sigma\) = 16, and for SELL-C-\(\sigma\), C = 4. On Phytium 2000+ for CSR5, \(\omega\) = 4, \(\sigma\) = 16, and for SELL-C-\(\sigma\), C = 4. This setting is used in accordance with 512-bit SIMD units on KNL, 256-bit SIMD units on AMD EPYC and 128-bit SIMD units on Physium. And \(\sigma\) = 512 on KNL, \(\sigma\)=256 on AMD EPYC, and \(\sigma\) = 256 on Physium 2000+ for SELL-C-\(\sigma\). Accordingly, HYB5 also uses the same configurations for its native formats.  
**Evaluation Methodology.** We use cross validation to evaluate our approach by randomly splitting the 965 matrices into two parts: 756 matrices for training and 200 matrices for testing. We learn a model with the training matrices. We then evaluate the learned model by applying it to make a prediction on the 200 testing matrices. We use a regression model, and the performance of the model is based on the accuracy of the predicted results. The performance of the parameter K predicted by the model is injected into HYB5 during runtime. We use the ratio of performance of predicted results to the Oracle performance to represent the accuracy of the model.