# CitiusSynapse: A Deep Learning Framework for Embedded Systems
## 3 Deep Learning Core and Accelerator Engine
### Accelerator Engine
#### 3.2.1 C5 blas  
BLAS is divided into vector-scalar operation routines (LEVEL 1), matrix-vector operation routines (LEVEL 2), and matrix-matrix operation routines (LEVEL 3) according to the operation data type. Each routine of BLAS is categorized as single (float), double, complex, or double complex according to the operation precision, and supports single and half (fp16) precision, which are frequently used in the latest DNNs. Table 2 presents the common routines used in the latest DNNs, among the BLAS operation routines that are provided by our framework. AXPY is often used in the scale layer and Eltwise layer, SCAL is used in the scale layer, GEMV is used in the batch normalization layer, and GEMM is used in the convolution layer and fully connected layer. The operations of GEMV and GEMM are distinguished depending on whether the matrix is transposed. Specifically, two GEMV operations may be distinguished (GEMV_T and GEMV_N) depending on whether matrix A is transposed (true or false), whereas GEMM \begin{table}
\begin{tabular}{c c} \hline \hline
**Layer Type** & **Layer Name** \\ \hline Vision layer & Convolution, Depthwise convolution, Deconvolution, Pooling (Max/Average/Global Average), Upsample, Focus 1 \\ \hline Recurrent layer & LSTM (Long Short-Term Memory) \\ \hline Normalization & Batch Normalization, Scale \\ \hline Activation & ReLU, ReLU6, Sigmoid, H-Swish, SiLU 1 \\ \hline Utility & Eltwise (Sum/Product), Concat, Reshape, Power, Permute, padding \\ \hline Common & Full-Connected (=Inner-Product) \\ \hline Output & Softmax, Yolo v2/v3/v5, EAST 2, kNN, PoseNet 3 \\ \hline Updater & SGD, Adam \\ \hline \hline \end{tabular}  
* Used only in Yolo v5. Available online: [[https://github.com/ultralytics/yolov5](https://github.com/ultralytics/yolov5)]([https://github.com/ultralytics/yolov5](https://github.com/ultralytics/yolov5)) (accessed on 23 August 2021).
* Available online: [[https://github.com/SURFZJY/EAST-caffe](https://github.com/SURFZJY/EAST-caffe)]([https://github.com/SURFZJY/EAST-caffe](https://github.com/SURFZJY/EAST-caffe)) (accessed on 23 August 2021).
* Available online: [[https://github.com/microsoft/human-pose-estimation.pytorch](https://github.com/microsoft/human-pose-estimation.pytorch)]([https://github.com/microsoft/human-pose-estimation.pytorch](https://github.com/microsoft/human-pose-estimation.pytorch)) (accessed on 23 August 2021).
* Available online: [[https://github.com/SURFZJY/EAST-caffe](https://github.com/SURFZJY/EAST-caffe)]([https://github.com/SURFZJY/EAST-caffe](https://github.com/SURFZJY/EAST-caffe)) (accessed on 23 August 2021).
* Available online: [[https://github.com/microsoft/human-pose-estimation.pytorch](https://github.com/microsoft/human-pose-estimation.pytorch)]([https://github.com/microsoft/human-pose-estimation.pytorch](https://github.com/microsoft/human-pose-estimation.pytorch)) (accessed on 23 August 2021).  
\end{table} 
Table 1: List of hidden and output layers of our framework. categorizes four operations (GEMM_NN, GEMM_NT, GEMM_TN, GEMM_TT) depending on whether matrices A and B are transposed. In general, an OpenCL device supports single instruction multiple data (SIMD) commands, and simultaneously loads/stores multiple nearby data using the data type of an OpenCL vector, or executes the same operation for multiple data. Our framework drastically reduces the overhead in operations and memory access using the characteristics of OpenCL. The maximum amount of data that can be simultaneously processed in OpenCL is determined based on the vector register size of the device. For instance, an ARM Mali-T860 has a 128-bit vector register and can simultaneously load/store up to four 32-bit data values. In contrast, an ARM Mali-G52 has a 256-bit vector register, and thus can simultaneously load/store up to eight 32-bit data values. Therefore, the vectorization size must be determined by considering the size of the vector register, in order to maximize the parallelism of OpenCL.