# Optimized Deep Neural Networks for Real-Time Object Classification on Embedded GPUs +
## 3 Heterogeneous and GPU-Only ConvMM Layers  
One approach to compute the convolution operation is Matrix multiplication-based convolution (ConvMM). Using this approach, first multi-dimensional input image and filters are transformed into two-dimensional matrices and then multiplied to achieve the results equivalent to the traditional convolution operation. The transformation step can be performed by copying the tiles of pixels from original image/filter to the matrices in a specific order. This is an operation where the sequence has to be considered and can be performed efficiently using the CPU [15]. Once transformed, these matrices can be multiplied efficiently using concurrent resources of GPU. Using the heterogeneous resources (CPU-GPU) of a system, an algorithm like this can benefit in terms of execution time [15; 21; 22]. However, extra memory transfers caused by the heterogeneous implementation can also break the overall performance.  
Figure 1 illustrates the flow of heterogeneous ConvMM layer where the transformation of data (input maps and filters) is done using the CPU and multiplication of this transformed data is performed using the GPU [15]. Before transformation, the input data is padded with zeros according to the architecture of the trained network.  
In this transformation step, multi-dimensional input maps and filters are converted to 2-dimeniosnal matrices. Data transformation is performed by copying the tiles from multi-dimensional maps and placing them in a specific order in the matrices as demonstrated by Figure 2.  
There are four data transfers between the CPU and the GPU as depicted in Figure 1. Out of four, two data transfers are made to perform the CPU-based transformation, which is a sequence-sensitive operation and is performed using the high operational frequency of CPU [15].  
Figure 1: Flow of the heterogeneous ConvMM layer.  
Figure 2: Transformation of multi-dimensional input maps to the 2-dimensional input matrix.  
If this CPU-based transformation step is parallelized and performed using the GPU, so these extra two transfers can be avoided and all tasks can be performed using the GPU as shown in Figure 3. This multi-dimensional input map can be concurrently converted to the input matrix using following formula. In this formula, _k_h_ and _k_w_ are representing the height and width of the filter. Number of rows, columns and dimensions of the input image are represented by _i_Row_, _i_Col_ and _i_Dim_ respectively. _o_Row_ and _o_Cols_ are representing the number of rows and columns of final output after convolution.  
\[Input\_matrix_{(row,col)}=\sum_{i=1}^{o\_Row}\sum_{j=1}^{col}(\sum_{k=1}^{i \_Dim}\sum_{h=1}^{k}\sum_{w=1}^{k\_w}Input\_map_{(i+h,j+w,k)}) \tag{1}\]  
where _o_Row_ = _i_Row_ - _k_h_ + 1; _o_Col_ = _i_Col_ - _k_w_ + 1; _row_ = \(i\) \(\times\) (_o_Col_) + _j; _col_ = (_k_ \(\times\) _k_w_ \(\times\) _k_h_ + (_h_ \(\times\) _k_w_) + _w_).  
If input map is of size 3 \(\times\) 3 (_i_Row_ \(\times\) _i_Col_) and size of the filter is 2 \(\times\) 2 (_k_h_ \(\times\) _k_w_), so the output map after convolution would be of size 2 \(\times\) 2 (_o_Row_ \(\times\) _o_Col_) with padding and stride of 0 and 1 respectively.  
By parallelizing the transformation step, the number of data transfers between the host and the device are minimized that can provide the substantial improvement in performance.