## 8 Implementation and Experiments
### Efficiency of Forward Computations in Convolutional Layers for Different Implementation Variants  
As mentioned earlier, there are seven implementation variants provided for forward computations in convolutional layers. Below, we list their code names with brief descriptions. The code names constitute suffixes of the actual full function names, e.g., do_forward_numba_cuda_tiles(.). Additionally, CUDA-related functions execute from within the so-called GPU kernels with computations designed for each single GPU thread (affix:_job). For more details, we direct the reader to the code and bibliographic references. 1. numpy_gemm--GEMM-based convolution; see (9), involving image-to-matrix and weights-to-matrix transformations. and a numpy.dot multiplication of matrices, executed from regular Python code. 2. numba_jit_direct--definition-based convolution; see (8), implemented directly by multiple loops and compiled and executed by means of numba.jit. Figure 22: Backward computations of error propagation for class Conv2D: the simplest numpy-based variant (GEMM). 3. numba_jit_gemm--GEMM-based convolution (as in variant 1); compiled and executed by means of numba_jit. 4. cv2--convolution based on an OpenCV function cv2.filter2D (faster than scipy.signal.convolve2d); involves multiple invocations of the function to suitably handle input-output channels. 5. numba_cuda_direct--definition-based convolution (as in 2); compiled and executed on GPU via numba.cuda. 6. numba_cuda_tiles--definition-based convolution that partitions inputs into tiles associated with CUDA thread blocks and involves collaboration of threads in order to copy inputs and weights into efficient _shared_ memory; compiled and executed on the GPU by means of numba.cuda. 7. numba_cuda_viafft--takes advantage of the fact that convolution (for \(q\)-th output channel) can be computed as \(\text{IFFT2}\left(\sum_{p}\text{FFT2}(W_{\cdot,\cdot,p,q})\cdot\text{FFT2}(X_{ \cdot,\cdot,p})\right)+w_{0q}\), where \(p\) iterates over input channels; involves invocations of multiple GPU kernels and is compiled and executed on the GPU by means of numba.cuda. Winograd-based implementations are currently not provided in the hmdl project. Figure 23 compares execution times (averaged over 10 repetitions) of convolutional layers with 64 input and 64 output channels and a batch of size 32 for different implementations, as well as filter and image sizes. Note the logarithmic scale on the vertical axes. In the first two graphs, we additionally plotted the times achieved using Keras (backed with TensorFlow) for reference. Details of the hardware and software environment on which the executions were performed are provided below. _\"NVIDIA Quadro M4000M\" environment--hardware: Lenovo ThinkPad P70, Intel Xeon CPU E3-1505M v5 2.8 GHz (3.7 GHz boost), 64 GB RAM, and NVIDIA Quadro M4000M GPU (4 GB/32 GB of dedicated/shared memory), manufactured by: Lenovo Group Limited, Beijing, China; software: nvcc 11.6 (V11.6.112), Python 3.9.7, numpy 1.20.0, numba 0.54.1, cv2 4.5.5-dev keras 2.8.0, and tensorflow 2.8.0; OS: Windows 10 Pro (22H2)._  
The following observations can be made based on the time measurements obtained and Figure 23. The numba_jit_direct and cv2 implementations are clearly the worst, with the latter being more efficient for larger filter and image sizes. The variants numpy_gemm and numba_jit_gemm perform similarly well with a slight advantage of the latter. Interestingly, for a small \(3\times 3\) filter, their results are comparable with numba_cuda_tiles, or actually better for images not bigger than \(8\times 8\). For larger images and filters, the advantage of GPU parallelism manifests, and numba_cuda_direct/tiles/viafft variants become clearly better than _gemm CPU variants. For a \(7\times 7\) filter, numba_cuda_tiles starts to be competitive with numba_cuda_direct, and it beats the latter regularly for even larger filters and images not smaller than \(16\times 16\).