# Parallel Simulations of the Sharp Wave-Ripples of the Hippocamp on Multicore CPUs and GPUs
## 2 Materials and Methods
### Parallel Simulations of the CA1 Network
#### 2.2.3 The GPU Algorithm  
The GPU version of the hippocampus network simulation relies on CUDA (Compute Unified Device Architecture). It is a parallel computing platform and programming model developed by NVIDIA for general-purpose computing on NVIDIA GPUs. It permits developers to utilise the substantial parallel processing capabilities of GPUs to accelerate compute-intensive applications. CUDA provides extensions to standard programming languages, including C, C++, and Fortran, enabling developers to write code that executeson the GPU. This is particularly well-suited for tasks that can be parallelised, such as matrix operations, image processing, and scientific simulations. CUDA capitalises on the underlying architecture of the GPU, including thousands of highly efficient cores that are optimised for parallel execution. In contrast to traditional CPUs, which are optimized for sequential processing, GPUs are capable of handling thousands of threads simultaneously, making CUDA an optimal choice for problems that can be decomposed into numerous smaller tasks that can be executed in parallel. The typical steps of a CUDA program involve:  
1. GPU memory allocation: the first step is to allocate memory on the GPU through the _cudaMalloc_ routine. 2. Data transfer: this step is responsible for transferring the required data from the host (CPU) memory to the device (GPU) memory. This step is of paramount importance, as GPUs possess their own dedicated memory space, and operations are conducted on data stored within this memory. The _cudaMcmcpy_ routine performs these memory transfers. 3. Kernel launch: the fundamental component of a CUDA program is the _kernel_, which is a function that is executed on the GPU. The kernel is initiated from the host code, with the number of threads and blocks previously specified. Each thread within a block executes operations on a portion of the data in parallel. 4. Results transfer: once the execution of the kernel is complete, the results are transferred from the device memory to the host memory. This enables the CPU to access and process the data as required. These transfers are again managed by the _cudaMcmcpy_ routine. 5. GPU memory de-allocation: subsequently, the allocated GPU memory is released by the _cudaTree_ routine and the program terminates. CUDA's ability to utilize the parallelism inherent in GPUs can lead to significant performance gains over traditional CPU-only implementations, particularly in applications that involve heavy numerical computations and data processing. By allowing developers to directly program the GPU, CUDA provides fine-grained control over performance, enabling optimizations that are critical in high-performance computing domains. The efficient transfer of data between the CPU and GPU is a crucial aspect of optimising GPU performance, given that data must be moved between these two components both before and after the execution of a computation. The PCIe bus, which connects the two, has a limited bandwidth in comparison to the GPU's internal memory, which creates the potential for memory transfer bottlenecks. The implementation of efficient transfer strategies allows us to reduce latency and increase throughput, thereby ensuring that the GPU is utilised to its fullest capacity without unnecessary idle time. Therefore, special effort has been put in minimizing memory transfers during the hippocampus CA1 network simulation. The adopted strategy consists of transfers from host to device of all the non-time dependent data after they are computed at the beginning of the simulation. Then, the main for loop iterating over the time steps start. The computation of the synaptic contribution, performed by matrix-vector multiplications, is implemented by exploiting the cuBLAS library, which contains highly optimized CUDA kernels for linear algebra operations. In particular, the _cublasDgemv_ routine performs the matrix-vector product. At the beginning of the for loop, this routine is activated four times to compute the EE, EI, II, and IE contributions.