# Runtime Construction of Large-Scale Spiking Neuronal Network Models on GPU Devices
## 1 Introduction  
Spiking neuronal network models are widely used in the context of computational neuroscience to study the activity of the populations of neurons in the biological brain. Numerous software packages have been developed to simulate these models effectively. Some of these simulation engines offer the ability to accurately simulate a wide range of neuron models and their synaptic connections. Among the most popular codes are NEST [1], NEURON [2], and Brian 2 [3]. NENGO [4] and ANNarchy [5] should also be mentioned. In recent years, there has been a growing interest in GPU-based approaches, which can be particularly useful for simulating large-scale networks thanks to their high degree of parallelism. This interest is also fueled by the rapid technological development of this type of device and the availability of increasingly performant GPU cards both for the consumer and for high-performance computing (HPC) infrastructure. A main driving force behind this development is the demand from current artificial intelligence algorithms andsimilar applications for massively parallel processing of simple floating point operations, and a corresponding industry with huge financial resources. Present-day supercomputers are reaching for exascale by drawing their compute power from GPUs. For neuroscience to benefit from these systems, efficient algorithms for the simulation of spiking neuronal networks need to be developed. Simulation codes such as GeNN [6], CARLsim [7; 8], and NEST GPU [9] have been primarily designed for GPUs, while, in recent times, popular CPU-based simulators have shown interest in integrating the more traditional CPU-based approach with libraries for GPU simulation [10; 11; 12; 13; 14; 15]. Furthermore, the novel simulation library Arbor [16], which focuses on morphologically-detailed neural networks, takes GPUs into account. In general, GPU-based simulators fall into one of three categories: those that allow the construction of network models at run time using scripting languages, those that require the network models to be fully specified in a compiled language, and hybrid ones that provide both options. The most extensively used compiled languages are C and C++ for host code and CUDA for device code (using NVIDIA GPUs), while the most widely used scripting language is Python. With scripting languages, simulations can be performed without the need to compile the code used to describe the model. Consequently, the time required for compilation is eliminated. Furthermore, in many cases, the use of a scripting language simplifies the implementation of the model, especially for users who do not have extensive programming language expertise. Approaches using compiled languages typically have much faster network construction times. To reconcile this benefit with the greater ease of model implementation using a scripting language, some simulators have shifted toward a code-generation approach [5; 6]. In this approach, the model is implemented by the user via a brief high-level description, which the code generator then converts into the language or languages that must be compiled before being executed in the CPU and GPU. The main disadvantage of code-generation based simulators is the need for new code generation and compilation every time model modifications such as changes in network architecture are necessary. The times associated with code generation and compilation are typically much longer than network construction times [11]. Examples of the code-generation based approaches include GeNN [6] and ANNarchy [5]. In GeNN, neuron and synapse models are defined in C++ classes, and snippets of C-like code can be used to offload costly operations onto the GPU. The Python package PyGeNN [17] is built on top of an automatically generated wrapper for the C++ interface (using SWIG ([[https://www.swig.org](https://www.swig.org)]([https://www.swig.org](https://www.swig.org)), accessed on 14 August 2023)) and allows for the same low-level control.