# Runtime Construction of Large-Scale Spiking Neuronal Network Models on GPU Devices
## 1 Introduction
Further, Brian2 GeNN [12] provides a code generation pipeline for defining models via the Python interface of Brian [3] and using GeNN as a simulator backend. Alternatively, Brian2 CUDA [14] directly extends Brian with a GPU backend. The hybrid approach is exemplified in CARLsim [8], which has also developed its own Python interface to communicate with its C/C++ library named PyCARL [18]. Much like PyNEST [19], the Python interface of the NEST simulator, CARLsim exposes its C/C++ kernel with a dynamic library, which can then interact with Python. However, like GeNN, they make use of SWIG to automatically generate the binding between their library and their Python interface. PyCARL directly serves as a PyNN [20] interface. NEST GPU [13] is a software library for the simulation of spiking neural networks on GPUs. It originates from the prototype NeuronGPU library [9] and is now overseen by the NEST Initiative and integrated with the NEST development process. NEST GPU uses a hybrid approach and offers the possibility to implement models using either Python scripts or C++ code. The main commands of the Python interface, the use of dictionaries, and the names and parameters of the neuron and spike generator models are already aligned to those of the CPU-based NEST code. In a previous version of NEST GPU, connections were first created on the CPU side and then copied from the RAM to GPU memory. This approach benefited from the standard C++ libraries and, particularly, the dynamic allocation of container classes of the C++ Standard Template Library. However, it had the drawback of relatively long network construction times, not only due to the costly copying of connections and other CPU-side initializations, but also because the connection creation process was performed serially. This work proposes a network construction method in which the connections are created directly in the GPU memory with a dynamic approach and then suitably organized in the same memory using algorithms that exploit GPU parallelism. This approach, so far applied to single-GPU simulations, enables much faster connection creation, initialization, and organization while preserving the advantages of dynamic connection building, particularly the ability to create and initialize the model at run-time without the need for compilation. Although this method was developed specifically in the framework of NEST GPU, the concepts are sufficiently general that they should be applicable with minimal adaptation to other GPU-based simulators as long as they are designed with a modular structure. Section 2 of this manuscript first introduces the dynamic creation of connections and provides details on the used data structures and the spike buffer employed in the simulation algorithm (Sections 2.1-2.3); details on the employed block sorting algorithm are in Appendix A. The proposed dynamic approach for network construction is tested on the simulation of two complementary weighted network models across different hardware configurations; we then compare the performance to other simulation approaches. Details on the network models, the hardware and software, and time measurements for performance evaluations are given in Sections 2.4-2.6. The spiking activity of a network constructed with the dynamic approach is validated statistically in Section 2.7 and Appendix B. The performance results are shown in Section 3 (with additional data in Appendices C and D) and discussed in Section 4.