# Runtime Construction of Large-Scale Spiking Neuronal Network Models on GPU Devices
## 2 Materials and Methods
### Creation of Connections Directly in GPU Memory
A loop is performed on the connection blocks, starting from the first block in which there are available slots up to the last allocated block, and the connections are created in each block by launching appropriate CUDA kernels to set the connection parameters described above. In each block \(b\), the index of each of the new connections is calculated from the CUDA thread index \(k\) according to the formula:  
\[i_{\text{newconns,b}}=N_{\text{prevconns,b}}+k\quad\text{with}\quad k=0,...,N _{\text{thr}}-1 \tag{3}\]  
where \(i_{\text{newconns,b}}\) refers to the subset of \(i_{\text{newconns}}\) on the current block, and \(N_{\text{prevconns,b}}\) is the number of new connections created in the previous blocks. The number of connections to be created in the current block, which corresponds to the number of required threads \(N_{\text{thr}}\), is computed before launching the kernel; if the block will be completely filled, the number of threads equals the block size, \(N_{\text{thr}}=B\). See Figure 1 for an example of how the connections are numbered and assigned to the blocks. The indexes of a source node \(s\) and a target node \(t\) are calculated from \(i_{\text{newconns}}\) using expressions that depend on the connection rule. Here, we provide both the name of the rules as defined in [21] and their corresponding parameter of the NEST interface. In case both the source-node group and the target-node group contain nodes with consecutive indexes, starting from \(s_{0}\) and from \(t_{0}\), respectively, the node indexes are as follows:  
* One-to-one (one_to_one): \[s =s_{0}+i_{\text{newconns}}\] (4) \[t =t_{0}+i_{\text{newconns}}\] (5) with \(N_{\text{newconns}}=N_{\text{sources}}=N_{\text{targets}}\). * All-to-all (all_to_all): \[s =s_{0}+\lfloor\frac{i_{\text{newconns}}}{N_{\text{targets}}}\rfloor\] (6) \[t =t_{0}+\text{mod}(i_{\text{newconns}},N_{\text{targets}})\] (7) with \(N_{\text{newconns}}=N_{\text{sources}}\times N_{\text{targets}}\). Random, fixed out-degree with multiapses (fixed_outdegree): \[s =s_{0}+\lfloor\frac{i_{\text{newcoms}}}{K}\rfloor\] (8) \[t =t_{0}+\text{rand}(N_{\text{targets}})\] (9) where \(K\) is the out-degree, i.e., the number of output connections per source node, \(\text{rand}(N_{\text{targets}})\) is a random integer between 0 and \(N_{\text{targets}}-1\) sampled from a uniform distribution, and \(N_{\text{newcoms}}=N_{\text{sources}}\times K\). * Random, fixed in-degree with multiapses (fixed_indegree): \[s =s_{0}+\text{rand}(N_{\text{sources}})\] (10) \[i_{\text{newconns}}=0,...,N_{\text{newconns}}-1. \tag{2}\]0 (11) where \(K\) is the in-degree, i.e., the number of input connections per target node, and \(N_{\text{newcoms}}=n_{\text{targets}}\times K\). * Random, fixed total number with multiapses (fixed_total_number): \[i_{\text{newconns}}=0,...,N_{\text{newconns}}-1. \tag{2}\]1 (12) \[i_{\text{newconns}}=0,...,N_{\text{newconns}}-1. \tag{2}\]2 (13)  
\[i_{\text{newconns}}=0,...,N_{\text{newconns}}-1. \tag{2}\]3 Example of connection creation using the all-to-all connection rule. (**A**) Each one of the four source nodes (green) is connected to all three target nodes (orange). The connections generated via this rule are identified with an index, \(i_{\text{newcoms}}\), ranging from 0 to 11 (blue disks). (**B**) The connections are stored in blocks that are allocated dynamically, where, for demonstration purposes, a block size of ten connections is used. The black squares represent previous connections (established using an earlier connect call), while the twelve connections generated via the considered instance of the all-to-all rule are represented by the same blue disks labeled with \(i_{\text{newcoms}}\) as in panel A. The new connections in different blocks are generated via separate CUDA kernels.