dataesr/scientific-paragraphs-categorization huggingface dataset:
While existing solutions, such as the XML-based data formats NeuroML  [27, 28]  and NineML (http://incf. github. io/nineml-spec/), the PyNN language  [23] , and the NSDF standard for simulator output  [29] , have proven useful, major challenges remain and are felt acutely in the case of large data-driven network models. One problem is a performance bottleneck: storing data about thousands of neurons or millions of synapses in verbose text-based files produces a large disk space footprint and may be challenging for reading/writing in parallel compute environments. Another is that existing formats describe either model structure or simulation output, but not both. Finally, for broad adoption of a modeling data format, it needs to be flexible enough to represent a variety of model types (point neuron, biophysically detailed, etc.
) and compatible with more specialized formats (e. g. , SWC for neuronal morphologies  [30] ), without compromising computational performance. Notably, similar challenges exist in experimental neuroscience (see, e. g. ,  [31] ). The situation is improving due to initiatives for experimental data formats, such as NWB:N  [32] , Neo  [33] , BIDS  [34] , Loom (https://linnarssonlab. org/loompy), NIX (http://g-node. github. io/nix/), or spacetx-starfish (https://github. com/spacetx/starfish), but for many types of experimental data the community is still far from a widespread adoption of universally agreed-upon formats (see also  [35, 36] ). These challenges contribute to difficulties in closing the virtuous experiment/ modeling loop and to the overall reproducibility crisis  [37, 38, 12]. Here we present the SONATA (Scalable Open Network Architecture TemplAte) data format, which provides an open-source framework for representing neuronal circuits, simulation configurations, and simulation outputs.
The format has been jointly developed by the Allen Institute and the Blue Brain Project to facilitate exchange of their large scale cortical models (e. g. ,  [39] [40] [41] ) and is supported by these organizations' software tools, such as BMTK (https:// alleninstitute. github. io/bmtk/; [26] ). Support for the format has also been added by other simulation tools-pyNeuroML  [27, 28] , PyNN  [23] , and NetPyNE  [24] -and an interface between SONATA and the NWB:N format  [32]  for neurophysiological data has been developed. As described below, SONATA utilizes computationally efficient binary formats for storing large datasets while also offering text-based formats for easy editing of less data-rich model components. SONATA represents all aspects of models and simulations, from network structure, to simulation parameters, to input and output activity.
It provides much flexibility for describing models at different levels of resolution, including hybrid models. Importantly, because SONATA is already supported by a number of widely used tools and applications, users can get all of the benefits of the format with no extra work on their part. Full specification of the format can be found at the SONATA GitHub page (https://github. com/AllenInstitute/ sonata), along with the open-source reference application programming interfaces (APIs). To enable broad applications in the field, SONATA is freely available and open to contributions from the community.
The SONATA specification which is built on top of these formats accommodates multiple cell and synapse model types and is designed to optimally handle a heterogeneous network. To achieve flexibility in defining models, SONATA provides recipes for storing arbitrary attributes, with some attribute names being reserved for basic standardization. Below, we describe the details of these elements of the SONATA format. A more complete description is given in the Online Documentation (https://github. com/AllenInstitute/sonata/ blob/master/docs/SONATA_DEVELOPER_GUIDE. md).
The "single compartment" type was created separately from "biophysical" simply for convenience, due to user requests. The "point_neuron" type is geared towards more abstract integrate-and-fire mechanisms, and "virtual" type represents inputs providing pre-generated spikes into the model. The mechanisms required for cell models are described by "model_template", with possible values including references to a NeuroML version 2 file or a NEURON hoc template. The reserved "morphology" attribute references a morphology file (e. g. , in the widely used SWC format) and the "dynamics_params" references files that can be optionally used to initialize or overwrite electrophysiological attribute values defined by the template. In Table  1 , node types with user chosen ids 0 and 1 are built using hoc templates from the Allen Cell Types Database (http://celltypes.
For instance, if a model has two types of neurons, "excitatory" and "inhibitory", they both may have the same lists of attributes, but different attribute values (e. g. , different magnitudes of the potassium Hodgkin-Huxley conductance at the soma); both types can then be stored in the same group, which simply provides different values for their attributes. The SONATA implementation of populations and groups utilizes HDF5 groups and datasets (see Online Documentation). (A, B ) Two examples are shown that demonstrate how for each node one can find its model attributes in either the node_group (for individually unique attributes) or the node_types table (for globally shared attributes). In (A), the unique attributes are only the node locations (x, y, z), indicated by empty triangles and circles on the left.
Activity format design. The SONATA activity format (also referred to as reports) is designed to efficiently support three types of data: spike trains, time series for node elements (e. g. , membrane voltage or Ca 2+ concentration in cell compartments) and time series that are not associated with specific node elements (such as voltages recorded with extracellular probes). The file formats are based on HDF5. The data stored in a spike train report consists of a series of node identifiers and spike times, stored in separate HDF5 datasets. Spike train reports are the default output of a simulation. Note that a pre-computed spike train, stored in the same SONATA format, can also be used to drive simulations with a desired input pattern. (SONATA format currently does not include software-specific spike generators such as NEURON's NetStim class, but support for these can be added per user requests; as of now, the format instead utilizes the more general approach of providing pre-generated spike trains.
) A node element report consists of a set of variables which are sampled at a fixed rate for some elements of interest from a selected set of cells (an element can be an electrical compartment or an individual synapse). Elements are given an identifier unique to the node and also contain a reference to their parent node's node_id. This allows elements of a specific node to be readily accessed through its node_id. One can therefore easily alter elements local to a given node without affecting the rest of the network. Note that elements can be divided into multiple parts, each with an identifier composed of an integer and an optional float value. This allows, e. g. , for representation of multiple electrical compartments in a single morphological section, with the datasets element_id and element_pos specifying the compartment's section id and its relative position within the node.
If the element_pos dataset is not present, for every recorded section all its compartments will be reported and they will appear in the dataset in morphological order. The time series associated with each element can be membrane voltage, synaptic current, or any other variable. In the report, a simulation frame is the set of all values reported at a given timestamp and a trace is the full time series of all values associated with one element (Fig  3A ). The requirements we followed in designing the node element report were: (i) support for large data sets both in total size (terabytes) and number of elements (millions of cells using multi-compartment models), (ii) random read access to specific frames and elements within a frame, (iii) high performance for different read access patterns (especially full frames and full cell traces) and (iv) high performance sequential parallel writing of full frames.
In the resulting design, data are stored in a single N×M matrix dataset, with rows being frames and columns being traces, whereas extra metadata provides a mapping between (cell, element) identifiers and columns within the frame (Fig  3A ). The format provides substantial flexibility, in particular permitting one to save different types and amounts of information for different cells. For example, one can choose to save membrane voltage and synaptic currents for all compartments and all synapses for a few cells, only somatic membrane voltage for several other cells, and nothing at all for all the other cells. This design also readily represents non-cell-element time series reports. In this case, instead of the cell elements, each row represents a channel storing a particular time series-for example, an electrode at which the extracellular voltage is recorded.
Performance benchmarks. Fig 3B  -3E  illustrates the effective I/O bandwidth (amount of useful data read/written per time unit) of SONATA multi-compartment and single-compartment reports, using 26,576 neurons (41,389,269 reported cell elements) with 1,000 time steps for the former and 217,000 neurons with 130,000 time steps for the latter (see Methods). We considered (i) the amount of data read/written, (ii) HDF5 chunk dimensions, (iii) only for write benchmarksthe amount of data written at each write operation (block size per process), and (iv) only for read benchmarksthe direction in which data is accessed (by frames or by traces). We did not consider the latter option in the write benchmark because simulators typically generate data which is ordered temporally, i. e. in frames. Note that HDF5 provides a storage layout in which the dataset is split into fixed size "chunks" (see Methods).
(Note that in the case of single-compartment reports, the total amount of data generated is significantly smaller compared to a multi-compartment report with the same simulation parameters, such as number of neurons and number of timesteps. Therefore, in order to split the HDF5 dataset into multiple chunks, the write block size is smaller than the multi-compartment write block size. ) Here the performance is also affected by the fact that, in some cases, multiple processes write to the same HDF5 chunk, which leads to lower effective bandwidth (compare 4K × 512 vs 4K × 1K). The read performance tests (Fig  3D  and 3E ) were run on a single-node, single-thread configuration, because this is the typical scenario of analysis and visualization use cases. In all cases, read bandwidth improves as the number of contiguous cells per operation increases and reaches 1 GiB/s and above.
PySONATA. PySONATA is a Python based API for reading SONATA files, opensourced under a BSD license and maintained at https://github. com/AllenInstitute/sonata. Users wishing to begin integrating the SONATA format into their own software are encouraged to use the PySONATA Python modules. Examples of how to use the module can be found at https://github. com/AllenInstitute/sonata/tree/master/tutorials/pySonata. The brain modeling toolkit. The Brain Modeling Toolkit (BMTK; https://github. com/ AllenInstitute/bmtk) is a Python based package for building, simulating and analyzing largescale neural networks across different levels of resolution. The BMTK is open-sourced under a BSD-3 license and has full support for generating and reading the SONATA data format (Fig  5 ). Modelers can use the BMTK Builder submodule to create their own SONATA based networks from scratch.
It supports cell template files, electrophysiological parameters, and morphology from the Allen Cell Types Database (http://celltypes. brain-map. org/) [42, 43]  as well as other cell model formats, including NeuroML version 2  [27, 28] , NEURON hoc files  [16] , or even user defined Python functions. For simulations, BMTK relies on an increasing array of simulation engines (NEURON  [16] , NEST  [17] , Dipde  [44] , etc. ), which allow users to run simulations of SONATA networks using either multi-compartment, point, or population based representations. The results of these simulations, regardless of the underlying simulator used to run them, are transformed into SONATA output format, allowing networks built and run with BMTK to be analyzed and visualized by any third-party software that supports SONATA.
Brion/Brain. The Blue Brain's C++ libraries for handling large scale data and simulation setup, Brion/Brain (https://github. com/BlueBrain/Brion), provide partial support for SONATA. Currently Brion provides a low level API to read circuit and simulation JSON configurations, spike and multi-compartment simulation outputs, SWC morphologies and query nodes in HDF5 files. It also provides a single threaded writer for multi-compartment simulation output reports. Brain provides a higher level API that makes it easier to work with full networks. All this functionality is also available in Python through the associated Python wrapping module. libSONATA. Blue Brain's libSONATA (https://github. com/BlueBrain/libsonata) is a library that provides support to read SONATA files. The library is open-sourced under a LGPLv3 license and offers an API for both Python and C++ applications.
Currently libSO-NATA supports reading circuit files, including nodes and edges populations. RTNeuron. Blue Brain's RTNeuron  [45]  is a framework for visualizing detailed neuronal network models and simulations. As it relies on Brion/Brain for data access, it currently provides basic support to visualize SONATA circuits and simulations. For instance, Fig 5B illustrates the RTNeuron visualization of a model of 300 biophysically detailed neurons, provided as an example in the SONATA specification GitHub repository (https://github. com/ AllenInstitute/sonata/tree/master/examples/300_cells). Here, one can see neuronal morphologies and the distribution of membrane voltage across the electrical compartments comprising these morphologies as the simulation evolves over time. pyNeuroML. NeuroML  [27, 28]  is a standardized format based on XML for declaratively describing models of neurons and networks in computational neuroscience.
Cellular models which can be described range from simple point neurons (e. g. leaky integrate and fire) to multicompartmental neuron models with multiple active conductances. Networks of these cells can be specified, detailing the 3D positions or populations, connectivity between them and stimulus applied to drive the network activity. Multiple libraries have been created to support user adoption of the NeuroML language, including jNeuroML (https://github. com/NeuroML/jNeuroML) in the Java language and pyNeuroML (https://github. com/NeuroML/pyNeuroML) in Python. The latter package also gives access to all of the functionality of jNeuroML (including the ability to convert NeuroML to simulator specific code, e. g. for NEURON) through Python scripts, by bundling a binary copy of the library. PyNeuroML has recently added support for importing networks and simulations specified in the SONATA format and converting them to NeuroML.
A related package currently under development, NeuroMLlite (https://github. com/NeuroML/NeuroMLlite) allows compact description of networks and can export the generated structures to SONATA. PyNN. PyNN is a simulator-agnostic Python API for describing network models of point neurons, and simulation experiments with such models  [23]. A reference implementation of the API for the NEURON, NEST and Brian simulators is available (http://neuralensemble. org/ PyNN), and a number of other simulation tools, including neuromorphic hardware systems, have implemented the API  [46, 47]. PyNN models can be converted to and from the NeuroML and SONATA formats with a single function call. Fig 5E  illustrates  an example where a model in SONATA format was loaded using the PyNN "serialization" module, a simulation was carried out using the PyNN NEST backend, and simulation output was saved in the SONATA format.
NetPyNE network model instantiations can be converted to and from the NeuroML and SONATA formats. SONATA complements NetPyNE by providing a standardized and efficient format to store and exchange large network models. This enables using other simulation tools to run and explore models developed with NetPyNE, and vice versa. As an example, we imported the 300-cell SONATA example with multicompartment cells into NetPyNE, visualized it using the NetPyNE GUI ( Neurodata without borders: Neurophysiology 2. 0. Neurodata Without Borders: Neurophysiology (NWB:N) 2. 0 is a data format for standardizing experimental data across systems neuroscience. We developed an extension for NWB:N 2. 0 to accommodate large-scale simulation data, and developed a conversion script from SONATA to NWB:N 2. 0 (https://github.
Discussion We have described SONATA, an open-source data format developed to answer the challenges of modern computational neuroscience, especially those inherent in large-scale data-driven modeling of brain networks. It is designed for memory and computational efficiency, as well as for working across multiple platforms, and at the same time enabling as much flexibility as possible for diverse applications. To achieve this, SONATA relies on commonly used data formats such as CSV, HDF5, and JSON, which can be used across platforms, can be read and written by many existing libraries in various programming languages, and (especially in the case of HDF5) have been proven to work efficiently in parallel computations with very large datasets. The SONATA specifications include network descriptions, simulation configuration, and input or output activity.
Initially developed jointly by the Allen Institute and the Blue Brain Project, SONATA is now supported by tools from many teams. As described above, tools such as BMTK  [26] , RTNeuron  [45] , PyNN  [23] , NeuroML  [27, 28] , and NetPyNE  [24]  include SONATA support. Functionality for conversion between SONATA and NWB:N [32] also exists. Further adoption of the SONATA framework will be beneficial for the projects aiming at increasing reproducibility across neuroscience, such as in systematic validation of neural data in the SciUnit project (https://scidash. org/sciunit. html). The SONATA data format and framework are reflected in the free and open-source PySONATA project hosted on GitHub (https://github. com/AllenInstitute/sonata), which is intended as a key resource for those wishing to add support for SONATA to their As an open living format, SONATA may be extended in the future to reflect developments in modeling and in experimental neuroscience.
HDF5 (Hierarchical Data Format version 5) is a technology designed for storing very large heterogeneous data collections and their metadata in a single container file. HDF5 defines a binary container file format for which the HDF Group provides an implementation in C. Bindings for several other languages exist as well. Basic concepts of HDF5 include groups, datasets and attributes. Making an analogy to filesystems, groups are similar to directories and datasets to files. The main differences between HDF5 and a general purpose filesystem are that a) a dataset is not a stream of bytes like a file, but consists of a multidimensional array with a single data type for all values and that b) groups and datasets can be annotated by means of attributes. HDF5 defines some basic data types common to most programming languages: integers, floats, strings.
Simulation output benchmarks. The simulation output benchmarks (Fig  3 ) were run on the aforementioned HPE SGI 8600 system. Since most simulators can run in parallel (multithread and/or multi-process), the benchmarking of the report generation was also done in parallel, on 16 nodes and 36 processes per node (using 1 core per process). All processes were periodically dumping data to a single, shared HDF5 file in the SONATA format. At each write operation, each process was writing several columns at its designated frame/trace region. The amount of data written at each operation is presented as the "Write block size per process" illustrated in the performance plots (the write block size applies for each process and for each write operation). Write benchmarks made use of the Neuromapp library (https://github.
Brain provides a higher level API that makes it easier to work with full networks. All this functionality is also available in Python through the associated Python wrapping module. libSONATA. Blue Brain's libSONATA (https://github. com/BlueBrain/libsonata) is a library that provides support to read SONATA files. The library is open-sourced under aLGPLv3 license and offers an API for both Python and C++ applications. Currently libSO-NATA supports reading circuit files, including nodes and edges populations. RTNeuron. Blue Brain's RTNeuron [45]  is a framework for visualizing detailed neuronal network models and simulations. As it relies on Brion/Brain for data access, it currently provides basic support to visualize SONATA circuits and simulations. For instance, Fig 5B illustrates the RTNeuron visualization of a model of 300 biophysically detailed neurons, provided as an example in the SONATA specification GitHub repository (https://github.
com/ AllenInstitute/sonata/tree/master/examples/300_cells). Here, one can see neuronal morphologies and the distribution of membrane voltage across the electrical compartments comprising these morphologies as the simulation evolves over time. pyNeuroML. NeuroML [27, 28]  is a standardized format based on XML for declaratively describing models of neurons and networks in computational neuroscience. Cellular models which can be described range from simple point neurons (e. g. leaky integrate and fire) to multicompartmental neuron models with multiple active conductances. Networks of these cells can be specified, detailing the 3D positions or populations, connectivity between them and stimulus applied to drive the network activity. Multiple libraries have been created to support user adoption of the NeuroML language, including jNeuroML (https://github.
com/NeuroML/jNeuroML) in the Java language and pyNeuroML (https://github. com/NeuroML/pyNeuroML) in Python. The latter package also gives access to all of the functionality of jNeuroML (including the ability to convert NeuroML to simulator specific code, e. g. for NEURON) through Python scripts, by bundling a binary copy of the library. PyNeuroML has recently added support for importing networks and simulations specified in the SONATA format and converting them to NeuroML. A related package currently under development, NeuroMLlite (https://github. com/NeuroML/NeuroMLlite) allows compact description of networks and can export the generated structures to SONATA. Fig 5E shows a simulation of 300 integrate and fire cells in SONATA which has been imported by pyNeuroML, converted to NeuroML and executed in the NEURON simulator.
This work was funded by the  Allen Institute  ( KD, YNB, SLG, AA ),  Paris-Saclay Institute of Neuroscience (APD) ,  Lawrence Berkeley National Laboratory  and  NIH  via grant  U19 NS104590  (BKD). This project/research (JH, JP, AD, MG, JGK, WAHVG, AVP, EM,  J-DC ) has received funding from the  European Union 's standardized files and provides much flexibility for adding new conventions or extensions. SONATA is already supported by several popular tools for model building, simulations, and visualization. It is free and open for everyone to use and build upon and will enable increased efficiency, reproducibility, and scientific exchange in the community. This is a  PLOS Computational Biology Software paper. Horizon 2020 Framework Programme for Research and Innovation  under the  Specific Grant  Agreement No.