dataesr/scientific-paragraphs-categorization huggingface dataset:
The API is rather low-level, consisting of a number of functions and low-level data structures such as numerical arrays. BioSig (http://biosig. sourceforge. net/) is a cross-platform open source library for biomedical signal processing (EEG, ECG, etc. ), available for MATLAB/Octave and for C/C++ (with wrappers for Python), which includes support for a large number of biomedical file formats. We have developed a novel, object-oriented API for representing electrophysiology data in memory and for reading/writing such data from/to multiple file formats. The Neo (from Neuroscience Electrophysiology Objects) API is defined as an object model, and is intended to be able to represent any electrophysiology dataset, from in vitro patch-clamp recordings through multi-electrode array recordings to EEG.
We have implemented the Neo object model as a cross-platform, open-source Python package, but it should be possible to implement it in any object-oriented programming language. An object-oriented API is a natural fit for neuroscience data, and in particular makes it easier to encapsulate data and the associated metadata, hence preserving the information about the experimental context that is needed to correctly interpret and analyze the data. The principal goal of developing Neo is to improve the interoperability of software for working with electrophysiology data. Such interoperability includes the exchange of data objects between software tools in memory as well as via the filesystem. The existence of a Neo implementation in a given language means, for example, that one scientist or research group can develop a library for electrophysiology data analysis and another a library for visualization of such data, then a third scientist can easily combine both libraries in his/her own workflow, without having to worry about plumbing the tools together, taking care of correct metadata conversion, etc.
By having an object model that is shared across programming languages, we hope to make conversion of data analysis routines between languages more straightforward. In this article we describe the considerations which led to the design of the API, we describe the object model and its Python implementation, then give some examples of using the Neo Python package in real-world situations. Finally, we discuss the advantages and disadvantages of the Neo approach with respect to the other approaches to data sharing and software interoperability mentioned above, and outline possible future directions.
SCOPE It was decided that the object model should contain only classes for representing electrophysiology data and for reading/writing such data from/to files or databases. Further, the behavior of these objects should allow only basic manipulations such as rearranging datasets or extracting sub-sets. In particular, we wished to specifically exclude visualization or analysis methods, even such simple ones as taking the mean. The motivation for this was to keep Neo as lightweight as possible. In our experience developing open-source software, small, focused libraries are much more likely to see widespread uptake. In future, we expect to extend the object model to support other data types closely-related to electrophysiology, such as calcium imaging and behavioral data, but these were not included in the initial design so as to avoid diluting our efforts.
• AnalogSignalArray: a record of a multichannel continuous analog signal with a fixed sampling interval, for example from an multi-electrode array or from an EEG recording. • Spike: one action potential characterized by its time and waveform. • SpikeTrain: a set of action potentials (spikes) emitted by the same unit in a period of time (with optional waveforms). Such data is generated by spike sorting from multi-electrode recordings, and directly by simulations. For each data type, the object model provides both a scalar representation (a single instance of a data item, for example, AnalogSignal and Spike) and an array representation (e. g. , AnalogSignalArray and SpikeTrain) in order to provide a choice between simplicity and efficiency in data analysis. For example, when working with single-electrode data, using the scalar objects is simpler and produces code that is easier to understand, without having to deal with array indexing.
IMPLEMENTATION IN PYTHON When implementing the object model in a specific language, easeof-use considerations come to the fore. If a package or library is to gain widespread adoption, it should be easy to build and install on different platforms, and should be interoperable, as much as possible, with existing tools and existing code for handling electrophysiology data in that language. In the case of Python, the latter consideration dictates that a Neo implementation must be based on NumPy  (Oliphant, 2007) , the de facto standard for numerical computing in Python. NumPy provides a powerful, N-dimensional array object that can contain integers, floating point numbers, strings or other Python objects. A given numerical array can have any combination of precision and byte-order. Most array operations are implemented in C or Fortran for efficiency.
The principal choice in implementing the Neo object model in Python is whether the classes for data objects should inherit from the NumPy ndarray class (a Neo class is a NumPy array) or whether to use composition (where the Neo class has a data attribute which is a NumPy array). Inheritance has two important advantages: (i) existing code that works with NumPy arrays will work with Neo data objects without modification; (ii) Neo objects gain most of the functionality of NumPy arrays "for free," i. e. , we do not have to re-implement this functionality. A number of operations on, and methods of, NumPy arrays have to be re-implemented or extended for Neo objects, in order to preserve the extra metadata carried by Neo objects, but these would in any case have to be implemented if taking the composition approach.
The principal disadvantage of the inheritance approach is complexity: creation of subclasses is one of the more difficult parts of the NumPy API. After weighing up these considerations, we chose to use inheritance. The second major choice was how to represent the measurement units of the data. Several Python packages are available for handling units and physical quantities in Python. The decision to use inheritance for NumPy support narrows down the choice. We chose to use the Quantities package (http://pythonhosted. org/quantities/), which provides a Quantity class that inherits from the NumPy ndarray class and adds a concept of physical dimensions and units. This means that trying to add a Quantity array with dimensions of voltage to an array with dimensions of current will produce an exception, while adding an array in millivolts to an array in volts will perform scaling so as to give the correct result.
The particular choice of Quantities is not limiting. It would be possible to replace it with another units package that subclasses ndarray, such as Pint (http://pint. readthedocs. org/), without changing the Neo interface. In summary, the classes for the Neo data objects all inherit from Quantity, and so gain checks for dimensional consistency for free, in addition to the large amount of functionality inherited from ndarray. Neo objects add further checks (for example, trying to add together two AnalogSignals with different sampling rates produces an exception) and further operations (for example, it is possible to slice AnalogSignals and SpikeTrains according to time as well as according to index). Other Python packages for working with time series data do not fill the niche that Neo does.
The core function of Neo is data representation. The Pandas package (http://pandas. pydata. org) is a general toolkit for working with heterogeneous data. Neo contains functionality specific for neural data analysis, such as grouping of channels by anatomical location. Another package, nitime (http:// nipy. org/nitime), provides algorithms for time-series analysis that aretailoredtowardneuroimagingdata. Incontrast,theNeopackage intentionallydoesnotprovidealgorithmsfordataanalysissincethis will vary widely across users. Overall, Neo provides functionality that is specific to neuroscience data (unlike Pandas) but not specific to particular applications within neuroscience (unlike nitime). This means that Neo is ideally situated to be a common format for neuroscientists, who may then analyze the data using the tools provided by other packages as desired.
The Neo package is available from the Python Package Index (https://pypi. python. org/pypi/neo/) and as Debian and Ubuntu packages through NeuroDebian  (Halchenko and Hanke, 2012). The core of Neo works with both Python 2 (version 2. 6 or later) and Python 3 (version 3. 2 or later). For some file formats, only Python 2 is currently supported. At the time of writing, the latest version is 0. 3. 1. Full documentation is available online at http:// neuralensemble. org/neo/. There is an extensive test suite, and the project makes use of continuous integration (https://travis-ci. org/NeuralEnsemble/python-neo) to rapidly catch regressions.
READING AND WRITING DATA WITH MULTIPLE FILE FORMATS In addition to implementing the Neo object model as a Python API, the Neo Python package also provides a set of input/output (IO) modules for various neurophysiology file formats, including proprietary formats (e. g. , Plexon, Spike2, NeuroExplorer, Axon, AlphaOmega, Micromed and Tucker-Davis), the formats used by various open-source and/or freely distributed tools (e. g. , Klustakwik, Elan, WinEdr) and generic file formats such as MATLAB, ASCII and HDF5. In most cases, the proprietary formats are read-only, while the more open and generic formats support both reading and writing. The Neo documentation provides guidelines for anyone wishing to add support for a new format. The inclusion of support for multiple file formats in the Neo package has several benefits: (i) it demonstrates the universality of the object model, which is able to represent the data from all of these different formats; (ii) it is an important part of realizing our goal of improving interoperability of different tools and the ease of sharing data between different projects; (iii) it provides an immediate benefit to users, thus driving uptake of the Neo software; (iv) tool developers can use Neo as a basis and immediately gain the benefit of supporting multiple file formats, without them having to implement such support themselves.
For each file format there is a separate Python class, each of which implements the same interface. Reading data for a given format can be as simple as: from neo. io import MyFormatIO reader = MyFormatIO(filename="myfile. dat") data = reader. read() However, given the large size of many neurophysiology datasets, it is not always desirable to load all the data into memory at once. The full interface therefore offers more fine-grained control. All read functions have two optional parameters: cascading and lazy. When cascading is set to false, only a single object is loaded and none of its references to other objects are populated. The lazy parameter does not influence whether linked objects are loaded, but determines the treatment of data objects: when lazy is true, the numerical data is not loaded, instead empty data objects are created including all properties and metadata.
This allows examining the contents of files with a substantially smaller memory footprint. IO classes can implement a method to load the full version of a lazily loaded object once the numerical data is required. The IO API also takes into account that a dataset does not necessarily come from a single file. Some software provides recordings in a directory or in a database. Such cases are taken into account through the attribute mode in each IO class. The behavior is changed only at instantiation: from neo. io import MyFormat1IO, MyFormat2IO reader1 = MyFormat1IO(filename="myfile. dat") # MyFormat1IO. mode = 'file' reader2 = MyFormat2IO(dirname="/path_to_dataset") # MyFormat2IO. mode = 'dir' Some file formats internally store arrays in a compact continuous way. In such cases, the class implementations use the NumPy "memmap" mechanism.
This reduces the memory footprint and allows transparent access to files that do not fit into memory. For the implementation of file readers, an early difficulty was collecting information about the specifications for proprietary file formats. This tedious task was accomplished using several approaches: (i) asking manufacturers to open the specifications; (ii) reading code from other packages; (iii) reading C header files; (iv) basic reverse engineering with hexadecimal editors. Nowadays, many manufacturers provide their file specifications in the user documentation. The main difficulty remains in the versioning of these specifications. In general, the internal file formats do not change much over time, but sometimes some problematic changes occur, making the classes non backward compatible.
RECORDING MULTIPLE TRIALS FROM MULTIPLE CHANNELS In this example we suppose that we have recorded from an 8-channel probe, and that we have recorded three trials. We therefore have a total of 8 × 3 = 24 signals, each represented by an AnalogSignal object. As shown in Figure  3 , our dataset (contained in a single Block, not explicitly shown) contains: • Three Segment objects, each representing data from a single trial, • One RecordingChannelGroup, composed of eight RecordingChannel objects. This information can be traversed or averaged in two different ways. The first is temporal traversal (by segment). For example, suppose you wish to correlate the overall neural response with the stimulus that was delivered in each segment. In this example, we treat each segment in turn, averaging over the channels in each: The second alternative is spatial traversal of the data (by channel), with averaging over trials.
RECORDING SPIKES FROM MULTIPLE TETRODES Here is a similar example in which we have recorded with two tetrodes and extracted spikes from the extra-cellular signals. The spike times are contained in SpikeTrain objects. As shown in Figure  4 , our data set (again contained in a Block) contains: • Three Segments (one per trial). • Two RecordingChannelGroups (one per tetrode), which contain: • Four RecordingChannels each • Two Unit objects (= two neurons) for the first RecordingChannelGroup • Five Units for the second RecordingChannelGroup. In total we have 3 × 7 = 21 SpikeTrains in this Block. There are three ways to access the spike train data: by segment, by recording channel or by unit. The code samples shown here are extracted from a complete example script available in the Neo source code distribution, see.
DISCUSSION We have presented here an object-oriented API for handling electrophysiology data in neuroscience, and an implementation of this API in the Python programming language. We have shown how this model is designed to allow for convenient and efficient representation of all kinds of electrophysiological data, from spikes to broadband continuous signals, with support for all common recording technologies, including multi-channel systems like tetrodes. Furthermore, we have illustrated the practical use of the Neo Python package through examples in typical real-world situations. The generality of the Neo object model for the representation of electrophysiological data coupled with the availability of a large set of IO modules for common file formats in the Python package makes Neo a very convenient and powerful base package to build upon when developing software to manipulate, analyze or display electrophysiological data in Python.
PROJECTS USING NEO The success of the Neo project and the suitability of the object model will be determined by whether other software tools for working with neurophysiology data adopt the Neo object model and whether they gain benefits from doing so. A number of projects and laboratories are already using Neo. The German INCF Node (G-Node) has adopted the Neo model for the representation of electrophysiological data in its data management platform (https://portal. g-node. org/data/). The purpose of the platform is to provide a centralised system for FIGURE 4 | An illustration of recording spikes from multiple tetrodes. After spike sorting, two units have been isolated from the first tetrode, and five units from the second. The SpikeTrain objects may optionally also contain the action potential waveforms for each spike (not shown).
organising and sharing of electrophysiology data. A major obstacle to data sharing and re-use in the field of electrophysiology is the large variety of data formats. The G-Node platform provides a unified way of representing electrophysiological data according to the Neo object model, as well as automated format conversion utilizing the Neo IO modules. Being compliant with the Neo model, the system's API achieves interoperability with other tools that use the Neo objects and facilitates integration of data access and data analysis. G-Node offers a Python client tool that provides a frontend to the data hosted on the G-Node data platform in terms of the Neo Python objects  (Sobolev et al. , 2014). The G-Node platform complements the Neo data representation with metadata storage using an unrestrictive format  (Grewe et al.
, 2011) , enabling extensive data annotation in a standardized way. PyNN  (Davison et al. , 2009 ) is a Python API for simulatorindependent neuronal network simulations. One aspect of simulator independence is reformatting the data produced by each simulator into a common format. Originally, this common format was based on NumPy arrays together with a small amount of metadata, which could be saved to file in a small number of non-standard formats. As of version 0. 8, PyNN has adopted Neo as the common format for output data. This has had several benefits: (i) the hierarchical structure of Neo allows much more of the structure of the simulation experiment to be preserved; (ii) a much more complete set of metadata can easily be provided; (iii) a much wider range of file formats is now available; (iv) data recorded in biological experiments and data generated by simulations can now be analyzed and visualized with the same tools; (v) reduced development effort for the PyNN development community, since a large piece of functionality has been "out-sourced" to Neo.
Plug-ins operate on Neo data objects. Because of the standard data model provided by Neo, plug-ins work with data from many different formats. This enables Spyke Viewer users to conveniently share methods implemented in plug-ins. OpenElectrophy  (Garcia and Fourcaud-Trocmé, 2009 ) is a graphical user interface built on top of Neo. This software implements four independent components: (i) SQL database management for Neo objects; (ii) a fast viewer for Neo objects; (iii) a complete off-line spike sorting tool-chain; (iv) a time-frequency toolbox (fast multi-channel wavelet scalogram plotting and transient oscillation detection). The main difficulty when designing a spike sorting tool-chain is the management of data in various and complex situations: (i) the hardware setup may use a single electrode, several independent electrodes, several independent https://github.
com/NeuralEnsemble/python-neo/blob/master/ examples/generated_data. py. N-trodes or electrode arrays; (ii) the protocol may involve one continuous recording or multiple chunks of recording; (iii) the nature of the recording may be full band signals, filtered signals or pre-detected spikes. Many spike sorting tools take into account only one combination of these situations because it is rather complex to support all the possible combinations. The Neo object model allows all these situations to be dealt with transparently in OpenElectrophy. Datajongleur (https://github. com/G-Node/Datajongleur) is a collection of scientific data objects. It supports management of data objects with numerical behavior in combination with relational databases, like SQLite (http://www. sqlite. org/, file based), or PostgreSQL (http://www.
postgresql. org/, server based). Datajongleur provides a Neo implementation that allows storing Neo objects in a relational database. As a consequence of the common use of the Neo Python package, it is now straightforward to take recorded data from experiments or simulation data generated by PyNN, store them on the G-Node data platform or in a relational database using Datajongleur, then analyze and visualize them in Spyke Viewer or OpenElectrophy, in a seamless pipeline. Although we have focused in this section on the advantages of Neo for tool developers, the Neo Python package is equally suitable for use by individual scientists analysing their data with their own scripts.
RELATED INTEROPERABILITY EFFORTS As outlined in the Introduction, there have been a number of previous efforts to solve the problem of incompatible file formats in neurophysiology and related fields, such as clinical electrophysiology. BioSig (http://biosig. sourceforge. net/) is a cross-platform open source library for biomedical signal processing, available for MATLAB/Octave and for C/C++. Python bindings can be generated from the C/C++ library using SWIG. It includes support for a large number of file formats. Neuroshare (http:// neuroshare. sourceforge. net/) is an API that must be implemented as a library by each equipment manufacturer that wishes to support it. The libraries are not in general open source and most of them are only available for Windows. NiBabel (http://nipy. sourceforge.
Kemp (2004)  compares EDF+ and SignalML, concluding that the SignalML approach is more flexible but that this flexibility comes at a price in terms of stability and increased effort for users. We expect that Neo will be interoperable with, and broadly complementary to these existing efforts. For example, the Neo Python package already offers a NeuroshareIO module for reading data files via the Neuroshare libraries, and it should be straightforward to add IO modules for EDF/EDF+, GDF and the HDF5-based standard file format being developed by the INCF. The particular advantages of the Neo Python package are (i) its rich object model can capture all of the complexity typically found in neurophysiology experimental protocols; (ii) it is written in pure Python without any C/C++ dependencies except the widely available NumPy package; as such it is easy to install and highly cross-platform; (iii) it focuses on neuroscience research, whereas most of the other tools cited above have a biomedical focus; (iv) by using the Python programming language, users of Neo gain all of the benefits associated with that language, such as readability, increased productivity, ease of use for system integration, and access to a powerful ecosystem for scientific computing.
SUSTAINABILITY Neo is developed as an open, collaborative project at http:// neuralensemble. org/neo, using GitHub for version control and issue tracking. Anyone is welcome to propose and implement changes, which will then be reviewed by one or more of the other contributors, and a decision on whether to accept the changes taken by consensus. 19 people from ten separate institutions have at some time contributed code or documentation to the Neo project; six people from five institutions have contributed within the past 12 months. This distributed development effort, in which the collaboration arises informally from shared needs and interests rather than formally through grant funding, gives us confidence that Neo development is sustainable.