Package: nwb-conversion-tools
NWB conversion toolsNWB Conversion Tools is a package for creating NWB files by converting and
combining neural data in proprietary formats and adding essential metadata.Under heavy construction. API is changing rapidly.Features:Command line interfacePython APILeverages SpikeExtractor to support conversion from a number or proprietary formats.InstallationTo install the latest stable release ofnwb-conversion-toolsthough PyPI, type:pipinstallnwb-conversion-toolsFor more flexibility we recommend installing the latest version directly from GitHub. The following commands create an environment with all the required dependencies and the latest updates:gitclonehttps://github.com/catalystneuro/nwb-conversion-toolscdnwb-conversion-tools
condaenvcreate-fmake_env.yml
condaactivatenwb_conversion_envNote that this will install the package ineditable mode.Finally, if you prefer to avoidcondaaltogether, the following commands provide a clean installation within the current environment:pipinstallgit+https://github.com/catalystneuro/nwb-conversion-tools.git@masterDependenciesNWB Conversion Tools relies heavily onSpikeExtractorsfor electrophysiology and onROIExtractorsfor optophysiology data.You can use a graphical interface for your converter withNWB Web GUI.Cataloguev0.9.3Buzsáki Lab:buzsaki-lab-to-nwbThis project is an ongoing effort for the Ripple U19 conversion of extracellular electrophysiology data to NWB format, including final publishing of each dataset on DANDI. Currently spans 7 major publications and over 14 TB of data on theDANDI Archive. Most of the data consists of raw recordings, LFP, spike sorted units, and behavior with can consist of a mix of mental state tracking, position tracking through mazes, and trial stimulus events.Shenoy lab:shenoy-lab-to-nwb:The Shenoy lab is one of the pioneers in developing BCIs for people with paralysis. They are part of theBrainGateteam
and were the winners of the 2019BCI award.
They use extracellular recordings from Utah arrays and Neuropixels in primates.v0.9.2Brody Lab:brody-lab-to-nwbThe Brody lab has a long history with extracellular electrophysiology experiements spanning multiple acquisition systems. This project served two purposes - to allow the conversion of older data from Neuralynx and SpikeGadgets to NWB, and also their newer, larger data using Neuropixels (SpikeGLX). These recordings, some of which exceeded more than 250 GB (several hours worth!), were paired with rich trials tables containing catagorical events and temporal stimuli.v0.8.10Feldman Lab:feldman-lab-to-nwbThe Feldman lab utilizes a Neuropixels (SpikeGLX) system along with multiple sophisticated behavior systems for manipulating whisker stimulation in mice. These give rise to very complex trials tables tracking multiple event times throughout the experiments, including multiple event trains within trials.v0.8.1Hussaini Lab:hussaini-lab-to-nwbv0.7.2Movson lab:movshon-lab-to-nwbv0.7.0Tank Lab:tank-lab-to-nwbNeuropixel (SpikeGLX) recordings of subjects navigating a virtual reality! Behavior contains a huge variety of NWB data types including positional and view angle over time,  collision detection, and more! Paired with aspecific extensionfor parsing experiment metadata.Groh lab:mease-lab-to-nwbUtilizing the CED recording interface, this project paired ecephys channels with optogenetic stimulation via laser pulses, and mechnical pressure stimulation over time - all of which are channels of data extracted from the common.smrxfiles!Giocomo lab:giocomo-lab-to-nwbOther labs that use NWB standardAxel lab:axel-lab-to-nwbBrunton lab:brunton-lab-to-nwbBuffalo lab:buffalo-lab-data-to-nwbJaeger lab:jaeger-lab-to-nwbTolias lab:tolias-lab-to-nwbFor DevelopersRunning GIN tests locallynwb-conversion-toolsverifies the integrity of all code changes by running a full test suite on short examples of real data from the formats we support. There are two classes of tests in this regard;tests/test_internalsdoes not require any data to be present and represents the 'minimal' expected behavior for our package, whereastests/test_on_datarequires the user to both perform a full install of dependencies (pip install -r requirements-full.txt) as well as download the associated data for each modality.Install testing dependenciesWe provide two easy ways of installing all the dependencies required for testing:The first is acondabased solution that creates an environment with all the dependencies already installed.gitclonehttps://github.com/catalystneuro/nwb-conversion-toolscdnwb-conversion-tools
condaenvcreate-fmake_env_testing.yml
condaactivatenwb_conversion_testing_envNote that this will also installdataladwhich is the endorsed way of downloading the testing data pluspytestandpytest-covwhich are the tools that we use on our continuous integration suit.The same can be accomplished by usingpip. In a clean environment run:gitclonehttps://github.com/catalystneuro/nwb-conversion-toolscdnwb-conversion-tools
pipinstall.[test,full]Notice that this method does not installdatalad.Downloading the dataDatalad(conda install datalad) is the recommended way for downloading the data. To do this; simply call:For electrophysiology data:dataladinstall-rghttps://gin.g-node.org/NeuralEnsemble/ephy_testing_dataFor optical physiology data:dataladinstall-rghttps://gin.g-node.org/CatalystNeuro/ophys_testing_dataFor behavioral data:dataladinstall-rghttps://gin.g-node.org/CatalystNeuro/behavior_testing_dataTest configuration fileOnce the data is downloaded to your system, you must manually modify the config file (example) located in./tests/test_on_data/gin_test_config.jsonso its correspondingLOCAL_PATHkey points to the correct folder on your system that contains the dataset folder (e.g.,ephy_testing_datafor testingecephys). The code will automatically detect that the tests are being run locally, so all you need to do ensure the path is correct to your specific system.The output of these tests is, by default, stored in a temporary directory that is then cleaned after the tests finish running. To examine these files for quality assessment purposes, set the flagSAVE_OUTPUTS=truein thegin_test_config.jsonfile and modify the variableOUTPUT_PATHin the respective test if necessary.Build the documentationFor building the documentation locally, the following procedure can be followed. Create a clean environment and type
the following commands in your terminal:gitclonehttps://github.com/catalystneuro/nwb-conversion-toolscdnwb-conversion-tools
pipinstall-e.[docs]These commands install both the latest version of the repo and the dependencies necessary to build the documentation.
Note that the argument-emakes you installeditableNow, to build the documention issue the following command in your terminal:sphinx-build-bhtmldocs./docs/_build/This builds the html under/docs/_build/(from your root directory, where you have installednwb-conversion-tools). This allows you to review the outcome of the process localy before commiting code.