Package: tank-lab-to-nwb
tank-lab-to-nwbNWB conversion scripts and tutorials. A collaboration with theTank lab, funded by the Simons Foundation.Install$pipinstalltank-lab-to-nwbUsageThere are two ways to go about converting Neuropixel and Virmen behavior data.(1) Theprimary processing pipelinesynchronizes the task data with the electrophysiology data
through  TTL pulse and writes the spiking output to the same NWB file.The required arguments for the use of the relevant functions are denoted in the comments of their
respective sections of the conversion script. These include the file or
folder locations of the data to be converted to NWB format, as well as several optional fields
such as Subject information (species/age/weight).After editing the conversion scriptconvert_towers.pywith the proper path for Neuropixel and behavior data,
the conversion can be executed from the terminal:$cdtank-lab-to-nwb
$pythontank_lab_to_nwb/convert_towers_task/convert_towers.pyAlternatively, the conversion can be done using a custom tailored jupyter notebookspikeinterface_pipeline.ipynbthat can be launched from the terminal:$jupyternotebooknotebooks/spikeinterface_pipeline.ipynb(2) The NWBFile can be inspected by reading it from a python script:frompynwbimportNWBHDF5IOfile_path='TowersTask_stub.nwb'io=NWBHDF5IO(file_path,'r')nwb=io.read()print(nwb)Alternatively, the NWB data can be visualized withnwb-jupyter-widgetsin a jupyter notebook:$jupyternotebooknotebooks/towers_task_custom_widget.ipynbfrompynwbimportNWBHDF5IOfromnwbwidgetsimportnwb2widgetfromtank_lab_to_nwb.nwbwidgetsimportcustom_timeseries_widget_for_behaviorfromnwbwidgets.viewimportdefault_neurodata_vis_specimportpynwbfile_path='TowersTask_stub.nwb'io=NWBHDF5IO(file_path,'r')nwb=io.read()default_neurodata_vis_spec[pynwb.TimeSeries]=custom_timeseries_widget_for_behaviornwb2widget(nwb)BackgroundBehavioral data mappingThe behavioral data is contained in.matfiles similar to the form:PoissonBlocksReboot4_cohort4_Bezos3_E65_T_20180202.matThis matlab file contains a struct (log) which contains several fields relevant for conversion.
The list of fields that are extracted from this struct can be found below.NWBFileLocation in Virmen (.mat) fileLocation in NWB fileDescriptionlog.session.startnwb.session_start_timedatetimewhen session started[not in file]nwb.session_descriptionadditional information about session (optional)[name of file]nwb.session_idunique identifier of the sessionSubjectLocation in Virmen (.mat) fileLocation in NWB fileDescriptionlog.session.startnwb.subject.ageage (days) in isoformat (optional)[not in file]nwb.speciesinformation about the species (optional)log.animal.namenwb.subject_ididentifier of the subject[not in file]nwb.genotypeinformation about the genotype (optional)[not in file]nwb.sexinformation about the sex of the subject (optional)LabMetaDataThe lab specific metadata is populated intank_lab_to_nwb/convert_towers_task/virmenbehaviordatainterface.pyusing the custom extensionndx-tank-metadatabuilt for extending the NWB LabMetaData schema with the required fields:Location in Virmen (.mat) fileLocation in NWB fileDescriptionlog.version.codenwb.lab_meta_data['LabMetaData'].experiment_namename of experiment runlog.version.namenwb.lab_meta_data['LabMetaData'].world_file_namename of world runlog.animal.protocolnwb.lab_meta_data['LabMetaData'].protocol_namename of protocol runlog.animal.stimulusBanknwb.lab_meta_data['LabMetaData'].stimulus_bank_pathpath of stimulus bank filelog.version.repositorynwb.lab_meta_data['LabMetaData'].commit_idcommit id for session runlog.session.endnwb.lab_meta_data['LabMetaData'].session_end_timedatetimewhen session endedlog.version.rig.rignwb.lab_meta_data['LabMetaData'].locationname of rig where session was run[not in file]nwb.lab_meta_data['LabMetaData'].num_trialsnumber of trials in the session[not in file]nwb.lab_meta_data['LabMetaData'].session_performanceperformance of correct responses in % (optional)*log.version.rignwb.lab_meta_data['LabMetaData'].rig.fieldsrig informationlog.version.mazesnwb.lab_meta_data['LabMetaData'].mazes.to_dataframe()maze informationsession_performance can be edited fromvirmenbehaviordatainterface.py.RigRig information is converted from alog.version.rigstruct object to a dictionary like as in this example:{'rig': 'NPX',
 'simulationMode': 1,
 'hasDAQ': 1,
 'hasSyncComm': 0,
 'minIterationDT': 0.01,
 'arduinoPort': 'COM18',
 'sensorDotsPerRev': array([2469.2, 2469.2, 2469.2, 2469.2]),
 'ballCircumference': 63.8,
 'toroidXFormP1': 0.3879,
 'toroidXFormP2': 0.392,
 'colorAdjustment': array([0. , 0.4, 0.5]),
 'soundAdjustment': 0.2,
 'nidaqDevice': 1,
 'nidaqPort': 1,
 'nidaqLines': array([ 0, 11], dtype=int32),
 'syncClockChannel': 5,
 'syncDataChannel': 6,
 'rewardChannel': 0,
 'rewardSize': 0.004,
 'rewardDuration': 0.05,
 'laserChannel': 1,
 'rightPuffChannel': 2,
 'leftPuffChannel': 3}MazesMaze information is converted to aDynamicTableobject that can be converted to a pandas dataframe by calling.to_dataframe()as in this example:idworldlStartlCuelMemorycueDurationcueVisibleAtcueProbability...blockPerform0154510naninfinf...0.7113012020naninfinf...0.7213022020naninfinf...0.7313030020naninfinf...0.7413038020naninfinf...0.7EpochsLocation in Virmen (.mat) fileLocation in NWB fileDescriptionindex oflog.blockstructurenwb.intervals['epochs'].idnumber of epoch in sessionlog.block.mazeIDnwb.intervals['epochs'].maze_idnumber of maze in epochlog.block.mainMazeIDnwb.intervals['epochs'].main_maze_idnumber of maze of "highest" level for subjectlog.block.easyBlockFlagnwb.intervals['epochs'].easy_epoch1 if block was flagged as easy (maze_id < main_maze_id)log.block.firstTrialnwb.intervals['epochs'].first_trialfirst trial run in an epoch[not in file]nwb.intervals['epochs'].num_trialsnumber of trials for each epochlog.block.startnwb.intervals['epochs'].start_timedatetime when epoch started with respect to the start time of the sessionlog.block.durationnwb.intervals['epochs'].durationepoch duration in secondslog.block.rewardMiLnwb.intervals['epochs'].reward_mlml of reward in an epochTrialsLocation in Virmen (.mat) fileLocation in NWB fileDescription[not in file]nwb.intervals['trials'].idunique identifier of trial for all epochsindex oflog.block.trialnwb.intervals['trials'].trial_ididentifier of trial within an epochlog.block.trial.trialTypenwb.intervals['trials'].trial_typetype of trial (L=Left, R=Right)log.block.trial.choicenwb.intervals['trials'].choice(L=Left, R=Right, nil=Trial violation)log.block.trial.startnwb.intervals['trials'].start_timestart time of trial with respect to the start time of the epochlog.block.trial.durationnwb.intervals['trials'].durationduration of trial in secondslog.block.trial.iterationsnwb.intervals['trials'].iterationsnumber of frames in a triallog.block.trial.iCueEntrynwb.intervals['trials'].iCueEntryiteration # when subject entered cue regionlog.block.trial.iMemEntrynwb.intervals['trials'].iMemEntryiteration # when subject entered memory regionlog.block.trial.iTurnEntrynwb.intervals['trials'].iTurnEntryiteration # when subject entered turn regionlog.block.trial.iArmEntrynwb.intervals['trials'].iArmEntryiteration # when subject entered arm regionlog.block.trial.iBlanknwb.intervals['trials'].iBlankiteration # when screen if turned offlog.block.trial.cueCombonwb.intervals['trials'].left_cue_presence, nwb.intervals['trials'].right_cue_presenceindicates if nth cue appeared on left or rightlog.block.trial.cuePositionnwb.intervals['trials'].left_cue_position, nwb.intervals['trials'].right_cue_positionposition in maze for each cuelog.block.trial.cueOnsetnwb.intervals['trials'].left_cue_onset, nwb.intervals['trials'].right_cue_onsetiteration number when cues appeared in triallog.block.trial.cueOffsetnwb.intervals['trials'].left_cue_offset, nwb.intervals['trials'].right_cue_offsetiteration number when cues disappeared in triallog.block.trial.excessTravelnwb.intervals['trials'].excessTravelparameter that measures extra distance run by subjectlog.block.trial.rewardScalenwb.intervals['trials'].rewardScaleMultiplier of reward for each correct trialBehaviorPosition, ViewAngle, Velocity, CollisionLocation in Virmen (.mat) fileLocation in NWB fileDescriptionlog.block.trial.timenwb.processing['behavior'].data_interfaces['Time']time vector for each frame measured by Virmenlog.block.trial.positionnwb.processing['behavior'].data_interfaces['Position'].spatial_seriesposition matrix for each frame (X(cm), Y(cm))log.block.trial.positionnwb.processing['behavior'].data_interfaces['ViewAngle'].spatial_seriesviewAngle for each frame (degrees)log.block.trial.velocitynwb.processing['behavior'].data_interfaces['Velocity']velocity matrix for each frame (X(cm/s), Y(cm/s))log.block.trial.collisionnwb.processing['behavior'].data_interfaces['Collision']for each frame 1= collision detected