DOI: 10.5061/dryad.02v6wwqdf
title: Data from: behaviorMate: An Intranet of Things approach for adaptable control of behavioral and navigation-based experiments

description: Investigators conducting behavioral experiments often need precise control
 over the timing of the delivery of stimuli to subjects and to
 collect the precise times of the subsequent behavioral responses.
 Furthermore, investigators want fine-tuned control over how various
 multi-modal cues are presented. behaviorMate takes an
 "Intranet of Things" approach, using a networked system
 of hardware and software components for achieving these goals. The system
 outputs a file with integrated timestamp-event pairs that
 investigators can then format and process using their own
 analysis pipelines. We present an overview of the electronic components
 and GUI application that make up behaviorMate as well as
 mechanical designs for compatible experimental rigs to provide
 the reader with the ability to set up their own system. A wide variety of
 paradigms are supported, including goal oriented learning, random
 foraging, and context switching. We demonstrate behaviorMate’s
 utility and reliability with a range of use cases from several
 published studies and benchmark tests. Finally, we present
 experimental validation demonstrating different modalities of
 hippocampal place field studies. Both treadmill with burlap belt and
 virtual reality with running wheel paradigms were performed to
 confirm the efficacy and flexibility of the approach. Previous
 solutions rely on proprietary systems that may have large upfront costs or
 present frameworks that require customized software to be
 developed. behaviorMate uses open-source software and a flexible
 configuration system to mitigate both concerns. behaviorMate has a
 proven record for head-fixed imaging experiments and could be
 easily adopted for task control in a variety of experimental
 situations.
description: # Data from: behaviorMate: An Intranet of Things approach for adaptable
 control of behavioral and navigation-based experiments ## Date of Data
 Collection 2022-2023 ## Contributors * Jack Bowler: Department of
 Neurobiology University of Utah, Salt Lake City, UT * George
 Zakka: Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia
 University, New York, NY * Hyun Choong Yong: Mortimer B. Zuckerman Mind
 Brain Behavior Institute, Columbia University, New York, NY * Wenke Li:
 Oden Technologies, New York, NY * Bovey Rao: Mortimer B. Zuckerman Mind
 Brain Behavior Institute, Columbia University, New York, NY *
 Zhenrui Liao: University of Edinburgh, Edinburgh, United Kingdom *
 James Priestley: Brain Mind Institute, École polytechnique fédérale de
 Lausanne * Attila Losonczy: Mortimer B. Zuckerman Mind Brain Behavior
 Institute, Columbia University, New York, NY ## Overview This
 supplementary dataset provides the data used for benchmarking the
 reliability of behaviorMate's hardware and software components as
 well as the data used for experimental validation. Our benchmarking data
 includes log files containing real-time sensor data produced by
 behaviorMate and video data processed using OpenCV to validate this sensor
 data. The experimental validation contains the raw and processed imaging
 data used to evaluate the formation of place fields in animals during
 running trials in a behaviorMate setup. In addition, all experimental
 validation imaging data includes the corresponding behavior data,
 collected concurrently. ## Data Formats * .tdml (TDML Log Files) * This is
 the raw output produced by behaviorMate containing real-time sensor data
 (position, licking, reward releases, odor, sound, etc). This format is
 difficult to work with but has the advantage of being able to be written
 to in real-time. This useful when experiments are extremely long and the
 entire log cannot be stored in memory. * .py (Python Script Files) * Used
 to convert the TDML output produced by behaviorMate at the end of
 experiments into Python Pickle Files which are much easier to work with. *
 .avi (AVI Movie Files) * Contain 2 movies of a moving treadmill and 2
 movies of a moving VR wheel. In each movie, the room is complete dark and
 the only thing visible in the movie is an extremely small LED attached to
 the side of the wheel or treadmill. We convert the coordinate (X, Y)  of
 the current pixel location of the light source into a linear distance
 along the track. This is the ground truth compared to the linear track
 position provided by behaviorMate at each sampled time point. * ipynb
 (Jupyter Notebook Files) * Used to process all data related to
 benchmarking and create the benchmarking figures. * .yml (YAML Files) *
 Used to create a conda environment with Python and all required Python
 packages.  - .pkl (Python Pickle Files) * Several items in the dataset are
 stored as Python Pickles: * Behavior data after being converted from TDML
 to Pickle format. * Locations of the light source across time, extracted
 from the AVI movies. * Raw and Motion-Correct imaging data. * Converted
 behavior data corresponding to the above imaging data. \** Note:
 benchmarking was not performed with an animal on the rig. Hence, the
 benchmarking data has no associated imaging data. The imaging dataset
 provided has an associated behavior dataset and were both collected on
 different days than the benchmarking data. ## Recommend Software *
 [Miniconda](https://docs.anaconda.com/miniconda/) - a lightweight
 distribution of the Anaconda Package manager that can be used to create
 the environment needed to run all scripts and Jupyter Notebooks. All data
 can be accessed and viewed using an environment with Python &gt;= 3.7.0.
 The following Python packages were used to create and interact with the
 data: * ipywidgets * jupyterlab * matplotlib * numpy * pandas * plotly *
 python-opencv * seaborn ## Supplementary File #1 benchmarking.zip  -
 collected with the standard 5-monitor + running wheel setup *
 environment.yml - used to create the required conda environment *
 tdml_pickler.py - script used to convert behavior data from TDML to Pickle
 format * VR * screen delay * benchmarking_data.txt - Text file describing
 how long it took for the visual scene to update in response to position
 updates. First row contains the columns (test#, frame_rate, motion_start,
 display_start, delay_start, motion_stop, display_stop, delay_stop).
 Subsequenct columns contain tab-separated values.  *
 "frame_rate" -  frames per second of high-speed camera *
 "motion_start" - frame when the wheel was spun *
 "display_start" - frame when the visual display was updated by
 vrmate * "delay_start" - motion_start - display_start *
 "motion_stop" - frame when the wheel was stopped *
 "display_stop" - frame when the visual display stopped updating
 * "delay_stop" - motion_stop - display_stop * benchmarking.ipynb
 - notebook used to convert the above metrics into screen delay bechmarking
 figure * screen_delay_benchmarking.json - behaviorMate settings file used
 for screen delay benchmarking. This settings file configures 5 virtual
 reality displays and position tracking. * position tracking * test_1 *
 benchmarking.avi - movie used to extract ground-truth position *
 behavior.pkl - position collected using behaviorMate * behavior.tdml - raw
 output of behaviorMate (converted to behavior.pkl) *
 benchmarking_clipped.mp4 - clipped version of benchmarking.avi *
 benchmarking_filtered.mp4 - contrast-enhanced version of benchmarking.avi
 * benchmark_positions.pkl - extracted ground-truth positions for test_1 *
 benchmark_times.pkl - extracted ground-truth times for test_1 *
 position_tracking.ipynb - notebook for using ground truth times and
 positions to create VR benchmarking figures *
 vr_position_benchmarking.json - behaviorMate settings file used for VR
 benchmarking. This settings file configures position tracking but no
 virtual reality displays. The difference between this file
 and treadmill_position_benchmarking.json is that it is configured for a
 running wheel instead of a belt treadmill. * Treadmill * test_1 *
 benchmarking.avi - movie used to extract ground-truth position *
 behavior.pkl - position collected using behaviorMate * behavior.tdml - raw
 output of behaviorMate (converted to behavior.pkl) * test_2 *
 benchmarking.avi - movie used to extract ground-truth position *
 behavior.pkl - position collected using behaviorMate * behavior.tdml - raw
 output of behaviorMate (converted to behavior.pkl) *
 benchmark_positions_1.pkl - extracted ground-truth positions for test_1  *
 benchmark_positions_2.pkl - extracted ground-truth positions for test_2  *
 benchmark_times_1.pkl - extracted ground-truth times for test_1  *
 benchmark_times_2.pkl - extracted ground-truth times for test_1  *
 position_tracking.ipynb - notebook for using ground truth times and
 positions to create treadmill benchmarking figures *
 treadmill_position_benchmarking.json - behaviorMate settings file used for
 treadmill benchmarking. This settings file configures position tracking
 but no virtual reality displays. Refer to the publication for more
 information on how to tailor settings files for your experiment. This
 configuration is only provided for reproducibility. All IP addresses
 included are private and non-routable. ## Supplementary File #2
 experimental_validation.zip - collected with BOTH the standard 5-monitor +
 running wheel setup AND the treadmill setup * jy033 - mouse #1 * 2p *
 Familiar_Treadmill * dataset.pkl - raw and motion-corrected imaging data *
 rois.pkl - roi masks after segmentation * sequences.pkl - extracted
 fluorescence traces * Novel_Treadmill * Same as
 "Familiar_Treadmill" * Familiar_and_Novel_VR * Same as
 "Familiar_Treadmill" * behavior * Familiar_Treadmill.pkl *
 Novel_Treadmill.pkl * Familiar_and_Novel_VR.pkl * jy050 - mouse #2 * Same
 as "jy033" * jy065 - mouse #3 * Same as "jy033" *
 jy083 - mouse #4 * Same as "jy033" * jy098 - mouse #5 * Same as
 "jy033" * jy099 - mouse #6 * Same as "jy033" ##
 Sharing/Access Information Links to other publicly accessible locations of
 the data: * benchmarking.zip
 - [https://s3.us-east-1.amazonaws.com/losonczylab.behaviormate/Data/benchmarking.zip](https://s3.us-east-1.amazonaws.com/losonczylab.behaviormate/Data/benchmarking.zip) - experimental_validation.zip - [https://s3.us-east-1.amazonaws.com/losonczylab.behaviormate/Data/experimental_validation.zip](https://s3.us-east-1.amazonaws.com/losonczylab.behaviormate/Data/experimental_validation.zip)