DOI: 10.5061/dryad.f7m0cfz80
title: Grid cells accurately track movement during path integration-based navigation despite switching reference frames

description: Grid cells, with their periodic firing fields, are fundamental units in
 neural networks that perform path integration. It is widely assumed that
 grid cells encode movement in a single, global reference frame. By
 recording grid cell activity in mice performing a self-motion-based
 navigation task, we discovered that grid cells did not have a stable grid
 pattern during the task. Instead, grid cells track the animal movement in
 multiple reference frames within single trials. Specifically, grid cells
 re-anchor to a task-relevant object through a translation of the grid
 pattern. Additionally, the internal representation of movement direction
 in grid cells drifted during self-motion navigation, and this drift
 predicted the mouse’s homing direction. Our findings reveal that grid
 cells do not operate as a global positioning system but rather estimate
 position within multiple local reference frames.
description: # Dataset Description This dataset contains raw electrophysiological and
 behavioral data from 3 experimental cohorts comprising 17 mice. ## File
 Organization The experimental data are organized in subdirectories
 corresponding to individual recording sessions. File names typically
 include the recording session identifier (e.g., `jp486-18032023-0108`) as
 the basename. The example session `jp486-18032023-0108` represents an
 electrophysiological recording session with associated behavioral data. ##
 Session-Level Files **{session}.desel** * Specifies the brain region
 targeted by each probe * Each line corresponds to one shank
 (Neuronexus/Cambridge probes were used) **{session}.desen** * Contains
 environment codes for each recording trial * Random foraging trials:
 "circ80" * Rest periods: "rest" * AutoPI task trials:
 "autopi" **{session}.environmentFamiliarity** * Indicates
 whether the environment was novel or familiar * All environments in this
 dataset are familiar, denoted as "fam" **{session}.par** *
 Contains parameters describing the recording session * Structure: ```
 number_channels_in_dat_file number_bits_per_data_point_in_dat_file
 sampling_rate unused_number number_shanks unused_number
 number_channels_shank_1 channel_0_e1 channel_1_e1 channel_2_e1
 channel_3_e1 number_channels_shank_2 channel_0_e2 channel_1_e2
 channel_2_e2 channel_3_e2 ... number_of_session_trials
 name_of_session_trial_01 name_of_session_trial_02 ... ``` * Sampling rate
 format: 1000000/frequency (value of 50 = 20000 Hz sampling rate) * Time
 conversion: sample_no/sampling_rate = seconds **{session}.protocol** *
 Protocol name (autopi_task.py_ml) * Duration in seconds (7200) * Rosmaster
 computer identifier and IP address **{session}.px_per_cm** *
 Pixel-to-centimeter conversion factor for position data
 **{session}.sampling_rate_dat** * Sampling rate in Hz for
 electrophysiological data **{session}.setup** * Recording room identifier
 where experiment was conducted **{session}.stimulation** * Laser
 stimulation status for each trial * All trials show "none" (no
 stimulation) in this dataset **{session}.log** * Text file documenting
 event names, parameters, and timestamps during task * Format: `event_name
 time_in_seconds parameter_value` * Example: `door 1679144594.012599534
 1.000000` **arenaCoordinates** and **bridgeCoordinates** * Coordinates of
 the arena and bridge during the AutoPI task extracted via object detector
 using AutopiPy. **{session}.mouseLeverPosition.csv** * Position data for
 mouse and lever generated via DeepLabCut * Columns: * `leverX`, `leverY`:
 Lever position coordinates * `leverOri`: Lever orientation per trial *
 `leverXHeading`, `leverYHeading`: Angles between lever and bridge *
 `leverPressX`, `leverPressY`: Lever position at press time *
 `leverBoxPLX`, `leverBoxPLY`: Lever box posterior left corner coordinates
 * `leverBoxPRX`, `leverBoxPRY`: Lever box posterior right corner
 coordinates * `mouseX`, `mouseY`: Mouse position coordinates * `mouseOri`:
 Mouse orientation * `mouseXHeading`, `mouseYHeading`: Mouse heading values
 (mean between ear positions) **leverPose** * Lever position data for each
 trial generated via DeepLabCut **leverBoxDistance.npy** * Mouse occupancy
 histogram data with distance from lever on x-axis **leverZoneMaxDistance**
 * Maximum distance from lever to be considered within the lever zone
 **{session}.pose.npy** * 2D numpy array containing animal position and
 orientation information * Columns: (time, x, y, z, yaw, pitch, roll) *
 Position in centimeters, orientation in Euler angles (radians) * Missing
 data marked as `np.NAN` * Yaw ranges from -180 to 180 degrees; 0 aligns
 with positive x-axis, π/2 upward, -π/2 downward **{session}.pose_kf.npy**
 * Kalman-filtered position and orientation data * Filter applied to mouse
 x,y positions to reduce tracking noise * 2D array implementing constant
 velocity model for state [x, y, dx, dy]ᵀ **navPathInstan.csv** *
 Instantaneous behavioral variables for each navPath timepoint * Columns: *
 `name`: Trial identifier (session_name_trial_number_journey_number) *
 `trialNo`: Trial number * `timeRos`: Computer clock time * `timeRes`:
 Recording time * `iTime`: Time from trial start * `iTimeProp`:
 Proportional trial time (0-1) * `distance`: Distance traveled (cm) *
 `distanceProp`: Proportional distance within trial (0-1; NaN indicates
 insufficient tracking accuracy) * `speed`: Animal velocity * `x`, `y`:
 Animal position * `targetDistance`: Distance to lever * `targetToAnimalX`,
 `targetToAnimalY`: Lever position relative to animal *
 `targetToAnimalAngle`: Angle between animal and lever *
 `cumSumDiffAngleAroundTarget`: Cumulative angular change around target *
 Note: NaN values indicate insufficient tracking accuracy for parameter
 calculation **navPathSummary.csv** * Comprehensive list of all NavPaths in
 the session * NavPaths represent behavioral segments * Columns: * `name`:
 Trial identifier including session, journey, and path type * `trial`:
 Session plus trial name * `trialNo`: Trial number * `journey`: Journey
 number * `type`: Path classification (all, search, homing, at lever) *
 `light`: Lighting condition (light/dark) * `nLeverPresses`: Lever presses
 per journey * `startTimeRos`, `endTimeRos`: Journey start/end in ROS time
 * `startTimeRes`, `endTimeRes`: Journey start/end in recording session
 time **trial_table_simple** * DataFrame containing session name, trial
 number, start/end times in samples (`startTime`, `endTime`),
 within-session times (`startTimeWS`, `endTimeWS` in ROS time), light
 conditions, and recording start/end times **sessionIntervals.npy** * Numpy
 array containing time offsets for each session trial, generated from the
 dat file **ses.pickle** * SpikeA session object generated with AutopiPy *
 Enables NavPath selection (search, homing, at lever) for analysis * The
 "name" column links to instantaneous data for
 electrophysiological analysis * Contains time intervals for constraining
 spike train analysis to specific behavioral epochs **ifr_autopi.pkl** *
 Instantaneous firing rates for all neurons as a function of time within
 the session **{session}.shank_neuron.csv** * Contains neuron ID, neuron
 number, and recording layer. This file may not be included in every
 session. **shank_layer.csv** * DataFrame mapping shank numbers to
 recording layer. This file may not be included in every session.
 **{session}.grid_cell_parameters.pkl** * Grid cell spatial parameters
 including period, offset, peak rate, and orientation for both rigid and
 flexible models, along with model fit metrics. This file may not be
 included in every session. This file is only generated for sessions with
 sufficient numbers of co-recorded grid cells suitable for model
 reconstruction. **{session}.rnn_ifr_normalization.pkl** * Normalization
 parameters (means and standard deviations) for instantaneous firing rate
 data used in RNN analysis.  This file may not be included in every
 session. This file is only generated for sessions with sufficient numbers
 of co-recorded grid cells suitable for model reconstruction. ### Spike
 Sorting Files (Kilosort and Phy) The following files are generated using
 Kilosort and Phy. For detailed information, visit:
 [https://phy.readthedocs.io/en/latest/terminology/](https://phy.readthedocs.io/en/latest/terminology/) * `params.py`: Python script containing spike sorting and data processing configuration parameters * `amplitudes.npy`: Amplitude values for each spike, saved by the spike sorting algorithm * `channel_map.npy`: 1D integer array of size (n_channels,) containing channel mapping. Raw data has n_channels_dat columns (may differ from n_channels if dead channels exist) * `channel_positions.npy`: Channel positions on the probe shank * `pc_features.npy`: Principal component features utilized in spike sorting * `pc_feature_ind.npy`: Principal component indices * `spike_templates.npy`: Template assignments for each spike prior to manual clustering * `spike_times.npy`: Spike detection timestamps * `spike_clusters.npy`: 1D array with n_spikes elements, providing cluster ID for each spike * `cluster_group.tsv`: Cluster information where each row represents a putative neuron or neuron group, including properties and classification * `cluster_info.tsv`: Detailed cluster information from spike sorting * `cluster_KSLabel.tsv`: Cluster label information * `cluster_ContamPct.tsv`: Principal component contamination percentage per cluster (cluster_id\tContamPct) * `cluster_Amplitude.tsv`: Cluster ID and amplitude values (cluster_id\tAmplitude) * `templates.npy`: Waveform templates for specific channels **Note:** Some subjects have identifiers beginning with "jp" instead of "mn". File structures remain consistent across all animals. ## Data Directory **protocol_m_rerender.png** * Graphical abstract of the experiment protocol. ## Results Directory **cells.csv** * DataFrame containing neuron classification information * Columns: * `cluId`: Unique cell identifier * `session`: Recording session name * `mouse`: Animal identifier * `refractory`: Refractory period metric * `refractoryClean`: Boolean indicating clean refractory period * `usable`: Whether cell is usable for analysis * `meanRate`: Mean firing rate * \[Additional columns as per original data] **clu_table.csv** * Probe mapping information * Columns: * `channel`: Recording channel number * `shank`: Shank number * `hemisphere`: Brain hemisphere (left/right) * `iso_distance`: Isolation distance * `session`: Recording session * `cluId`: Cell identifier **behavior_180_EastReferenceQuadrant.csv** * Behavioral metrics with East as reference (0°) * Contains parameters including search length, duration, speed, heading errors, and homing accuracy * Columns: * `sessionName`: Session identifier * `name`: Trial name * `valid`: Whether trial is valid * `trialNo`: Trial number * `startTime`, `endTime`: Trial start and end times * `startTimeWS`, `endTimeWS`: Timestamps * `duration`: Trial duration * `light`: Light condition * `arenaRadiusCm`: Arena radius in centimeters * `nLeverPresses`: Number of lever presses * `nJourneys`: Number of journeys * \[Various search and homing behavioral metrics] * `leverX`, `leverY`: Lever position coordinates * `subject`: Animal identifier * `date`: Session date **instan_behavior_180_EastReference.csv** * Instantaneous behavioral variables at each timepoint * Columns: * `name`: Trial identifier * `trialNo`: Trial number * `timeRos`, `timeRes`: Timestamps * `iTime`: Instantaneous time * `iTimeProp`: Time proportion * `distance`: Distance measure * `distanceProp`: Distance proportion * `speed`: Instantaneous speed * `x`, `y`: Position coordinates **trialsTable** * Comprehensive trial information including timing, light conditions, search/homing metrics, and lever positions **allSessionDf_with_leverVector_and_last_cohort.csv** * Dataset with lever vector calculations and final cohort data with model predictions in toroidal coordinates from RNN model * Columns: * `index`: Index * `mouse`: Animal identifier * `session`: Session name * `condition`: Experimental condition * `trial`: Trial identifier * `trialNo`: Trial number * `xPose`, `yPose`: Animal position coordinates * `hdPose`: Head direction pose * `x`, `y`: Processed position coordinates * `px`, `py`: Predicted x, predicted y * `withinPathTime`: Time within path * `recTime`: Recording time * `speed`: Animal speed * `distance`: Distance traveled * `v0_cos`, `v0_sin`, `v1_cos`, `v1_sin`: Predicted toroidal coordinates of the animal position * `lv0_cos`, `lv0_sin`, `lv1_cos`, `lv1_sin`: Animal position in toroidal coordinates * `mvtDirError`: Movement direction error * `absMvtDirError`: Absolute movement direction error * `shortCondition`: Abbreviated condition name * `light`: Light condition (light/dark) * `path`: Path type **myMaps.pickle** * Dictionary containing 2D firing rate maps for all cells across 34 different conditions (e.g., 'circ80_1', 'searchToLeverPath_light', etc.) **myMapStats.csv** * Statistical measures for spatial maps * Columns: * `peakRate`: Peak firing rate * `gridScore`: Grid score metric * `gridSpacing`: Grid spacing (when applicable) * `meanRate`: Mean firing rate * `info`: Information content * `sparsity`: Sparsity measure * `condition`: Recording condition * `session`: Session name * `mouse`: Animal identifier * `cellId`: Cell identifier **mySpatialAutos.pickle** * Spatial autocorrelations for all conditions (34 keys matching myMaps.pickle) **mySpatialAutosLever.pickle** * Spatial autocorrelations for lever-centered conditions **mapCorDf.csv** * Correlation coefficients between firing rate maps across conditions * Contains r-values for map comparisons (NaN indicates insufficient data) * Columns: * `condition`: Experimental condition pair * `r`: Pearson correlation coefficient (NaN means no valid correlation) * `session`: Session name * `mouse`: Animal identifier * `cellId`: Cell identifier **mapCorDf_circ80_inter.csv** * Map correlations for inter-session * Same column structure as mapCorDf.csv **assignment_gridScore.csv** * Grid scores for all cells across different trial conditions and shuffling iterations * Columns: * `session`: Session name * `cluId`: Cluster/cell identifier * `gridScore`: Computed grid score value * `trialCode`: Trial condition code * `shuffling`: Whether data was shuffled (True/False) **assignment_HDScore.csv** * Head direction scores including peak rates for directional tuning analysis * Columns: * `session`: Session name * `cluId`: Cluster/cell identifier * `HDScore`: Head direction score * `peakRate`: Peak firing rate * `trialCode`: Trial condition code * `shuffling`: Whether data was shuffled (True/False) **gridRotationDf.csv** * Grid rotation analysis comparing reference frames * Includes session, condition, light state, rotation angle, and light name **occ_matched_all_scenario.csv** * Occupancy-matched grid scores across different matching scenarios * Columns: * `Unnamed: 0`: Row index * `cellId`: Cell identifier * `condition`: Experimental condition * `grid_score`: Grid score * `mouse`: Animal identifier * `match`: Matching criterion **MatcheGridScoreWithOccHdDownsampling.csv** * Grid scores after occupancy and head direction downsampling procedures **myLeverMaps.pickle** * 2D firing rate maps for lever-anchored cells in lever-centered coordinates **myLeverMapsArena.pickle** * Lever-anchored cell maps in arena coordinates **myLeverMapsStats.pickle** * Statistical measures for lever-centered maps including angular information **myLeverHistos.pickle** * 1D firing rate histograms as function of distance from lever for light/dark conditions **leverMapCor.pickle** * Correlation matrix between lever-centered maps across conditions and rotations **myLeverMapsHDInfo_new_function.pickle** * Head direction information for lever maps using updated analysis function **myLeverMapsHDInfoShuffle100.pickle** * Shuffled head direction statistics (100 iterations) for lever maps **myLeverMaps_last_lever_sessions_80.pickle** * Lever maps from last day of recording where the lever is placed on the arena for a random foraging session (but no task is involved) **searchIFRDict.pickle** * Instantaneous firing rates during search paths for all neurons **homingIFRDict.pickle** * Instantaneous firing rates during homing paths for all neurons **leverIFRDict.pickle** * Instantaneous firing rates relative to lever position **gc_pairs.csv** * Grid cell pairs for correlation analysis * Contains cell IDs, indices, session, and mouse information * Columns: * `cluId1`, `cluId2`: Identifiers for paired cells * `index1`, `index2`: Indices for paired cells * `session`: Session name * `mouse`: Animal identifier **res_ifr.csv** * Instantaneous firing rate associations between cell pairs across multiple conditions **gc_crosscorrs.pickle** * Cross-correlations between grid cell pairs **gc_crosscorrs_10_0.5_shuffle200.pickle** * Shuffled cross-correlations **gc_stcfm_10_0.5.pickle** * Spike-triggered cross-correlation matrices **gc_pairs_stcfm.pickle** * Pair-wise spike-triggered correlations across different conditions **atlever_gc_stcfm_10_0.5.pickle** * Spike-triggered correlations at the lever **atlever_gc_pairs_stcfm.pickle** * Pair-wise spike-triggered correlations at the lever **atlever_gc_stcfm_rotations_internal.pickle** * Internal rotational analysis of spike-triggered correlations at the lever **atlever_gc_stcfm_rotations_internal_stats.pickle** * Stats for internal rotational analysis of spike-triggered correlations at the lever **atlever_gc_stcfm_rotations.pickle** * Spike-triggered correlations at the lever with rotational analysis **atlever_gc_stcfm_rotations_stats.pickle** * Stats for spike-triggered correlations at the lever with rotational analysis **gc_stcfm_rotations.pickle** * Spike-triggered cross-correlation matrices with rotational analysis **gc_stcfm_rotations_stats.pickle** * Stats for spike-triggered correlations with rotational analysis **dfBridgeVonMises.csv** * Bi-variate Von Mises distribution parameters for bridge anchoring strength * Columns: * `condition`: Experimental condition * `session`: Session name * `trialBin`: Trial bin number * `level_3`: Hierarchical level * `k0`, `k1`: Concentration parameters * `k`: Combined concentration parameter * `mu0`, `mu1`: Mean direction parameters **dfLeverVonMises.csv** * Bi-variate Von Mises distribution parameters for lever anchoring strength (k0, k1, k, mu0, mu1) * Same column structure as dfBridgeVonMises.csv **dfLeverAnchoring_leverlocked.csv** * Anchoring strength analysis by time interval * Columns: * `Unnamed: 0`: Row index * `session`: Session name * `interval`: Time interval * `level_2`: Hierarchical level * Von Mises parameters (k0, k1, k, mu0, mu1) **reconstuctionDFAutoPI.csv** * Path reconstruction data for AutoPI task including movement vectors and predictions * Contains similar structure as allSessionDf **reconstuctionDFOF.csv** * Path reconstruction for open field sessions * Contains similar structure as allSessionDf **reconstruction_shuffling.csv** * Shuffled reconstruction analysis * Columns: * `Unnamed: 0`: Row index * `mvtDirError`: Movement direction error * `session`: Session name * `condition`: Experimental condition * `shuffleIter`: Shuffle iteration number * `mvl`: Mean vector length **sessionList_pathReconstruction.csv** * Sessions used for path reconstruction by grid cell count * Columns: * `session`: Session name * `gridCells`: Number of grid cells in the session **model_distance_with_bins.csv** * Distance predictions with spatial binning for model validation * Columns: * `predictedLeverCoordv0`, `predictedLeverCoordv1`: Predicted lever coordinates * `actualCoordv0`, `actualCoordv1`: Actual coordinates * `distance_pred_to_actual`: Distance between predicted and actual positions * `bin`: Distance bin assignment **model_mvl_shuffled.csv** * Mean vector length analysis with shuffled controls **myControlMaps.pickle** * Control condition firing rate maps **myControlMapStats.csv** * Statistics for control condition maps * Columns: * `gridScore`: Grid score * `peakRate`: Peak firing rate * `info`: Information content * `sparsity`: Sparsity measure * `condition`: Experimental condition * `session`: Session name * `mouse`: Animal identifier * `cellId`: Cell identifier **myControlSpatialAutos.pickle** * Spatial autocorrelations for control conditions **rf_lever_control_light.csv** * Reference frame control analysis in light conditions * Columns: * `cellId`: Cell identifier * `mvl`: Mean vector length * `session`: Session name * `mouse`: Animal identifier **rf_lever_control_dark.csv** * Reference frame control analysis in dark conditions * Same structure as rf_lever_control_light.csv **myDoughnutCorrelations.pickle** * Doughnut correlation analysis (rotate the firing map of grid cells and correlate it with itself) **myLeverCellStatsSplitByMedian_homingDir_all.pickle** * Lever cell statistics split by median homing direction **myLeverCellStatsSplitBySearchLengthMedian_faster.pickle** * Statistics split by search length median **myLeverCellStatsTrialMatrix_all.pickle** * Trial matrix statistics for lever cells **rotation_per_trial_after_correction.csv** * Trial-by-trial rotation values after correction procedures * Columns: * `Unnamed: 0`: Row index * `session`: Session name * `condition`: Experimental condition * `light`: Light condition * `trialNo`: Trial number * `rotation`: Rotation angle * `abs_rotation`: Absolute rotation angle **rr_anchoring_strength_analysis_by_trial.csv** * Anchoring strength (k-values) across trial bins for different conditions * Columns: * `condition`: Experimental condition * `session`: Session name * `trialBin`: Trial bin * Von Mises parameters (k0, k1, k, mu0, mu1) * `shortCondition`: Abbreviated condition * `light`: Light condition * `trialBinName`: Trial bin name * `anchor`: Anchor type **shuffle_rf3_lever_dataframe.csv** * Shuffled map similarity for random foraging with lever on the arena * Columns: * `Unnamed: 0`: Row index * `cellName`: Cell identifier * `correlation`: Correlation value **shuffled_lever_grid_coordinates_anchoring_strength_analysis.pickle** * Shuffled model prediction coordinates for anchoring strength analysis **within_trial_orientation_subtraction.csv** * Within-trial orientation differences for trial-by-trial rotation analysis * Columns: * `session`: Session name * `trialNo`: Trial number * `difference`: Orientation difference value * `cartesian_maps_cropped.png`: Stack of grid cell firing rate maps in Cartesian coordinates * `toroidal_maps_cropped.png`: Stack of grid cell firing rate maps in toroidal coordinates * `heading_error.png`: Graphical abstract of heading error * `error_at_peri.png`: Graphical abstract of error at periphery * `leverPred2.png`: Explanation figure of how anchoring strength is derived and how predicted lever coordinates are calculated * `anchoring_exclude_jp486_top.png`: Same figure as Figure 6, excluding the animal with the most grid cells recorded * `anchoring_exclude_jp486_bottom.png`: Same figure as Figure 7-8, excluding the animal with the most grid cells recorded * `schematic_cartesian_to_toroidal_3.png`: Schematic illustrating Cartesian to toroidal transformation * `jp451_lstm_firstRF.npy`: Test session used for simulations. Contains first random foraging animal position data * `jp451_lstm_lastRF.npy`: Test session used for simulations. Contains last random foraging animal position data ### Analysis Directories * `grid_spacing_and_dispersion/`: Contains grid spacing and dispersion simulation analyses * `hd_grid_simulation/`: Head direction and grid cell simulation analyses * `simReanchor/`: Simulation results for reference frame re-anchoring * `simReanchorRotation/`: Re-anchoring simulations with different scenarios ## Units of Measure and Data Conventions **Units:** * Distance measurements: centimeters (cm) * Time/duration measurements: seconds (s) * Position coordinates: centimeters (cm) * Angles: radians (unless otherwise specified) * Firing rates: Hz (spikes per second) **Empty Cells:** Empty cells or NaN values in CSV/TSV files indicate missing data due to insufficient tracking accuracy, failed computations, or data not applicable to specific conditions. This applies consistently across all data files in the dataset. ## Data Access Instructions The dataset is split into 20 files due to its large size. To combine and extract the data: ### Linux/Unix Terminal Commands: ```bash # Combine files cat folder.tar.gz.part.* &gt; folder.tar.gz # Extract archive tar -xzf folder.tar.gz -C /path/to/destination_directory ``` ## Code Availability Analysis code is available in the following GitHub repositories: * [https://github.com/PJJ19/Peng_et.al_2025_noInt](https://github.com/PJJ19/Peng_et.al_2025_noInt) * [https://github.com/PJJ19/autopi_grid_cell_decoding](https://github.com/PJJ19/autopi_grid_cell_decoding) ## File Format Specifications * `.npy`: NumPy binary format * `.pickle` / `.pkl`: Python pickle format * `.csv`: Comma-separated values * `.tsv`: Tab-separated values * `.png`: Portable Network Graphics images