DOI: 10.5061/dryad.vt4b8gv2g
title: Hummingbirds use compensatory eye movements to stabilize both rotational and translational visual motion

description: To maintain stable vision, behaving animals make compensatory eye
 movements in response to image slip, a reflex known as the optokinetic
 response (OKR). Although OKR has been studied in several avian species,
 eye movements during flight are expected to be minimal. This is because
 vertebrates with laterally placed eyes typically show weak OKR to
 nasal-to-temporal motion (NT), which simulates typical forward locomotion,
 compared to temporal-to-nasal motion (TN), which simulates atypical
 backward locomotion. This OKR asymmetry is also reflected in the
 pretectum, wherein neurons sensitive to global visual motion also exhibit
 a TN bias. Hummingbirds, however, stabilize visual motion in all
 directions through whole-body movements and are unique among vertebrates
 in that they lack a pretectal bias. We therefore predicted that OKR in
 hummingbirds would be symmetrical. We measured OKR in restrained
 hummingbirds by presenting gratings drifting across a range of speeds. OKR
 in hummingbirds was asymmetrical, though the direction of asymmetry varied
 with stimulus speed. Hummingbirds moved their eyes largely independent of
 one another. Consistent with weak eye-to-eye coupling, hummingbirds also
 exhibited disjunctive OKR to visual motion simulating forward and backward
 translation. This unexpected oculomotor behavior, previously unexplored in
 birds, suggests a potential role for compensatory eye movements during
 flight.
description: <em>Animals</em> Six adult male
 Anna’s hummingbirds (<em>Calypte anna</em>) were caught on the
 University of British Columbia campus (07 Feb 2022 – 04 March 2023).
 Animals were individually housed in 0.61 x 0.61 x 0.91-m cages and
 provided with<em> ad libitum</em> artificial nectar
 (Nektar-Plus, Nekton, Pforzheim, Germany & Nectar 9, Roudybush,
 Woodland, California, USA). All procedures were approved by the University
 of British Columbia Animal Care Committee in accordance with the
 guidelines set out by the Canadian Council on Animal Care. Animals were
 released back to the wild following the experiments described
 herein. <em>Method of
 restraint</em> To allow the release of the birds
 after our experiments, we developed a method for non-invasive restraint of
 hummingbirds. Head-restraint is required to study OKR in the absence of
 the vestibulo-ocular reflex. We restrained hummingbirds in a flight
 posture by wrapping them in an elastic “ACE Brand” bandage (3M, Maplewood,
 USA) and placing them on a plastic bed tilted 10° above horizontal (see
 Fig. 1A). To restrain the head, we fashioned a bill guide using the
 opening of a 30 mL Luer syringe (Henke Sass Wolf, Dudley, USA). The bill
 guide was oriented to hold the bill horizontally, as is typical in flight
 (Fig. 1B). Birds were hand-fed a sucrose solution
 before each recording session. We then wrapped the bird in the elastic
 bandage, gently positioned the bill through the guide, and placed the
 animal on the plastic bed. We were careful to place the body close enough
 to the bill guide so that the animals were unable to retract their head.
 Extension of the head was prevented as the opening of the bill guide was
 smaller than the widest part of the bill. After each recording session,
 animals were returned to their home cage. <em>Eye
 tracking </em> An overview of the eye-tracking
 setup is provided in Figure 1. We used a bright pupil method to detect and
 track both eyes of each hummingbird [38]. Two cameras were positioned
 using tripods on the left- and right-side of the bird, at the height of
 the animal and along the optical axis of the eyes (~20 deg nasal from
 lateral). Bright pupil eye tracking uses an infrared light to increase the
 contrast between the pupil and the iris (analogous to the “red eye
 effect”). For illumination, we used 850 nm IR ring lights (Feyond
 FY76-72V4-F, Shenzhen City, China). These IR lights were attached to 50 mm
 Nikkor camera lenses (Kabushiki-gaisha Nikon, Tokyo, Japan). Videos were
 acquired using IR-sensitive cameras (Prosilicia GE680, Allied Vision,
 Stadtroda, Germany) operating at 120 Hz with recording performed using
 Streampix version 6 (NorPix Inc., Montreal, Canada). Recording occurred in
 the dark. The IR lights produce illumination outside of the visual range
 for hummingbirds [39]. <em>Visual
 Stimuli</em> Stimuli were generated using
 PsychoPy (version 2023.1.3) and presented on two 144Hz LCD monitors (ASUS
 VG248QG, ASUSTeK Computer Inc., Taipai, Taiwan) set to sRGB color mode,
 resulting in gratings with a Michelson contrast of 0.99. The monitors were
 oriented perpendicular to the bird and positioned 30 cm away on either
 side (Fig. 1A). To allow a clear view of the bird for each camera, the
 monitors were height-adjusted so that the lower margin of the screen was
 just below that of the bird. Thus, visual stimuli were presented to the
 upper and lateral visual fields, occupying ~83 deg horizontally and ~45
 deg vertically on either side of the animal. Previous
 electrophysiology studies in hummingbirds used random dot fields to
 determine neuron direction preferences and then sinewave gratings to
 measure spatiotemporal tuning in the preferred direction [7,31,33]. We
 selected stimulus conditions to match these studies, but used sinewave
 gratings throughout. This allowed us to assess OKR at stimulus speeds
 beyond what we could produce with dots and replicate stimulus speeds
 across a consistent set of spatial and temporal frequencies (Table
 1). Within a recording session, we randomized the
 direction (NT or TN) and speed of stimuli presented on each monitor.
 Hummingbirds must feed frequently, limiting the maximum length of
 recording sessions to under 40 minutes. We therefore presented stimuli for
 each spatial frequency on separate days, except for spatial frequencies of
 0.015625 cyc deg<sup>-1 </sup>and 4 cyc
 deg<sup>-1</sup>, which were presented on the same day but
 with a feeding break in between. Stimulus velocity was defined as the
 product of temporal and spatial frequency (Table 1). During binocular
 stimulation, stimulus speeds presented to both eyes were always matched
 (e.g., 1 deg s<sup>-1</sup> TN to the right eye and 1 deg
 s<sup>-1</sup> NT to the left eye). Table
 1: See manuscript Each recording session consisted of a
 set of trials repeated three times and randomized within each set. Each
 trial began with a static pattern for 1 s, followed by a stationary
 sinewave grating for 1 s, followed by 10 s of motion of that same sinewave
 grating in either the TN or NT direction. As we did not wish to physically
 cover an eye for monocular trials and risk injury to the birds, two
 methods were used for monocular stimulation: 1) the monitor for the
 unstimulated eye was either turned gray after the static pattern, 2) or
 changed to the sinewave grating but held stationary. We analyzed data from
 the former method as the latter represents an unequal speed combination,
 though data from both methods were qualitatively similar. For binocular
 trials, the stimulus direction for each monitor was set independently to
 produce either regressive stimulation (TN & TN), progressive
 stimulation (NT & NT), or rotation (TN & NT) or (NT
 & TN). We refer to binocular NT & TN combinations as
 rotational stimulation; however, as our stimulus monitors did not curve
 around the animal, these conditions technically represent
 pseudo-rotation. <em>Analysis of oculomotor
 behavior</em> To extract the position of the
 pupil from each video, we used a combination of thresholding and contour
 segmentation in python (version 3.11.8). We trained a custom DeepLabCut
 [40] model (ResNet 152) to track the corneal reflection and eyelids. To
 track the pupil, we used data from the same DeepLabCut model to determine
 a region-of-interest around the eye, upscaled this portion of the video
 frame, and then used thresholding and contour detection in OpenCV (v.
 4.9.0) for pupil detection. Analysis of raw pupil
 position was performed in MATLAB R2023b (Mathworks, Natick, USA). We first
 removed positions if the confidence from DeepLabCut was low (&lt;
 95%). We next used the vertical distance between upper and lower eyelids
 to detect blinking, removing pupil position data 100 ms before and 400 ms
 after each detected blink to eliminate blink-artifacts. We then removed
 periods of pupil position data when the pupil area was changing rapidly,
 as asymmetrical dilation or constriction of the pupil could be erroneously
 interpreted as changes in position. Finally, we filled small gaps in the
 pupil position stream using linear interpolation to facilitate saccade
 detection. To convert the filtered pupil position from
 pixels to degrees, we set the pupil position for each trial relative to
 the median position prior to motion onset. Pupil displacement was then
 converted to millimeters based on objects of known length in the video
 frame, digitized for each recording session using ImageJ 1.53k (NIH, USA).
 We removed data for trials in which the corneal reflection moved more than
 1 mm, as this indicated that the bird rolled its head during the trial
 (6.7% of total trials). Finally, we converted displacement to degrees
 based on the assumption that the avian eye rotates around the midpoint
 between the pupil and the back of the eye [41]. We determined this value
 (2.181 ± 0.102 mm; mean ± sd) by measuring the half distance between the
 outer edge of the scleral ring and the medial margin of the eye for five
 eyes from four specimens available from previous lab projects. The
 eye-in-head position was then calculated as θ = tan<sup>-1
 </sup>(rel. position mm / 2.181 mm). Importantly, this assumption
 does not impact the main conclusions of this work and resulted in a
 similar range of movement as determined using alternative methods
 [42]. We next automatically removed saccades using peak
 detection, analyzing the median-filtered velocity of the pupil for spikes
 above a speed threshold. This speed threshold was manually adjusted
 (between 5-15 deg s<sup>-1</sup>) for each session based on
 visual inspection of pupil position across trials.
 Inspection of the parallel streams of data from each eye
 indicated that blinks and saccades often occurred simultaneously in both
 eyes, as in other birds [43,44]. For this reason, we considered a blink or
 saccade in either eye to constitute a break in OKR and computed the
 velocities of the eyes between each of these breaks. This allowed us to
 compare, for the same set of time windows, the motion occurring
 simultaneously in each eye. To compute OKR velocities, we fit each segment
 between breaks with a linear regression and extracted the slope (deg
 s<sup>-1</sup>), fitted values (deg), and quality of the fit
 (Root Mean Squared Error; RMSE). The magnitudes (deg) of rapid eye
 movements (blinks and saccades) were determined using these fitted values
 as the offset between the last and first elements in each adjacent
 segment. <em>Statistical
 analysis</em> We performed plotting and
 hypothesis testing in R version 4.4.0. For comparisons of OKR between
 conditions, we removed segments with poor fit quality (RMSE &gt; 0.35)
 or with many (&gt;10%) missing data points for either eye (together
 12.7% of total fits). Significance was assessed by computing bootstrapped
 means and 95% confidence intervals (1000 iterations) using the R package
 <em>boot</em> [45]. Unless otherwise specified, bootstrapped
 values were computed by pooling fitted values from both eyes and
 bootstrapping across trials and individuals.
description: This folder contains all files and code related to the paper titled
 "Hummingbirds use compensatory eye movements to stabilize both
 rotational and translational visual motion". Because of the size of
 some files (high-speed video, in particular), intermediate files are not
 included, as the total size of all files associated with this project is
 &gt;5 tb. An example raw video is included to test pupilTracker.py.
 However, all videos are uploaded as high-quality MP4 files following
 consultation with journal staff. The organization of these files is as
 follows: --- **analysisScripts** -- contains major code for collection of
 pupil data (python), processing of pupil data (MATLAB), and analysis and
 visualization (R). \-- pupilTracker.py -- INPUT: video, DLC file, OUTPUT:
 file in "rawData". DESCRIPTION: extracts the position of the
 pupil using the video and a DeepLabCut file associated with that video.
 The resulting file combines the DeepLabCut output with the position of the
 pupil, tracked as a contour using openCV. \--
 ocularAnalysis_autoProcess.mlx -- INPUT: kinematic data from
 "rawData" + stimulus file, OUTPUT: synced kinematic data in
 "processedData". DESCRIPTION: This is the major workhorse of the
 analysis. Raw position data, produced by pupilTracker.py is imported and
 processed. This includes synchronziation with the stimulus files,
 filtering, computing relative pupil position in degrees, segmenting into
 slow and rapid eye movements, and computing statistics for those segments
 of data. \-- ocularAnaylsis.Rmd -- INPUT: files from
 "processedData", OUTPUT: figures, summary results. DESCRIPTION:
 This script produces the summary stats and figures of the paper. Some
 minor touches to the panel layout were made using Adobe Illustrator. ---
 **rawData.zip** -- contains all files output by the combination of
 DeepLabCut and pupilTracker.py. Files are organized according the the day
 of recording. \-- Folders -- The naming convention for each folder is
 hummingbird_recordingData_spatialFrequency. For example, the folder
 'hummingbird_20240315_0.0625' contains all raw data recording on
 15 March 2024. On this date, the spatial frequency of the stimulus was
 0.0625 cycles per degree. Within each of these folders, you will find a
 series of .csv files. The data for recording sessions (a stimulus set
 presented to an individual bird) are contained across three files that
 share the same recording time in their file names - an L file, and R file,
 and an OKR_stimulus file. For example, 09-48-47.000L.csv,
 09-48-47.000R.csv, OKR_stimulus_09-48-47.000.csv are a set and contain the
 data recorded at 09:48:47.000. \--- L and R files --- These contain the
 position data from the left and right side of the animal, respectively.
 For these files, Columm A (dt) contains the time in seconds relative to
 the start of the first stimulus presentation. Column B contains video
 frame number, starting from 0. Columns C through AL contain the x and y
 position (pixel units) along with the confidence (0-1) output by
 DeepLabCut for each tracked feature. Columns AM through AR contain the x
 position, y position, and area of the pupil tracking using OpenCV (pixel
 units). AM-AO are the result of blob detection. AP-AR are the result of
 contour detection. The latter method was used in all analysis. \---
 OKR_stimulus files --- These describe the stimulus conditions from the
 recording session. These include the Left and Right Spatial Frequency CPD,
 Direction, Temporal Frequency, Speed PPS, dt, index, and Time. Spatial
 Frequency CPD is that of the stimulus in cycles per degree. Direction is
 0, 180, blank, or station. 0 is nasalward for the monitor on the left
 side; temporalward for the right side. 180 is temporalward for the monitor
 on the left side; nasalward for the right side. Station indicates that the
 stimulus on that side remained stationary (no motion). Blank indicates
 that the monitor switched to 50:50 gray at the start of stimulus motion.
 See the paper for details. The dt column contains the time of motion onset
 relative to the start of the first stimulus presentation. In other words,
 the dt columns in these files are used to synchronize the stimulus data
 with the kinematic data in the L and R files. The Time column and index
 column are unused, but indicate global time and the order of stimulus
 presentation prior to randomization, respectively. To see the association
 between recording times and indivdual bird identities, see
 ocularAnalysis_autoProcess.mlx. --- **processedData.zip** -- contains the
 output from ocularAnalysis_autoProcess.mlx. These files are read into R
 for processing. \-- SFComp -- This folder contains the data output by
 ocularAnalysis_autoProcess.mlx for the original set of experiments - six
 individual's response to stimuli at five spatial frequencies and a
 range of speeds. \-- T1CPD -- This folder contains the data output by
 ocularAnalysis_autoProcess.mlx for the follow-up experiment, presented in
 Figure 4E - six individual's response to stimuli at 1 cycle per
 degree and an expanded range of speeds. Files within these folders are
 named as follows: birdID_spatialFrequency_recordingTime (e.g.
 280_0.25_15-38-54.000.csv). \--- .csv files --- Each csv file contains the
 following columns: * frame - frame count of video * dt - time (seconds)
 relative to start of first stimulus presentation * periTime - time
 (seconds) relative to motion onset for that trial (-2s through 10s) *
 trialID - a unique trial identifier generated randomly for plotting
 purposes * SpatialFrequency - spatial frequency of stimulus (cycles per
 degree) * Speed - speed of stimulus motion (degrees per second) * Temporal
 Frequency - temporal frequency of stimulus (Hz) * LeftDirection -
 direction of stimulus motion on the left monitor relative to the bird. TN:
 Temporal-Nasal, NT: Nasal-Temporal. * RightDirection - direction of
 stimulus motion on the right monitor relative to the bird. TN:
 Temporal-Nasal, NT: Nasal-Temporal. * blinkBreak - binary indicating
 whether a blink was detected at that frame in either eye. * saccadeBreak -
 binary indicating whether a saccade was detected at that frame in either
 eye. * break - union of blinkBreak and saccadeBreak. * **The remaining
 columns start with either 'Left' or 'Right',
 corresponding to data for left and right eyes (e.g. Left_angle_X)**. *
 \_angle_x - horizontal position of the eye (degrees). Data are replaced
 with NaN if break is 1. * \_angle_y - UNUSED - vertical position of the
 eye (degrees). Data are replaced with NaN if break is 1. * \_angle_xRAW -
 horizontal position of the eye (degrees). * \_angle_yRAW - UNUSED -
 vertical position of the eye (degrees). * \_pupil_xRAW_mm - horizontal
 position of the eye (millimeters). * \_pupil_yRAW_mm - UNUSED - vertical
 position of the eye (millimeters). * \_blink - binary indicating whether a
 blink was detected for that frame in that eye. * \_saccade - binary
 indicating whether a saccade was detected for that frame in that eye. *
 \_purk_rel_x - horizontal position of the corneal reflection
 (millimeters). * \_purk_rel_y - vertical position of the left corneal
 reflection (millimeters). * \_sp_trial - trial (1-3) for a given
 combination of stimulus conditions. * \_sp_startTime - start time of the
 slow phase (seconds) relative to trial start (periTime). * \_sp_cycle -
 running count of the number of slow phases occurring within a given trial.
 * \_sp_duration - duration of a given slow eye movement (seconds) *
 \_sp_slope - horizontal slope of a given slow eye movement (degrees per
 second) * \_sp_mag - horizontal magnitude of a given slow eye movement
 (degrees) * \_sp_startPosition - horizontal position of the eye at the
 start of a given slow phase (degrees) * \_sp_endPosition - horizontal
 position of the eye at the end of a given slow phase (degrees) * \_sp_RMSE
 - Root Mean Squared Error (degrees) of linear regression to a given slow
 phase * \_sp_fit - fitted values (degrees) from linear regression for each
 slow phase * \_sp_filt - median filtered version of horizontal position of
 the eye (degrees) for a given slow phase * \_sp_frac - fraction of rows
 for a given slow phase that contained raw data used for fitting *
 \_sp_aveVel - average velocity of the eye during a given slow phase
 calculated from median filtered data * \_sp_qp_mag - horizontal change in
 position (degrees) of the eye between successive fits -- i.e. the
 start_position of movement n - end_position of movement n-1 This also
 contains validation_results.csv, used by the R script to create panels A
 and B of Figure 4. See below. This file is included here for easier
 access. --- **validationData** \-- contains csv files and a MATLAB script
 for created panels A and B of Figure 4. For this analysis, 10 features on
 the head and face of 2 birds were automatically tracked using DLTdv8
 ([https://biomech.web.unc.edu/dltdv/](https://biomech.web.unc.edu/dltdv/)). \-- RAW_dltdv8Files -- contains the files output by DLTdv8 when digitizing. The files are named as Direction_Bird_Speed_SpatialFrequency. Only those that end in "xypts" contain any data, as this is a 2d analysis. The data contained are the frame in the video and the x and y position of the point (pixel units). These xypts files are copied into the outer validationData folder with the xypts ending removed. \-- validationData_bootstrapping.m -- This script gathers the xypts and computes the mean and 95% confidence intervals for the position of the head using these 10 points and a bootstrapping function, after filtering to remove random jitter. The results are in millimeters. \-- all_results.csv -- Contains the output of validationData_bootstrapping.m. This file contains the following columns: * frame - frame of video * Direction - direction of stimulus motion (NT or TN) * Bird - bird identity * SpatialFrequency - spatial frequency of stimulus (cycles per degree) * Mean_X - mean horizontal position of the head and face, calculated from 10 manually digitized points (millimeters) * CI_Lower_X - Lower bound of 95% CI for horizontal head position (millimeters) * CI_Upper_X - Upper bound of 95% CI for horizontal head position (millimeters) * CI_Lower_Y - Lower bound of 95% CI for vertical head position (millimeters) * CI_Upper_Y - Upper bound of 95% CI for vertical head position (millimeters) \-- validation_results.csv -- Contains the same data as all_results.csv plus the following columns: * periTime - time relative to motion onset within a trial (seconds) * Left_pupil_xRAW_mm - horizontal position of the left eye (millimeters). * Left_pupil_yRAW_mm - vertical position of the left eye (millimeters). * Right_pupil_xRAW_mm - horizontal position of the right eye (millimeters). * Right_pupil_yRAW_mm - vertical position of the right eye (millimeters). --- **eyeCalibrationImages** -- contains images and extract data for the conversion from mm to degrees of eye-in-head angle. Measurements for each eye, made using ImageJ, are contained with the calibrationValues.txt file along with the math for calculating the approximate distance between the pupil and the eye's center of rotation. --- **testVideo** -- contains an example (30 second clip) from a head restrained animal along with the output from the DeepLabCut model. This is what is required to test pupilTracker.py. The default settings should be adjusted properly for this test video. Running just this 30 second clip takes ~6 minutes on my system. The one difference between the \_combo file output and those in rawData.zip is the addition of the timecode column on the far left. This was added manually by viewing the videos to identify the first motion onset, and then setting the dt for that row to the first value of dt in the stimulus file from psychopy. --- **videos** -- contains all videos organized by recording data and spatial frequency. These videos are high quality MP4 files. --- **stimulusCode_example.py** -- An example python script used for generated stimulus. The included example is setup for a spatial frequency of 0.25 cycles per degree. We cannot confirm that this script will work out of the box on all systems. --- **Movie_1_10.1098_rspb.2024.2015** -- Animation showing eye position changes in Anna's hummingbird (*Calypte anna*) detected using video oculography. (This is uploaded as 'Supplemental Information')