Author response: Variability in locomotor dynamics reveals the critical role of feedback in task control (2019)

Article Figures and data Abstract eLife digest Introduction Results Discussion Materials and methods Data availability References Decision letter Author response Article and author information Metrics Abstract Animals vary considerably in size, shape, and physiological features across individuals, but yet achieve remarkably similar behavioral performances. We examined how animals compensate for morphophysiological variation by measuring the system dynamics of individual knifefish (Eigenmannia virescens) in a refuge tracking task. Kinematic measurements of Eigenmannia were used to generate individualized estimates of each fish’s locomotor plant and controller, revealing substantial variability between fish. To test the impact of this variability on behavioral performance, these models were used to perform simulated ‘brain transplants’—computationally swapping controllers and plants between individuals. We found that simulated closed-loop performance was robust to mismatch between plant and controller. This suggests that animals rely on feedback rather than precisely tuned neural controllers to compensate for morphophysiological variability. eLife digest People come in different shapes and sizes, but most will perform similarly well if asked to complete a task requiring fine manual dexterity – such as holding a pen or picking up a single grape. How can different individuals, with different sized hands and muscles, produce such similar movements? One explanation is that an individual’s brain and nervous system become precisely tuned to mechanics of the body’s muscles and skeleton. An alternative explanation is that brain and nervous system use a more “robust” control policy that can compensate for differences in the body by relying on feedback from the senses to guide the movements. To distinguish between these two explanations, Uyanik et al. turned to weakly electric freshwater fish known as glass knifefish. These fish seek refuge within root systems, reed grass and among other objects in the water. They swim backwards and forwards to stay hidden despite constantly changing currents. Each fish shuttles back and forth by moving a long ribbon-like fin on the underside of its body. Uyanik et al. measured the movements of the ribbon fin under controlled conditions in the laboratory, and then used the data to create computer models of the brain and body of each fish. The models of each fish’s brain and body were quite different. To study how the brain interacts with the body, Uyanik et al. then conducted experiments reminiscent of those described in the story of Frankenstein and transplanted the brain from each computer model into the body of different model fish. These “brain swaps” had almost no effect on the model’s simulated swimming behavior. Instead, these “Frankenfish” used sensory feedback to compensate for any mismatch between their brain and body. This suggests that, for some behaviors, an animal’s brain does not need to be precisely tuned to the specific characteristics of its body. Instead, robust control of movement relies on many seemingly redundant systems that provide sensory feedback. This has implications for the field of robotics. It further suggests that when designing robots, engineers should prioritize enabling the robots to use sensory feedback to cope with unexpected events, a well-known idea in control engineering. Introduction Animals routinely exhibit dramatic variations in morphophysiology between individuals but can nevertheless achieve similar performance in sensorimotor tasks (Sponberg et al., 2015; Bullimore and Burn, 2006). Further, individual animals can experience rapid changes in their own morphophysiological features, such as extreme weight changes that occur during and between bouts of feeding. For example, mosquitoes can consume more than their body weight (Van Handel, 1965) and hummingbirds can consume up to 20% of their body weight (Hou et al., 2015) in a single feeding. How neural control systems accommodate these changes is not known. The behavioral performance of an individual animal is determined via an interplay between its ‘controller’ and ‘plant’ (Kiemel et al., 2011; van der Kooij and Peterka, 2011; Cowan et al., 2014; Dickinson et al., 2000; Hedrick et al., 2009). The plant typically includes musculoskeletal components that interact with the environment to generate movement (Hedrick and Robinson, 2010; Sefati et al., 2013; Maladen et al., 2009). The controller typically includes sensory systems and neural circuits used to process information to generate motor commands (Cowan and Fortune, 2007; Kiemel et al., 2011; Lockhart and Ting, 2007; Roth et al., 2014). From the perspective of control theory, one might expect the dynamics of the controller to be precisely tuned to the dynamics of the plant, resulting in an optimal control law that reduces variability in task performance (Todorov, 2004; Franklin and Wolpert, 2011; Bays and Wolpert, 2007). Were this the case, variations across individuals in morphophysiological features of their plants should manifest in commensurate differences in each animal’s controller. Alternatively, the central nervous system (CNS) may be implementing robust feedback control that attenuates morphophysiological variability at the behavioral level without the need for precise tuning. Investigating these relationships requires separate estimates for plants and controllers. However, the classical input–output system identification of behavioral tasks—using only the sensory input and the behavioral output—is limited to generating closed-loop control models of behavioral responses. Data-driven system identification of the underlying neural controllers or locomotor plants requires additional observations such as a measurement of the control output. Electromyograms (EMGs) are the most commonly used proxy for the output of the neural controller. EMGs allow separate data-driven estimates of the controller and plant (Kiemel et al., 2011; van der Kooij and Peterka, 2011) but require understanding the coordination strategy across multiple groups of muscles that interact in fundamentally nonlinear ways (Ting and Macpherson, 2005). We studied refuge tracking in a species of weakly electric fish Eigenmannia virescens (Figure 1A), a system that permits identification of input–output dynamics as well as the locomotor plant via behavioral observations alone. Like an ‘aquatic hummingbird’, Eigenmannia precisely hover in place, making rapid and nuanced adjustments to its position in response to the movement of the refuge in which it is hiding (Rose and Canfield, 1993; Roth et al., 2011; Uyanik et al., 2019b; Figure 1—video 1). During refuge tracking, individual Eigenmannia generate fore-aft thrust forces using undulatory motions of their ventral ribbon fin. Undulations are initiated at the rostral and caudal ends of the fin resulting in counter propagating waves that travel towards each other (Sefati et al., 2013; Ruiz-Torres et al., 2013). The two traveling waves meet at a position along the ribbon fin known as the nodal point (Figure 1—video 2). In a task in which the fish maintains position in a stationary refuge, Eigenmannia shift the rostrocaudal position of the nodal point as a function of steady-state swimming speed (Sefati et al., 2013; Figure 1—figure supplement 1), providing a behavioral proxy for the controller’s output, without reliance on EMGs. Figure 1 with 4 supplements see all Download asset Open asset Experimental analysis of refuge tracking behavior. (A) Eigenmannia virescens maintains its position, y⁢(t), in a moving refuge. r⁢(t) is the position of the moving refuge. (B) A feedback control system model for refuge tracking. The feedback controller C⁢(s) maps the error signal, e⁢(t), to a control signal, u⁢(t). The plant P⁢(s), in turn, generates fore-aft movements y⁢(t). We used measurements of the nodal point as u⁢(t) to estimate P⁢(s) in each fish, which was subsequently used to infer C⁢(s). (C) Eigenmannia virescens partitions its ribbon fin into two counter-propagating waves that generate opposing forces (Sefati et al., 2013; Ruiz-Torres et al., 2013). The rostral-to-caudal and caudal-to-rostral waves collide at the nodal point (green line). Movements of the nodal point serve as a proxy for the control signal, u⁢(t). Gray lines indicate the positions of the peaks and troughs of the traveling waves over time. In this trial, the refuge was following a sinusoidal trajectory at 2.05 Hz. (D) Movement of the nodal point u⁢(t), position of the fish y⁢(t), and position of the refuge r⁢(t) of 13 trials at 0.95 Hz. Semi-transparent lines represent data from individual trials, green and blue lines are the means. We measured tracking performance and moment-to-moment position of the nodal point in three fish during a refuge tracking task. Despite the fact that tracking performance of the three fish were similar, there were nevertheless large variations in the movement of the nodal point, reflecting morphophysiological differences between individuals. We used computational techniques, specifically data-driven system identification of feedback control models, to explore how neural control systems cope with individual variability in locomotor dynamics. Results We measured the performance of three fish in a refuge tracking task by comparing the position of the refuge r⁢(t) with the position of the fish y⁢(t) (Figure 1A). These measurements are used in a feedback control model in which ‘error signal’ e⁢(t) is defined as the difference between r⁢(t) and y⁢(t) (Figure 1B). To separately estimate the controller C⁢(s) and the plant P⁢(s), a measurement of the control signal u⁢(t) is necessary. In Eigenmannia, the position of the nodal point can be measured over time (Figure 1C, Figure 1—video 2). During refuge tracking behavior, we observed that the position of the nodal point appears to have a linear relationship to y⁢(t) making it a candidate readout of u⁢(t) (Figure 1D). Using u⁢(t) and y⁢(t), we estimated P⁢(s) for each fish and calculated their corresponding controllers C⁢(s). We used C⁢(s) and P⁢(s) of each fish to computationally manipulate the interplay between the controller, plant and the sensory feedback. Estimating a data-driven plant model We estimated a data-driven model for the plant dynamics P⁢(s) of each fish. For the purpose of visualizing the plant models graphically, it is useful to treat the plant, P⁢(s), as a filter through which motor commands are processed. At a given frequency, ω, the filter’s behavior can be represented as a complex number, called a phasor, that is obtained by evaluating the transfer function P⁢(s) at that frequency, namely P⁢(j⁢ω), where j=-1. The locus of phasors as a function of frequency is called the frequency response function. We estimated the frequency responses of the locomotor dynamics of each fish using the position of the fish y⁢(t) and its nodal point u⁢(t). To visualize the variability across plants, we used a Nyquist plot. On such a plot, the gain and phase of the response of the plant at a given frequency are represented in polar form, where the x axis is the real part and the y axis is the imaginary part—both dimensionless. The gain is the distance from the origin 0+j0, and the phase is the angle measured counterclockwise from the real axis. Nyquist plots of each individual's estimated plant revealed substantial differences between locomotor plants of individual fish (Figure 2A). Despite these differences, the frequency responses shared a common structure: the locomotor dynamics of each fish acted as a low-pass filter on the movements of the nodal point. This common structure facilitated the application of parametric modeling, reducing the complexity of analysis while enabling computational manipulations of the model system. We used the physics-based parametric model of locomotor dynamics of Eigenmannia described by Sefati et al. (2013) for the plant (see Materials and methods for derivation): (1) P⁢(s)=km⁢s2+b⁢s Here, m, k, and b represent mass, gain, and damping, respectively, and s is complex frequency in Laplace domain (see, e.g. Roth et al., 2014). We estimated the parameters in the parametric plant model for each fish based on their individualized frequency responses via numerical optimization (see Materials and methods) (Figure 2B). Frequency responses for the estimated parametric models are illustrated in Figure 2C (see black lines in Figure 2A for corresponding Nyquist plots). Finally, we estimated the plant for a ‘merged’ fish, in which the data from the three fish were concatenated as a single fish. The differences in the frequency responses between individuals resulted in substantial differences (about twofold) in estimated model parameters (Table 1). Moreover, the merged fish has plant dynamics that differ from each of the individual fish (Figure 2A, bottom), highlighting the need to use individualized plants for the analysis of the control system of each fish. Figure 2 Download asset Open asset System identification of plant dynamics. (A) Nyquist plots of the estimated plant models for each individual fish and the merged fish. Blue, green and cyan point clouds correspond to bootstrapped data estimates for the low (0.55 Hz), medium (0.95 Hz) and high (2.05 Hz) frequencies, respectively. Black circles and cross marks represent the mean for measured and bootstrapped data, respectively. Black lines represent the response of the estimated model with diamonds indicating their values at the test frequencies. Each point on the black lines correspond to the complex-valued frequency response of the plant model at a specific frequency. For example, the purple dot on the lower-right panel corresponds to the merged-plant-model’s response at 0.7 Hz. The gain and phase associated with this response are shown on the associated Bode plots given in (C). (B) Reconciliation of physics-based and data-driven models. The solid lines represent the natural feedback control system used by the fish for refuge tracking. The dashed lines represent copies of signals used for parametric system identification. P^⁢(s) represents the parametric transfer function for the plant dynamics of the fish with ‘unknown’ system parameters. The parametric system identification estimates these parameters via minimizing the difference (modeling error) between the actual output of the fish y⁢(t) and the prediction of the model. (C) Gain and phase plots of the frequency response functions of the estimated parametric models for each fish and the merged fish corresponding to the black lines in (A). Table 1 Estimated parameters of the plant model for each fish as well as the merged fish. Fish 1Fish 2Fish 3Mergedk⁢(N/m)0.32280.22000.16220.2385b⁢(N⁢s/m)0.04170.01860.02180.0269m⁢(k⁢g)0.00250.00250.00250.0025 Examining the effects of feedback on behavioral variability Despite the differences in plant dynamics, fish produced remarkably similar tracking performance, consistent with previously published reports (Cowan and Fortune, 2007; Roth et al., 2011). This behavioral robustness could be achieved via precise tuning between the controller and plant dynamics of each fish. Alternatively, the central nervous system (CNS) may be implementing robust feedback control without the need for precise tuning. To test these hypotheses, we built feedback control models that permit computational manipulation of the relationships between controller and plant. Specifically, we swapped the controllers and plants between fish using these computational models (Figure 3). If each fish required precise tuning for consistent behavior, we would expect to see increased variability for the swapped models. Alternatively, a robust feedback controller might be insensitive to mismatch between C⁢(s) and P⁢(s) pairs. Figure 3 Download asset Open asset Computational brain swapping between fish. We computationally swapped the individually tailored controllers between fish. The feedback control diagrams illustrate computational brain swaps. The controller from the light gray fish, C2⁢(s), is used to control the plant of the dark gray fish, P1⁢(s), and vice versa. We used the second order model proposed by Cowan and Fortune (2007) to represent the input–output behavioral response of the fish: (2) G⁢(s)=A⁢ωn2s2+2⁢ζ⁢ωn⁢s+ωn2 We estimated the model parameters for each fish using the position of the fish y⁢(t) and the refuge r⁢(t). In other words, we generated individualized parametric transfer functions that capture the input–output behavioral performance of each fish. Parameters varied by about 15–20% (Table 2). Table 2 Estimated parameters of the second order input-output model for each fish as well as the merged fish. Fish 1Fish 2Fish 3MergedA0.530.600.520.57ξ0.580.550.520.55ωn⁢(r⁢a⁢d/s)8.559.517.888.50 We investigated how the variability in plant dynamics (parameters varied by about twofold; Table 1) is mitigated at the level of behavior (parameters varied by 15–20%; Table 2). Specifically, we inferred a controller for each fish using models of their respective plant dynamics and input–output behavioral responses. Given the plant P⁢(s) and behavioral performance G⁢(s) of each individual fish, we can infer the controller for each fish using the following equation (see Materials and methods and Cowan and Fortune, 2007): (3) C⁢(s)=G⁢(s)(1-G⁢(s))⁢P⁢(s) Estimates of controllers and plants for individual fish allowed us to implement computational manipulations in the system models. Given a model of each fish’s plant and controller, we computed the transfer function of the closed-loop system using each fish’s own controller when matched with its own plant (‘matched’, Figure 4A), via the equation (see Equation 11, Materials and methods): (4) Gmatched,i⁢(s)=Pi⁢(s)⁢Ci⁢(s)1+Pi⁢(s)⁢Ci⁢(s), for i=1,2,3. Then, to test the hypothesis that the animals rely on precise tuning between their plants and controllers, we substituted the controller of each fish with the plant dynamics of other fish (see ‘swapped’ in Figure 4A), that is a simulated brain transplant, namely (5) Gswapped,i,j⁢(s)=Pi⁢(s)⁢Cj⁢(s)1+Pi⁢(s)⁢Cj⁢(s) whereby each controller j=1,2,3 is paired with another fish’s plant i≠j, for a total of 6 ‘swapped’ cases. If the controllers and plants need to be co-tuned, then we would expect a significant increase in variability in the swapped models. Figure 4 Download asset Open asset Effects of feedback on behavioral variability. At the top of each column is a control diagram that corresponds to all figures in that column. (A) Frequency-response gain and phase plots of the estimated input–output models for the matched (orange), swapped (gray), and merged (green) fish in a feedback control topology. (B) Variability of the matched (orange) and swapped (gray) models. Dashed line is the behavioral variability observed in the tracking data across fish. (C) Same as in (B) but for the loop gain, P⁢(s)⁢C⁢(s). (D) Same as in (C) but for the loop gain, P⁢(s)⁢C⁢(s). To quantify such changes in variability and to evaluate the fitness of a given computational model for explaining biological data, we defined two metrics termed ‘model variability’ and ‘behavioral variability’. Model variability quantifies the variability of the complex-valued frequency responses of matched or swapped models across a range of frequencies. Behavioral variability, on the other hand, provides a conservative estimate of the variability observed across trials in the real fish. See Model variability and Bootstrap estimate of behavioral variability in Materials and methods. Using these metrics, we computed the variability of the matched models and compared against the behavioral variability observed in real fish. Unsurprisingly, variability of the matched models (orange region in Figure 4B) remained well below the behavioral variability across the entire frequency range of interest (dashed line in Figure 4B). Surprisingly, however, the model variability remained below the behavioral variability even for swapped models (Figure 4B, gray region versus dashed line). These results highlight the fact that sensory feedback can attenuate the variability of closed-loop models despite mismatch between the controller and plant pairs. In other words, feedback models do not require precise tuning between the controller and plant to achieve the low variability we observed in the behavioral performance of the animals. Having established that variability of the closed-loop models is robust to the relations between the plant and controller in a feedback system, we examined the role of feedback. This was achieved by examining the loop gain, that is the dynamic amplification of signals that occurs in feedback systems (Figure 4C). The only difference between the loop gain model and the closed-loop model is the absence and presence of feedback, respectively. For our model, this was calculated as the product of the plant and controller in both matched and swapped cases. This removal of feedback revealed dramatic variability in the loop gain at frequencies below about 1 Hz (Figure 4D). This variability was well above the behavioral variability observed in fish. In contrast, at frequencies above 1 Hz, the model variability was slightly reduced. These results indicate that sensory feedback attenuates behavioral variability in the biologically relevant range of tracking frequencies at a cost of slightly increased variability at high frequencies. Parametrizing the range of neural controllers As predicted by Cowan and Fortune (2007), each of the feedback controllers obtained above for the averaged fish responses had high-pass filtering characteristics despite the differences in their dynamics (Figure 5A). What is the range of neural controllers that, when used in this feedback control topology, leads to behavior that is indistinguishable from the real fish? In other words, how well tuned to the plant does the neural controller need to be to achieve these behavioral performances? Figure 5 Download asset Open asset controllers the behavioral variability. (A) Bode plots for the estimated controllers for the matched (orange) and merged (green) fish under feedback Each controller high-pass filter behavior across the frequency range of Gray represent the range of controllers that produce behavioral responses consistent with behavioral variability. (B) The range of behavioral input–output transfer functions consistent with behavioral variability. We used the to a range of controllers that generate similar behavioral responses 2011). Specifically, this a parametric transfer function all controllers for the merged Here, is the transfer function of the merged plant and is any and function. Equation 6 all controllers for the merged plant However, we were in the of controllers that indistinguishable behavioral from real fish. To achieve we computed the range for the input–output system dynamics, of real fish Specifically, we calculated the for the gain and phase responses of the input–output transfer function models estimated while the behavioral variability (see Materials and The in Figure serve as the range of frequency responses that are consistent with behavioral variability of the real fish. For each of these transfer functions G⁢(s) that were consistent with the behavioral variability of the fish, we to generate corresponding controllers using Equation The for gain and phase responses of these controllers gray the of controllers that, when within the feedback control topology, produce behavioral consistent with the performance of the real fish (see Figure 5A). that the controllers calculated in Equation 3 the structure of Equation 6 when for the associated plant dynamics P⁢(s) of each in other words, the controller estimates are to be for the associated These results indicate that the neural controllers need not be precisely tuned to their associated plant dynamics. We found a range of controllers that the fish could implement to generate consistent behavioral We that each of these controllers had high-pass filtering Discussion task control animals to cope with dramatic but nevertheless common variations of their plant dynamics between individuals. Further, individuals can experience variations in plant dynamics over time such as in weight during (Van Handel, et al., during and or objects These changes in mismatch between the neural controller and the locomotor plant of individual animals. This mismatch is similar to that by swapping plants and controllers across individuals, that moment-to-moment variability can be through sensory feedback. the interplay between the task plant, behavioral performance, and requires understanding the of the closed-loop control topology. Given the range of morphophysiological features observed across individuals within a our results that there is a range of controller manifest as each individual could use to achieve consistent biologically relevant behavioral performances. As a we expect to see more variation at the level than is revealed by task performance for that rely on closed-loop data-driven and physics-based models of locomotor dynamics A of this is the identification of a data-driven plant model for the locomotor dynamics of a animal based on behavioral observations To achieve we a system identification that to a physics-based parametric transfer function model with a data-driven model the a model from for is an for the dynamics of a system. For a used model in is the model for dynamics in the and 1993; and physics-based models have to be in the dynamics of biological and there are for at the of all motor may to a complex model that does not capture behavior. Data-driven system identification are used to a model based on data Kiemel et al., 2011; Uyanik et al., In data-driven system identification may a in which only a model structure is an or frequency response However, data-driven typically generate numerical transfer function estimates to represent animal behavior. Alternatively, the that we in this the structure of a specific physics-based model but its parameters relying on data-driven system identification to those parameters. In this case, about the underlying model and data-driven system identification. identification can provide a between data-driven and physics-based We the parametric model of Sefati et al. (2013) for the plant but estimated the model parameters using data-driven system identification results that the data-driven estimates for the plant dynamics the structure of this model (Figure 2A). Effects of variability in plant dynamics results two on variability in plant dynamics. On the one hand, estimates of the closed-loop controllers were to the dynamics of the plant of individual fish. This was an of our strategy for the the controllers from the plant and closed-loop dynamics. On the other hand, our closed-loop control models were robust to variability of the plant or controller, indicating that precise tuning is not for this behavior. A analysis that these results are not to this but rather are a of feedback control systems, see and for a Specifically, the frequency function of the feedback controller C⁢(s) with to plant P⁢(s), in the closed-loop The of the controller to the plant dynamics is a frequency function. It on the gain and phase of both the measured closed-loop transfer function as well as the plant