Author response: Working memory capacity of crows and monkeys arises from similar neuronal computations
2021-10-26

Article Figures and data Abstract Editor's evaluation eLife digest Introduction Results Discussion Materials and methods Data availability References Decision letter Author response Article and author information Metrics Abstract Complex cognition relies on flexible working memory, which is severely limited in its capacity. The neuronal computations underlying these capacity limits have been extensively studied in humans and in monkeys, resulting in competing theoretical models. We probed the working memory capacity of crows (Corvus corone) in a change detection task, developed for monkeys (Macaca mulatta), while we performed extracellular recordings of the prefrontal-like area nidopallium caudolaterale. We found that neuronal encoding and maintenance of information were affected by item load, in a way that is virtually identical to results obtained from monkey prefrontal cortex. Contemporary neurophysiological models of working memory employ divisive normalization as an important mechanism that may result in the capacity limitation. As these models are usually conceptualized and tested in an exclusively mammalian context, it remains unclear if they fully capture a general concept of working memory or if they are restricted to the mammalian neocortex. Here, we report that carrion crows and macaque monkeys share divisive normalization as a neuronal computation that is in line with mammalian models. This indicates that computational models of working memory developed in the mammalian cortex can also apply to non-cortical associative brain regions of birds. Editor's evaluation In this study, Hahn et al., taught crows to perform a multi-item working memory task designed to mimic traditional monkey tasks. Using a combination of behavior and electrophysiology, the authors convincingly show that the neural mechanisms that limit working memory capacity in primates also limit working memory capacity in crows. Such cross-species comparisons are fundamental to understanding the computational constraints that are placed on cognition and the brain. https://doi.org/10.7554/eLife.72783.sa0 Decision letter Reviews on Sciety eLife's review process eLife digest Working memory is the brain’s ability to temporarily hold and manipulate information. It is essential for carrying out complex cognitive tasks, such as reasoning, planning, following instructions or solving problems. Unlike long-term memory, information is not stored and recalled, but held in an accessible state for brief periods. However, the capacity of working memory is very limited. Humans, for example, can only hold around four items of information simultaneously. There are various competing theories about how this limitation arises from the network of neurons in the brain. These models are based on studies of humans and other primates. But memory limitations are not exclusive to mammals. Indeed, the working memory of some birds, such as crows, has a similar capacity to humans despite the architecture of their brains being very different to mammals. So, how do brains with such distinct structural differences produce working memories with similar capacities? To investigate, Hahn et al. probed the working memory of carrion crows in a change detection task developed for macaque monkeys. Crows were trained to memorize varying numbers of colored squares and indicate which square had changed after a one second delay when the screen went blank. While the crows performed the task, Hahn et al. measured the activity of neurons in an area of the brain equivalent to the prefrontal cortex, the central hub of cognition in mammals. The experiments showed that neurons in the crow brain responded to the changing colors virtually the same way as neurons in monkeys. Hahn et al. also noticed that increasing the number of items the crows had to remember affected individual neurons in a similar fashion as had previously been observed in monkeys. This suggests that birds and monkeys share the same central mechanisms of, and limits to, working memory despite differences in brain architecture. The similarities across distantly related species also validates core ideas about the limits of working memory developed from studies of mammals. Introduction Working memory (WM) can hold information for a short period of time to allow further processing in the absence of sensory input (Cowan, 2017; Oberauer et al., 2018). By bridging this gap between the immediate sensory environment and behavior, WM is a cornerstone for complex cognition. It is a very flexible memory system, yet severely limited in its capacity. While this capacity is often seen as a general cognitive bottleneck, for simple stimuli, like colors, the capacity is very similar between humans, monkeys, and crows (Balakhonov and Rose, 2017; Buschman et al., 2011; Cowan, 2001; Luck and Vogel, 1997). Different models have been proposed to conceptualize how this capacity limit arises. This work motivated many psychophysical and electrophysiological experiments that in turn led to a spectrum of more refined models of WM (Ma et al., 2014). ‘Discrete models’ of WM argue that a fixed number of items can be stored. Once this capacity is reached, an additional item can only be maintained if it replaces a previous item (Awh et al., 2007; Fukuda et al., 2010; Luck and Vogel, 1997; Vogel and Machizawa, 2004; Zhang and Luck, 2008). ‘Continuous models’ describe WM as a flexible resource that is allocated to individual items. A minimum amount of this resource has to be allocated to each item for successful retention, thereby resulting in a capacity limit (Bays and Husain, 2008; van den Berg et al., 2012; Wilken and Ma, 2004). On the neurophysiological level, models of WM capacity suggest that interference between memory representations (‘items’) within the neuronal network is a source of information loss and capacity limitation (Bouchacourt and Buschman, 2019; Lundqvist et al., 2016; Lundqvist et al., 2011; Schneegans et al., 2020). Interference may arise due to divisive normalization that appears as competition between items, related to oscillatory dynamics (Lundqvist et al., 2016; Lundqvist et al., 2011), WM flexibility (Bouchacourt and Buschman, 2019), and neuronal information sampling (Schneegans et al., 2020). Divisive normalization is a computational principle that acts upon neurons when presenting multiple stimuli simultaneously, it normalizes neuronal responses by creating ‘a ratio between the response of an individual neuron and the summed activity of a pool of neurons’ (Carandini and Heeger, 2011, p. 51). An effect related to divisive normalization can be observed when two stimuli are presented either individually or simultaneously within the receptive field of a visual sensory neuron. The neuron’s firing rate when the stimuli are presented simultaneously becomes normalized by the populations’ responses to each individual stimulus (Carandini et al., 1997; Heeger, 1992). This effect also occurs in relation to attentive processes and has been formalized into the ‘normalization model of attention’ (Reynolds et al., 1999; Reynolds and Heeger, 2009). Normalization of neuronal responses is commonly observed in many species throughout the animal kingdom, not just in sensory, but also in cognitive domains (Carandini and Heeger, 2011). Investigations into WM capacity and model predictions focus mostly on humans and monkeys. By extending this work to include birds, one can gain a unique comparative perspective. Crows have a similar limit in WM capacity and neuronal correlates of WM are comparable to monkeys’ (Balakhonov and Rose, 2017; Nieder, 2017). But while the neuronal architecture of sensory areas is similar between birds and mammals, higher associative areas, critical for WM, do not share a common architecture between the species (Stacho et al., 2020). Therefore, an outstanding question is whether modern models of WM such as the ‘flexible model’ capture WM capacity in general, or if their predictions (e.g., divisive normalization) are confined to the mammalian neocortex. To resolve this, it is crucial to investigate the avian brain to understand how its different organization can produce such similar behavioral and neurophysiological results. While the neuronal correlates of WM maintenance in birds have been investigated in some detail (Diekamp et al., 2002a; Hartmann et al., 2018; Rinnert et al., 2019; Rose and Colombo, 2005; Veit et al., 2014), a neurophysiological investigation of WM capacity limitation is still lacking. The avian forebrain structure, nidopallium caudolaterale (NCL) is a critical component of avian WM. The NCL is considered functionally equivalent to the mammalian prefrontal cortex (PFC) (Güntürkün and Bugnyar, 2016; Nieder, 2017), as it receives projections from all sensory modalities (Kröner and Güntürkün, 1999), projects to premotor areas (Kröner and Güntürkün, 1999), and is a target of dopaminergic innervation (Waldmann and Güntürkün, 1993). To investigate the neurophysiology of WM capacity in birds, we adopted a task design developed for monkeys (Buschman et al., 2011) to use it with carrion crows (Corvus corone). Our animals were trained to memorize an array of colors and to indicate which color had changed after a short memory delay, while we performed extracellular recordings of individual neurons in the NCL using multi-channel probes. We expected to find a clear correlate of WM representations in NCL neurons and a load-dependent response modulation based on divisive normalization of neuronal responses. This would allow us to evaluate if the behavioral WM capacity observations of crows fit a ‘discrete’ or ‘continuous’ WM resource model. If the neuronal responses also fit the contemporary neurophysiological models of WM capacity limitations (Bouchacourt and Buschman, 2019; Lundqvist et al., 2016; Lundqvist et al., 2011; Schneegans et al., 2020), it would further suggest that crows and monkeys have convergently evolved a similar neurophysiological basis for WM capacity despite a different architecture of the critical forebrain structures. Results The WM capacity of crows is similar to that of monkeys The behavioral performance was influenced by the number of colored squares on the screen. It significantly decreased with an increasing number of ipsilateral squares (median performances, load 1: 95.88%, load 2: 78.31%, load 3: 58.21%; Friedman test: Χ2 = 92.00, p < 0.001, Figure 1B). We ran a generalized linear model with ipsilateral load (i.e., load of hemifield where a color changed), contralateral load (i.e., load of hemifield without a color change) and their interaction as predictors for performance (R2adj 0.78, F(460,456) = 555.00, p < 0.001). We found that the number of ipsilateral colors significantly reduced performance (βipsi –0.177, t(459) = –18.00, p < 0.001), whereas the number of contralateral colors did not (βcontra –0.021, t(459) = –1.77, p = 0.0772; Figure 1C). There was also a significant interaction between ipsilateral and contralateral load (β = –0.024, t(458) = –4.28, p < 0.001). We compared this model to a reduced model, where we omitted the non-significant βcontra and found that this reduced model (R2adj 0.78, F(460,457) = 828.00, p < 0.001) explained the performance equally as well (|ΔLLR| = 0.0102). Therefore, we conclude that contralateral load by itself did not significantly affect performance. We calculated the capacity K (see Materials and methods) for all full WM loads (i.e., two to five items). The capacity K peaks at four items (mean ± SEM: 3.05±.038, Figure 1—figure supplement 1). These observations are very similar to observations made in primates (Buschman et al., 2011) and fully reproduce our earlier behavioral findings (Balakhonov and Rose, 2017). Figure 1 with 1 supplement see all Download asset Open asset Behavioral overview. (A) Behavioral paradigm (reproduced from Balakhonov and Rose, 2017). The birds had to center and hold their gaze for the duration of the sample and delay period, and subsequently indicate which colored square had changed. (B) Boxplot of performance for different ipsilateral loads (i.e., on the side where the change occurred). Horizontal lines indicate significant differences between loads, box indicates the median, 1st, and 3rd quartile (whiskers extend to 1.5 times the inter-quartile range). (C) Mean performance matrix for ipsi- and contralateral load combinations (values are rounded to the nearest integer). Additional contralateral items at an ipsilateral load of 1 barely affected performance (bottom row). At higher ipsilateral loads additional contralateral items reduced performance more clearly (middle and top row). Statistical modeling revealed an interaction at these higher loads (see text). Neurons of the NCL encode the color identity and maintain it in WM We recorded 362 neurons from the NCL of two crows performing the WM task (delayed change localization). All reported effects were also present in each individual bird (Figure 3—figure supplements 1–2), we, therefore, pooled the data for population analysis. A large subset of neurons responded to the presence of a color (i.e., at load 1) by substantially increasing or decreasing their firing rate relative to baseline. This change in firing rate occurred selectively, depending on the presented color either in the sample (Figure 2A) or the delay period (Figure 2—figure supplement 1). For most neurons, this difference in firing rate between the two possible colors became attenuated when the load increased from one to two colors, and it was further attenuated from two to three colors. To quantify this effect, we calculated the amount of information about the color identity at a neuron's favorite location as the percent explained variance (PEV) during a memory load of one, two, or three items in bins of 200 ms (see Materials and methods for details). Most neurons did not sustain information about color (measured as a significant PEV, henceforth ‘information’) throughout the entire sample or memory delay but rather had shorter periods in which the information was significant (Figure 2A bottom). Figure 2 with 2 supplements see all Download asset Open asset Color discrimination in the neuronal response (information, percent explained variance [PEV]) generally decreases with load, but some neurons show the opposite effect. Shown are the three ipsilateral load conditions (i.e., load increases on the same side as the neuron’s favorite location). Ipsilateral loads are one (blue), two (yellow), and three (red). The labels ‘color A’ and ‘color B’ always refer to the same pair of colors at the neuron’s favorite location, irrespective of the load condition. (A) Example of a sample neuron with color information decline at load 1 (blue), load 2 (yellow), and load 3 (red). Top: raster plot, where every dot represents a single spike during the individual trials (rows of dots); middle: peri-stimulus-time histogram (PSTH) of average firing rate (solid line for color ID 1, dashed line for color ID 2) with the standard error of the mean (shaded areas); bottom: percent explained variance of color identity (a measure of information about color) along the trial, the line at the top of the y-axis indicates significant bins. (B) Same as in (A) for an example of a delay neuron with information gain at a higher load. To better capture the time points when the individual neurons carried information, we performed a hierarchical clustering analysis of the PEV values of the individual neurons at load 1 (see Materials and methods for details). We found a total of seven clusters that were organized into two overarching groups (Figure 3A). Group 1 contained neurons (n = 227) that showed peak information during the sample and early delay phase, while group 2 contained neurons (n = 135) that showed peak information during the delay phase. For each neuron, we then calculated if it carried a significant amount of color information by applying a permutation test (for all bins at load 1, see Materials and methods). The individual neurons were then further classified into three groups depending on the phase in which they had a significant amount of information (Figure 3B). Overall, 37.57% (n = 136) of neurons were significant during the sample phase, 9.39% (n = 34) of neurons were significant during the memory delay, and 14.64% (n = 53) of neurons were significant during both the sample phase and the memory delay (all proportions of neurons were significantly higher than expected by chance [binomial test, see Materials and methods, all p < 0.001]). Refer to Figure 2A for an example neuron, significant at load 1 with a large differentiation in firing rate between color identities (a large PEV) and a loss of differentiation with increasing ipsilateral load. Further inspection of individual neuronal activity revealed, however, that a substantial number of neurons responded differently. Instead of losing information at higher loads, many neurons gained information (e.g., Figure 2B, Figure 2—figure supplement 2). Thus, we additionally performed the permutation testing for loads 2 and 3 to determine which neurons had significant information (see Materials and methods). We found that many of the neurons that did not have significant information at load 1 did have significant information at load 2 and load 3 (Figure 3C). For the memory delay, more than half of the significant neurons we detected were only significant for either load 2 or load 3, compared to only 36% of neurons that were significant at load 1 (Figure 3C middle). By including the higher loads in our analysis, we found a total of 249 (68.78%) sample neurons and 94 (25.97%) delay neurons. For the population analyses, we subsequently pooled all significant neurons into three groups (one per load). These pooled groups were then each subdivided into sample and delay neurons (i.e., ‘sample-load1’, ‘delay-load1’, ‘sample-load2’, etc., see Table 1 in the Materials and methods for an overview). Figure 3 with 2 supplements see all Download asset Open asset Overview of recorded neurons. (A) The neuronal population can be best described by seven individual clusters. (B) Percentages of neurons (total n = 362) with significant color information at load 1, during the sample and the delay. (C) Percentages (rounded) of significant neurons in individual load conditions for sample (n = 249), delay (n = 94), and sample and delay (n = 133). The pieces of the pies depicting significance at a specific load relate to the number of significant neurons in the respective phase (e.g., 36% of the 94 delay neurons (i.e., 34 neurons) are significant at load 1 (all pieces contain blue), these are the same neurons that make up the of the total 362 neurons in Table 1 Overview of significant The that a neuron of the respective group had a significant percent explained variance (PEV) in the respective load condition. The that a neuron of the respective group did not have a significant PEV in the load respective condition. The pooled groups contained only neurons with a for the respective load condition. 1 2 3 1 and 2 1 and 3 2 and 3 1, and group group group 3 The neuronal population has information with increasing load The clustering analysis that the population of neurons as a did sustain the color information throughout the entire (Figure 3A). the information of each of the three populations’ and the populations’ time this result (Figure Figure supplement 1). the of the stimulus the average information a that ms after stimulus and at an throughout the memory delay, the array Results obtained from neurons of the of monkeys distinct of WM capacity (Buschman et al., 2011). This that increasing ipsilateral load (i.e., load in the hemifield the target for which information is affect neuronal processing while increasing contralateral load This effect be further in birds due to the full of their and to the behavioral results and in line with the results from monkeys, we found a effect of ipsilateral load on the information as was a in information when the load increased from one item to two items (Figure and The of a item only decreased the maintained information further (Figure The load was more during the sample period than during the memory delay where the information at a the load effect was only present for ipsilateral If the number of items on the contralateral side was the information about the colors at the favorite location did not change (Figure To our results to the results obtained in monkeys we also the of Buschman et al., for testing the ipsilateral load effect during the sample and delay phase, by each phase into an early and a and second ms of the and and second ms for the We did find a significant in information with an in load from one three in the early = p < 0.001, = and = p < 0.001, = sample period and the early = p = = and = p = = delay period (Figure There was a large and significant between one and two items multiple early and sample p < 0.001, early delay p < 0.001, delay p = and one and three items multiple early and sample p < 0.001, early delay p = delay p but difference between loads 2 and 3 multiple all p The maintenance of a significant amount of information at higher loads for three items, early sample = p < sample = p < early delay = p < delay = p < 0.001) and its when items were to the hemifield are of a flexible resource and not an WM. if is a flexible in error trials a but amount of resource still be allocated to an Indeed, error analysis see Materials and methods) for the load 2 and 3 conditions further this The amount of information in the early and sample phase 2: = p = = p < load 3: = p < Figure and was significantly than in trials 2: = p = = load 3: = p = = was further maintenance throughout the memory delay at load (Figure PEV at loads 2 and 3 in error trials delay, all This indicates that a to report which color had changed at higher loads and three ipsilateral from a amount of information encoding during the sample phase that was not maintained throughout the delay. A possible would be on error the color information was after the sample phase, it was not into a that a was not to on The amount of information on trials is not with the simple (all or model, but fit the and model’ and Luck, 2008). Figure with 2 supplements see all Download asset Open asset encoding at the population (A) Color information explained variance [PEV]) decreases with an increasing ipsilateral load (i.e., on the same side as the neuron’s favorite but not with an increasing contralateral load (i.e., on the opposite side to the neuron’s favorite location). (B) On trials color is during the early and phase of the sample to a during the early and delay. On error trials color information can be found in the early sample phase at load and in the sample phase at loads 2 and 3 of load 1 error trials was omitted due to their very Statistical comparisons of error information were performed on and sample each early and delay each areas, and error indicate the standard error of the loads produce divisive neuronal responses We to understand the neuronal mechanisms the information loss at higher WM For we how the responses of individual sample and delay neurons changed when the load increased from one color to two colors. For the populations’ and the an increasing number of items reduced the amount of information about the color identity (Figure This effect was due to neurons that had a large difference of firing between the color A and color at load 1 PEV, information about and reduced differentiation at load 2 PEV, or information about Figure and Heeger, 2011) can this effect. the computation that when two stimuli are presented simultaneously. In a a neuronal response becomes to with a normalization of the stimuli (Carandini and Heeger, 2011). to our context, a of would be a reduced differentiation between two color identities at load 2 differences in firing rate (for each stimulus by at load 1 would be normalized at load 2 in information Figure therefore, that was for neurons with significant information at load We tested for in the NCL by a and a sensory interaction for each neuron for the sample phase and the memory delay phase (Reynolds et al., see Materials and methods for details). indicates how the neuronal response is by a color at the favorite location of the neuron in relation to a color either is presented indicates how the color with the color when both are presented simultaneously. of both and between and The of a color the response to the color by either the firing rate of the color the a higher firing rate than the or increasing the firing rate for the color the a higher firing rate than the If was this to or neuronal responses be an at the population level, resulting in a significant between and with a of around (Bouchacourt and Buschman, We compared for the sample and delay phase as one see Materials and methods for for two groups of neurons information at load and neurons information at load 1 or at load Figure supplement 1). We found that was present in both sample and delay (Figure sample neurons had a of (R2adj = p < 0.001, = and delay neurons had a of (R2adj = p < 0.001, = As the were not significantly different from this indicates that and color had an on neuronal responses. We show that was in the neuronal and as a of this neurons had generally information about the color identity at load Figure with 2 supplements see all Download asset Open asset Divisive was for neuronal responses of neurons losing information (A) but not for neurons color information at load 2 indicates how the neuronal response is influenced by a relative to a second color when either is presented interaction indicates how the neuronal response is influenced by either color when both were simultaneously. to indicate an of both colors. or indicate a of a (A)