Source: https://jov.arvojournals.org/article.aspx?articleid=2474935
Timestamp: 2019-04-19 14:41:50+00:00

Document:
Rishabh Jain, Rachel Millin, Bartlett W. Mel; Multimap formation in visual cortex. Journal of Vision 2015;15(16):3. doi: 10.1167/15.16.3.
An extrastriate visual area such as V2 or V4 contains neurons selective for a multitude of complex shapes, all sharing a common topographic organization. Simultaneously developing multiple interdigitated maps—hereafter a “multimap”—is challenging in that neurons must compete to generate a diversity of response types locally, while cooperating with their dispersed same-type neighbors to achieve uniform visual field coverage for their response type at all orientations, scales, etc. Previously proposed map development schemes have relied on smooth spatial interaction functions to establish both topography and columnar organization, but by locally homogenizing cells' response properties, local smoothing mechanisms effectively rule out multimap formation. We found in computer simulations that the key requirements for multimap development are that neurons are enabled for plasticity only within highly active regions of cortex designated “learning eligibility regions” (LERs), but within an LER, each cell's learning rate is determined only by its activity level with no dependence on location. We show that a hybrid developmental rule that combines spatial and activity-dependent learning criteria in this way successfully produces multimaps when the input stream contains multiple distinct feature types, or in the degenerate case of a single feature type, produces a V1-like map with “salt-and-pepper” structure. Our results support the hypothesis that cortical maps containing a fine mixture of different response types, whether in monkey extrastriate cortex, mouse V1 or elsewhere in the cortex, rather than signaling a breakdown of map formation mechanisms at the fine scale, are a product of a generic cortical developmental scheme designed to map cells with a diversity of response properties across a shared topographic space.
Map formation is a core process in neural development (Cang & Feldheim, 2013; Chklovskii & Koulakov, 2004; Goodhill, 2007; Kaas, 1997). A major focus of theoretical work on map formation has been the primary visual cortex (area V1; Antolík & Bednar, 2011; Carreira-Perpinán & Goodhill, 2004; Durbin & Mitchinson, 1990; Goodhill, 1993; Keil & Wolf, 2011; Kohonen, 1982; Koulakov & Chklovskii, 2001; Miller, 1994; Obermayer, Blasdel, & Schulten, 1992; Obermayer, Ritter, & Schulten, 1990; Paik & Ringach, 2012; Swindale, 1982, 2000; Swindale & Bauer, 1998; Wolf, Bauer, & Geisel, 1994; Yu, Farley, Jin, & Sur, 2005). V1 is atypical among visual cortical areas, however, in that most V1 cells respond to the same basic shape—an elongated edge-like feature—with multiple continuous parameters including orientation, spatial frequency, and ocular dominance (Bosking, Crowley, & Fitzpatrick, 2002; Hubel & Wiesel, 1968; Hübener, Shoham, Grinvald, & Bonhoeffer, 1997; Lennie, 2003; Nauhaus, Nielsen, Disney, & Callaway, 2012; Swindale, Shoham, Grinvald, Bonhoeffer, & Hübener, 2000; Yu et al., 2005). This type of single feature/multiparameter map has been called a “polymap” (Swindale, 2000), and though the spatial layout of the polymap's multidimensional parameter space may be complex, it retains the simple property that cells' response preferences tend to progress gradually across the map, and in such a way that each cell's preferred stimulus is roughly the average of its neighbors'.
In contrast to this, midlevel visual areas such as V4 contain cells selective for a multitude of categorically distinct shape features, perhaps dozens to hundreds or more (Anzai, Peng, & Essen, 2007; Op de Beeck et al., 2008; DiCarlo, Zoccolan, & Rust, 2012; Gallant, Braun, & Van Essen, 1993; Kobatake & Tanaka, 1994; Pasupathy & Connor, 1999, 2001; Rothschild & Mizrahi, 2015; Rust & DiCarlo, 2010; Sato, Uchida, & Tanifuji, 2009), yet all sharing the same coarse topographic organization. Unlike a single-feature polymap, a signature characteristic of a multifeature map, hereafter a “multimap,” is fine-scale response heterogeneity (Rothschild & Mizrahi, 2015). In particular, each neuron in a multimap is in general surrounded by cells with categorically different response preferences whose average is not a meaningful quantity (Figure 1).
Conceptual illustration of a multimap. Top panel: In a multimap, cells with a diversity of response types (indicated by different colored symbols) are finely intermixed, while all sharing the same coarse-scale topographic organization. In an extrastriate visual area, the different symbols might represent different types of junctions, contour shapes, etc. Bottom panel: During multimap development, learning is restricted to the most active cortical neighborhood, called the LER (purple circle). A second threshold further restricts RF plasticity to only the most active neurons within the LER (yellow circles in the example shown). This competition within the LER promotes response-type diversification (yellow circles vs. all other features). At the same time, colearning by cells of the same type within the LER promotes smooth map formation for that type.
From a developmental perspective, the need to simultaneously form multiple interdigitated submaps each representing a categorically distinct feature type—where each submap may have its own continuous parameters (making it in effect a “subpolymap”)—is challenging in that all of the cells that will eventually make up the submap for response-type A must compete with their immediate neighbors to differentiate themselves from the other response types, while cooperating with their dispersed A-type “conspecifics” to achieve uniform visual field coverage for the A type at all orientations, scales, etc. All the while, the cells that will eventually take on response types B, C, D, etc., must simultaneously differentiate and develop their own well-formed submaps, all within the same shared topographic space (Figure 1).
Previous map formation schemes have generally included a distance-dependent (e.g., Gaussian or Mexican hat) local smoothing mechanism that correlates the learning rates of neighboring cells, pushing them towards similar developmental outcomes (Antolík & Bednar, 2011; Carreira-Perpinán & Goodhill, 2004; Durbin & Mitchison, 1990; Goodhill, 1993; Keil & Wolf, 2011; Kohonen, 1982; Miller, 1994; Obermayer et al., 1992; Obermayer et al., 1990; Swindale, 1982, 2000; Yu et al., 2005; though see Hansel & van Vreeswijk, 2012; Koulakov & Chklovskii, 2001). Some form of smoothing is necessary to promote the development of map topography and to produce smooth layouts of continuous map parameters, but conventional compulsory smoothing mechanisms, which homogenize neural responses locally, are fundamentally incompatible with the fine-scale intermixing of response types that is the hallmark of a multimap. We hypothesized that multimap formation requires both (a) activity-dependent learning that encourages neurons to locally diversify to cover a spectrum of different response types, as in classical competitive learning systems (Carpenter & Grossberg, 2010; Rumelhart & Zipser, 1985); coupled with (b) a permissive smoothing mechanism that allows—but does not compel—cells to colearn when they lie within the same strongly activated cortical neighborhood. In the following, we show that these spatial and activity-dependent learning criteria together, but not separately, produce well-formed multimaps from initially unstructured neural populations.
To study the requirements for multimap development, we compared the performance of a hybrid rule with both spatial and activity-dependent learning criteria to two control learning rules, one of which included only a spatial, and the other only an activity-dependent learning criterion (Table 1).
Receptive field update equations for Spatial, Activity, and Hybrid learning rules.
consisted of a global rate term α(t) and a product of spatial and postsynaptic activity-dependent terms defined in Table 1.
Illustration of Spatial, Hybrid, and Activity learning rules. (A) An oriented bar stimulus activates an input pattern p on a 33 × 33 retina. (B) Resulting bump of activity on the cortex, where column height represents neural response. Response of each neuron R was computed as Pearson's correlation coefficient between the corresponding components of p and the neuron's RF matrix wx. Inset illustrates a 33 × 33 RF weight vector for a particular cortical neuron. The green dashed square indicates the smaller central region of the retina over which cortical RFs were initialized and stimuli were delivered; this avoided boundary cutoff effects. The blob just inside the green border shows this cell's initial area of sensitivity, located in rough topographic correspondence to its location in the cortical array. (C) Same cortical response bump as in (B), with color representing each neuron's learning rate L according to Equation 2 and Table 1 for the three learning schemes. M indicates the maximally active neuron (or center of mass of the activity bump, with similar results). (D) Illustration of RF update for low, medium, and high learning rates. Zoomed portion of RF weight matrix just before learning is shown at left; red ellipse shows Gaussian fit to sensitive region; blue box marks retinal stimulus. After learning, RF looks more like the most recent stimulus.
Time course of simulation parameters for Spatial and Hybrid learning. Spatial: (A) Gaussian scale parameter σ (measured in cortical lattice units) and (B) global learning rate α both decreased linearly over simulation time, as in previous SOM models. Hybrid: (C) During the early phase of development (up to 150,000 steps), in which 100% of the neurons in the LER crossed the learning threshold, the Hybrid rule mainly acted to smooth the initial topographic jitter. Once basic map topography was established, differentiation within local neighborhoods was encouraged by a jump in the learning threshold (% scale on the left axis) and a corresponding increase in the global learning rate (right axis). Interestingly, when learning threshold and learning rate were increased more gradually (lowest red line), spatial correlations left over from the topographic refinement phase persisted in the final map, appearing as a coarse columnar structure reminiscent of Spatial learning outcomes (see Figure 11).
Learning a single-feature map: Contrasting Spatial, Hybrid, and Activity learning rules. (A) Mature map (after 450,000 stimulation steps) shows preferred orientation (hue) and tuning (saturation). Only central 44 × 44 area of cortex is shown. (B) RF profiles for the line of cortical neurons indicated by arrows in (A). Red ellipses were fit to RF weight vectors, showing RF center, preferred orientation, aspect ratio, and size. (C) Cell-to-cell changes in orientation preference and RF center location are plotted for same line of cells. Dashed black line marks horizontally aligned averages of the two quantities; r value is Pearson's correlation coefficient. (D) Combined plot showing preferred orientation and RF center for each cortical neuron. Gray lines show TO between each neuron's cortical site (lying on a regular grid) and its RF center in scaled retinal coordinates (marked by a colored dot). Dot color code is same as in (A). (E) A random sample of TOs is collected; average absolute TO is shown as a gray bar at upper left. Smaller blue bar shows length of vector sum of all TOs, demonstrating a lack of a global orientation bias. The length of the gray lines was scaled up by a factor of 2.2X. (F) Quantifying uniformity of visual field coverage: Histogram of number of RF centers falling in a randomly placed square box sized to contain five centers on average (red data). Green bar shows hypothetical result for a square grid of RF centers, blue shaded region corresponds to RF centers drawn randomly from a 2-D uniform distribution.
Smoothing role of LER in Hybrid learning. (A) Left column: Initial TOs for each RF were drawn uniformly from a circle of radius r centered on the neuron's cortical site. Three levels of initial RF jitter are shown (r = 0.62, 6.2, and 18.5 cortical lattice units), chosen to achieve initial average TOs of 0.5, 5.0, and 15.0 lattice units. By default, simulations were initialized with medium jitter (r = 6.2). Purple circle indicates an LER of diameter = 7; a larger LER was used in Figure 4 (diameter = 15). Right column: Outcomes of Hybrid learning rule. (B) When initial jitter was mostly contained within the LER (true for low and medium jitter cases), mature map showed refined topography. Otherwise, map failed to organize. Overlaid green dots indicate the low, medium and high jitter cases shown in (A).
was used to define a circular learning-eligible region (LER) of radius r centered at location xM within which the learning rate was flat.
In Figure 2C, bar height again indicates neural response, while color shows each cell's learning rate L. Top and slice views make clear that in spatial learning, L is tied to a neuron's distance from M and not its activity level—though the two are correlated. In hybrid learning, within the LER (indicated by a purple circle) L is determined only by the neuron's response level, independent of its distance from M. As an alternative to centering the LER on the maximally activated neuron, in a few experiments the LER was placed at the centroid of the maximally activated cortical neighborhood. Results were similar in all major respects (data not shown). Activity learning was equivalent to Hybrid learning with an infinite LER. For each neuron at each time step, learning pushed the neuron's RF in the direction of p(t) (Figure 2D). The time courses of α(t), σ(t), and %(t) are given in Figure 3.
As expected, spatial learning produced maps resembling those in V1 of carnivores and primates, with quasiperiodic orientation columns and interspersed “pinwheels” (Carreira-Perpinán & Goodhill, 2004; Kaschube et al., 2010; Figure 4A, left frame). We examined the progression of RFs along a representative line of cortical cells (marked by arrows). RF location, size, orientation, and aspect ratio are represented by red ellipses derived from two-dimensional (2-D) Gaussian fits to the sensitive zone in each cell's synaptic weight matrix (Figure 4B, C, left frame). Average RF aspect ratio was AR = 2.11 ± 0.18. RF centers are shown as colored dots, plotted in scaled retinal coordinates. Gray lines indicate topographic offsets (TOs) of RF centers from their associated cortical lattice positions. Average absolute TO for the line of cells was 2.13 ± 1.17 cortical lattice units—that is, a cell's RF center was on average offset from its nominal visual field position by 2.13 cortical intercell spacings (translated into visual coordinates). To quantify map smoothness, we measured the average change in RF center position between neighboring neurons (ΔP = 1.24 ± 0.52 lattice units, down from an initial value of 6.4), as well as average changes in preferred orientation (Δθ = 14.21 ± 10.18°, down from 45°; Figure 4C, left frame). We checked for an orientation bias in the overall pattern of TOs, and found none (see Figure 4D, E and caption text). Finally, to quantify the uniformity of visual field coverage, we measured the number of RF centers that fell within a randomly positioned square box sized to contain five RF centers on average. A pdf of the actual number of RF centers/box is shown in Figure 4F (left frame, red data), compared to the same measurement performed on a regular grid of RFs (green data) or RFs drawn randomly from a 2-D uniform distribution (gray-blue data). Compared to the uniform distribution, Spatial learning produced an excess of both small and large hit counts, corresponding to areas of dispersed and clumped RFs, respectively.
To verify that our instantiation of Spatial learning was not limited to organizing maps with a single stimulus parameter (orientation), but was capable of developing a bona fide polymap with multiple continuous parameters (as has been previously shown with Kohonen-style learning rules; Goodhill, 1993; Swindale, 2000), we repeated the simulation experiment for the same single feature but now with two parameters—orientation and spatial frequency (Supplemental Figure S1A). The resulting orientation map was very similar to that generated by the spatial rule in the orientation-only case (Supplemental Figure S1C; compare to Figure 4A, left panel). However, the cortex now showed coordinated orientation and spatial frequency structure (Supplemental Figure S1D, E). In particular, in keeping with previous studies (Hübener et al., 1997; Nauhaus et al., 2012; Swindale, 2000), the iso-orientation and isospatial-frequency contours tended to be orthogonal to each other (Supplemental Figure S1F). This test confirmed that Spatial learning was capable of generating maps of a single basic response type with multiple parameters (the developed RFs are shown in Supplemental Figure S1B), in accord with the prevailing model of V1. This result is in effect a control condition for the spatial learning experiment described below: In contrast to its effectiveness in mapping multiple continuous parameters across the visual field for a single basic feature type (as in a conventional polymap) the Spatial rule, owing to its compulsory local smoothing operation, will be incapable of forming a multimap.
We next simulated map development with a Hybrid rule (i.e., that included both spatial and activity-dependent learning rate factors). Compared to spatial learning, the mature map was noisier, tending towards a “salt and pepper” appearance at a fine scale (Figure 4A, center frame; see also (Espinosa & Stryker, 2012; Hooser, 2007; Kaschube et al., 2010; Ohki, Chung, Ch'ng, Kara, & Reid, 2005; Rothschild & Mizrahi, 2015; Rothschild, Nelken, & Mizrahi, 2010; Yen, Baker, & Gray, 2007; Yoshimura, Dantzker, & Callaway, 2005). Average cell-to-cell changes in RF orientation (Δθ = 35.0 ± 25.8) and position (ΔP = 1.59 ± 0.82) were larger than in Spatial learning (Figure 4C, center frame). However, other characteristics were represented with greater specificity. Orientation tuning was slightly sharper (AR = 2.39 ± 0.15; Figure 4B, center frame), average TOs were halved (TO = 1.01 ± 0.58; Figure 4D, E center frame), and visual field coverage was more uniform (Figure 4F, middle frame). We performed Kolmogorov-Smirnov tests and confirmed that the Spatial and Hybrid AR, Δθ, ΔP, and TO distributions were highly significantly different from each other (p < 0.0001).
We noted that even though the LER was a circular region with a diameter of 15 cortical lattice units (see purple circle in Figure 2C, middle frame) and remained fixed in size throughout the simulation, the Hybrid rule drove down average TOs from an initial value of 5 to a final value near 1, one-fifteenth the diameter of the LER. Though unintuitive, this emergence of topography on a much finer scale than the learning radius is consistent with classical results in map formation (e.g., see Kohonen, 1990). To confirm that the LER diameter nonetheless plays some role in determining map outcomes, we ran additional simulations to explore the relationship between the size of the LER and the ability of the Hybrid rule to produce a sharpened topographic map. We found that when the LER was large enough to contain the majority of the cells responsive to a stimulus despite their initial random jitter, the positional noise was suppressed during development and the final map was well formed (Figure 5A, upper and middle rows). However, if the initial RF jitter was so large that many of the cells responsive to a given stimulus fell outside the LER, the hybrid rule was incapable of refining the map (Figure 5A, lower row). The relationship between the initial RF jitter, the LER diameter, and the quality of map outcome is summarized in Figure 5B. Thus, the scale of the LER is important in that it must be large enough to rein in the initially jumbled RF centers, but not so large that the hybrid rule loses any meaningful sense of locality (and effectively degenerates to the Activity rule; see below).
The results of Hybrid learning shown in Figure 4 were produced using highly simplified inputs—a single oriented bar presented on the retina at each developmental step (Figure 2). This scenario meant that learning would always take place in a single well-defined region in the map, and that RFs would essentially become isomorphic to individual stimuli. To test whether Hybrid learning can handle more complex inputs that activate neurons over a much larger region of the cortex (i.e., not mostly contained within the scale of a single LER), we repeated the Hybrid learning experiment of Figure 4 with stimuli consisting of four randomly placed/oriented bars (Supplemental Figure S2A). We found that maps formed using the complex stimuli were very similar to those developed with simple inputs (Supplemental Figure S2C, D), having a local salt-and-pepper organization, low topographic distortion but a for a few strays, and simple, oriented RFs (Supplemental Figure S2B). Thus, Hybrid learning was competent to process cluttered inputs containing multiple items in variable arrangements. Since this issue was not our main focus, however, and we saw no significant difference in the outcomes using simple versus complex stimuli, we reverted to the use of simple stimuli in the remaining simulations described in the paper.
The Activity rule produced sharply tuned RFs (AR = 2.32 ± 0.16; Figure 3B, right frame), but lacking any mechanism to bias nearby neurons to learn together, was unable to form a map at all (Δθ = 45.00° ± 29.38°, ΔP = 6.76 ± 3.76, TO = 5.12 ± 2.04; Figure 4, right column).
In summary, when the input stream contained a single basic feature type, both Spatial and Hybrid learning schemes produced viable maps; the Spatial rule emphasized fine-scale smoothness but with significant topographic distortion; Hybrid maps were locally noisy but had more uniform visual field coverage. The Activity rule produced sharply tuned cells but was incapable of forming a topographic map.
We next compared the capacity for multimap formation using Spatial versus Hybrid learning rules (the activity rule was dropped from consideration given its inability to form a map of any kind). Each neuron's RF (i.e., weight vector) was expanded to include inputs from four mutually exclusive stimulus channels at each retinal location. A stimulus consisted of a spatial pattern—an oriented bar, in order to maintain consistency with the preceding experiments—presented in one of the four channels. Each neuron was initialized in the same coarse retinotopic fashion as before, but where the initial circular Gaussian blob-shaped sensitive zones were aligned between the four channels for each neuron (meaning that the cell initially received all four types of input from a given part of the retina). The amplitudes of the blob-shaped initial RF sensitivity profiles in the four channels were drawn from an exponential distribution, so that different cells initially showed different preferences for stimuli appearing in different channels (Figure 6A). During development, stimuli were presented at random positions/orientations as before, now also randomly alternating between the four stimulus channels. Given that a neuron, over the course of development, could retain some sensitivity to more than one stimulus channel, it's nominal response type (color coded) was assigned at any given time according to which stimulus channel generated the cell's maximal response over all channels, positions, and orientations. To accommodate the increased number of response types that could develop in this model, the number of neurons in the cortical network was increased 4-fold.
Learning a four-feature multimap with Spatial versus Hybrid rules. (A) Initial state of cortical array shows random initial type preferences; colors here represent four feature types. Four-column bar graphs (enlarged in inset) represent each neuron's response to its best stimulus of each type. Spatial rule (left panel): (B) At maturity, cortical map is partitioned into single-feature islands with mixed-type borders. (C) Individual islands contain compressed orientation maps. Only blue-type islands are shown for clarity, other three single-type maps were similar. (D) Distant separation of same-type islands led to gaps in retinotopic coverage within each type map (only blue-type islands are shown). Hybrid rule (right panel): (E) Neurons are sharply tuned, and four types are completely intermixed across the cortex. (F) RF centers within a single type map are roughly uniformly distributed over the cortex (only blue-type map is again shown). (G) Combined orientation-position map for blue type shows uniform visual field coverage, and entire spectrum of orientations contained within each local region. Similar results were found in simulations with up to 16 types.
The outcomes of Spatial versus Hybrid learning were radically different in this case where the inputs were of multiple types (Figure 6). As expected, spatial learning homogenized local areas of the cortex. As a result, the cortical map was partitioned into discrete single-type islands (Figure 6B, C), with poor tuning and incomplete, patchy coverage of retinotopic space for each type (Figure 6B through D). The breakdown in retinotopic coverage, which grew worse as the number of types increased (data not shown), could be explained by a simple geometric model (Figure 7). (We note that Spatial learning with two independent feature channels is equivalent to a previously proposed model of ocular dominance column formation with no between-eye correlations; Goodhill, 1993).
Geometric model explaining breakdown of retinotopic coverage in multifeature spatial learning. The breakdown in coverage could be explained by the following simple model, where N is the number of feature types. We found island radius rI is roughly proportional to σ(0), the initial scale of the Gaussian spatial interaction function g(). If islands are modeled as squares with width 2rI, and islands of a given type are assumed to occur at uniform intervals across the cortical surface, then edge-to-edge distance between two nearest islands of the same type is . This means that for purposes of co-training, cells of a given type only lie within reach of cells of their same type within their own island, and thus do not know about, and cannot form a continuous map with, neurons of the same type in other islands. The dependence of the island separation on means the coverage problem grows worse as the number of feature types increases. (A) Feature preference maps when learning eight independent types, with small, medium, and large LERs (corresponding to r = 3, 7, and 10 cortical lattice units). Grouping of a single feature type into islands is clearly evident (only a single feature type is shown. Red circles are fit to the type islands. (B) Conceptual model of patchy multimap development by the spatial rule; colors represent four independent feature types. (C) Island radii estimated from the circular fit corresponded very closely to σ(t = 0). Box plots mark 25th and 75th percentile limits of the distribution and ends of the whiskers mark the highest and lowest values of the data lying within 1.5 times the interquartile range of the box edges (the fence). (D) Estimated minimum distances between same-type island pairs based on circular fits (median = horizontal line within the box) matched closely the theoretical predictions in each of the three cases.
In contrast, Hybrid learning allowed the complete intermixing of different response types at a fine scale (Figure 6E), while also producing uniform coverage of both retinotopic space and preferred orientation within each of the four submaps (Figure 6F, G). To determine quantitatively the degree to which each of the feature-specific submaps was well organized, we measured the changes in retinotopic position and preferred orientation between all pairs of nearest neighbors within each submap (where nearness was determined by the cells' RF centers in visual coordinates). As shown in Figure 8A, distances between nearest neighbors of the same type were concentrated towards medium inter-RF distances, with fewer short and long distances compared to either (a) a uniform random distribution of the same number of RF centers (black random histogram), or (b) a random selection of same number of units from the multimap without regard to type preference (blue shuffled histogram). Likewise, changes in preferred orientation between nearest neighbor cells were biased towards small values compared to what we would expect from random orientation preference assignment (i.e., 45°), or the average change in preferred orientation between neighboring units when the same number of units was drawn randomly from the multimap without regard to type preference (Figure 8B). Together, these measurements support the conclusion that the Hybrid rule can produce multiple interdigitated feature maps that are individually smoothed both in terms of visual field coverage and unit-to-unit changes in a continuous RF parameter.
Local order in the four-feature multimap developed with the Hybrid rule. (A) Distribution of RF centers' distances (ΔP) to their nearest neighbors of the same feature type showed fewer short and large distances within the same type (four thin red curves, one for each type; thick red curve shows the average), and were thus more uniformly distributed in space, as compared to the two controls: “Shuffled” (blue curves) shows data when type identities were shuffled; “Random” (black curves) shows data for randomly distributed RF centers. (B) Similarly, the distribution of orientation differences (Δθ) to nearest neighbors of the same type showed fewer very small and very large angles compared to the shuffled and random controls, indicative of a more even progression of orientation preferences across each submap than would be expected by chance.
To determine whether the Hybrid rule would scale to more response types, we simulated up to 16 types and found that: (a) type commitment tends to occur early in development (Figure 9), and (b) once type commitment has occurred, development of each type-specific submap occurs virtually independently since each neuron eventually crosses the activity-dependent learning threshold only for stimuli of its own type (Figure 10). Hybrid learning can therefore produce well-formed multimaps even for a large number of independent response types. The main physical limitation on the number of interdigitated maps is that nearest neighbors within each submap grow further apart as the number of types increases, thus increasing “wirelength” demands, though this limitation would be less severe in a three-dimensional (3-D) cortex compared to the 2-D model studied here.
Early feature type commitment in the Hybrid algorithm. At the outset of multimap development, each cortical cell was initialized with a level of sensitivity that was randomly drawn from an exponential distribution for each of its N features channels (see inset in Figure 6A, for N = 4). A neuron's primary type was defined as the type whose best stimulus (of that type) generated the cell's maximum response over all stimuli of all types. As shown here, the fraction of cells that switched type fell rapidly towards 0 with advancing simulation time, indicating that under Hybrid learning, cortical units committed early to their respective type maps, after which development of the different features maps proceeded virtually independently.
Independence of development of individual feature maps within a multimap under the Hybrid rule. An area of cells at the center of the cortical array of radius (corresponding in size to an LER with r = 7, N = 4) was tracked during a simulation to measure the mean activity of the fraction of neurons crossing the learning threshold θ(t) as a function of simulation time. (A) As the number of feature types was increased by a factor of two in a series of runs, the length of simulation time was also doubled. Following the initial period of topographic refinement (which also varied in length by powers of two), the abrupt jump in the mean firing rate corresponds to the abrupt jump in learning threshold (to upper sixth %tile, see Figure 3). For purposes of faster simulation times, we shortened (∼10 fold, 150,000 to 15,000) the topographic refinement phase by using a higher learning rate (results were qualitatively and quantitatively similar to runs with lower learning rates). (B) The near invariance in the mean activity of suprathreshold neurons over time, when time was normalized by the number of feature types, attested to the independence of development of the different feature maps, regardless of the number of types involved.
Developmental outcomes for Hybrid rule using different time courses of %tile(t) and α(t). Orientation and TO maps for six time courses of learning threshold and global learning rate as shown in Figure 3. At the end of the initial period of topographic refinement, during which all neurons in the LER could learn, when the learning threshold and rate were increased gradually (leftmost column), the final map showed coarse columnar structure and map contortion similar to that seen under spatial learning. When the learning threshold abruptly jumped to allow only the top 3% of active neurons in the LER to learn and at a high rate (rightmost column), the mature map became highly differentiated (i.e., salt-and-peppery) at a fine scale and showed very low topographic distortion. Intermediate cases led to intermediate results.
Our main finding is that a Hybrid learning rule that restricts learning to those neurons that (a) are contained anywhere within a strongly activated cortical neighborhood, designated the LER, and (b) that are strongly activated themselves, allows for the development of a multimap consisting of multiple functionally distinct feature maps that are physically interdigitated and in topographic register within the same cortical area. We showed that having both an LER that transiently defines the cortical neighborhood where synaptic plasticity can occur, but having the learning rate within the LER depend only on a cell's activity level (i.e., with no fine-scale dependence on spatial location), is necessary to allow multimap formation: When the learning rate is influenced by a cell's location within the LER, as is the case in Spatial learning, the effect is to correlate closely neighboring cells' learning rates and to smooth the map at the fine scale, which works against the local differentiation of RF response types needed for multimap formation. Or, when a cell's learning rate is determined only by its activity level, regardless of its location in the cortex, a map does not form at all.
Important parameters in multimap formation are the learning threshold and learning rate, which control the transition between smoothing and differentiation (Figure 3). Early in development, a low learning threshold, which encourages most or all of the neurons in a neighborhood to learn together, and a low learning rate, which leads to a gentle averaging process within the learning neighborhood, together promote smooth topographic map formation with little or no response-type differentiation. This is appropriate since the neurons in all the maps that will eventually form must agree on the same basic topography. Once this initial stage of topographic map formation is complete, raising the learning threshold so that only a fraction of the most active neurons within a neighborhood learn together, and raising the learning rate so that active neurons differentiate from their neighbors quickly, leads to the symmetry-breaking competitive interaction that partitions the map into different response-type channels.
This differentiation process is similar to that seen in classical competitive learning schemes (Carpenter & Grossberg, 2010; Rumelhart & Zipser, 1985). For example, suppose an input containing a feature of type A is presented to the system, and only the top 25% of the neurons responding to the input in the LER are allowed to learn. That cohort of neurons will begin to differentiate itself from the other cells in the neighborhood by updating their RFs en masse in the direction of feature type A. When an input of type B stimulates the neighborhood, neurons in the type A cohort are less likely to respond since their RFs have begun to specialize, so a different 25% of the neurons will tend to be selected based on their elevated activity levels, and begin to represent stimuli of type B. With repeated presentations of their preferred stimulus types, each cohort will continue to differentiate from the others until it becomes virtually unresponsive to inputs of other types (Figure 6). Once the cortex has partitioned itself into different response-type channels in this way, as the activity threshold for learning increases further, the neurons within each channel begin to differentiate further to represent different parameter values of their respective response types, including orientation, scale, etc. When the threshold becomes so high that no further learning occurs, multimap formation is complete.
Is a Hybrid rule the default in cortical map development?
We showed that the Hybrid rule is not limited to organizing multimaps. When presented with inputs representing only a single stimulus type, the hybrid rule produces V1-like orientation maps with a salt-and-pepper appearance at the fine scale (Espinosa & Stryker, 2012; Hooser, 2007; Kaschube, 2014; Ohki et al., 2005; Rathelot & Strick, 2006; Rothschild & Mizrahi, 2015; Rothschild et al., 2010; Yen et al., 2007; Yoshimura et al., 2005; note that salt-and-pepper maps are not limited to visual cortex; see also Rathelot & Strick, 2006; Rothschild & Mizrahi, 2015; Rothschild et al., 2010). Given that a Hybrid rule is competent to produce maps in both single and multifeature cases (i.e., both unimaps and multimaps), we may speculate that the default rule used in cortical development is of this general kind, since it encourages topographic map formation where applicable, but under the conditions illustrated in Figures 4 and 6, imposes no constraint on the eventual similarity, or not, of immediately adjacent neurons. This permits neurons with heterogeneous response properties to develop in close quarters. According to this interpretation, the salt-and-pepper layout seen in V1 in rodents would be viewed not as a crudely organized sensory cortex, but as a generic, well-organized cortical visual area that happens to represent only a single basic feature type. Indeed, as shown in Figure 4, in exchange for accepting larger transitions in orientation preference from cell to cell on average (Figure 4C), the salt-and-pepper maps organized by the Hybrid rule achieve more uniform visual field coverage than the smooth maps organized by the spatial rule; the avoidance of gaps in coverage may be the more urgent design criterion when relatively few neurons are available in a particular cortical area. For its part, spatial learning might be brought to bear in special cases—for example, in V1 in carnivores and primates, where (a) a single feature type dominates so that constructing a smooth map is even possible, (b) enough neurons are available so that the warping of visual field coverage caused by simultaneous smoothing along multiple parameter dimensions is functionally insignificant, and (c) subsequent processing stages somehow benefit from spatial pooling of cells with gradual cell-to-cell parameter changes (e.g., for motion processing).
Finally, it is worth noting that a crude form of spatial learning can be achieved using low learning thresholds and learning rates within a hybrid learning scheme, since these settings allow mass colearning within a local neighborhood (Figure 11; see also Chklovskii & Koulakov, 2004; Koulakov & Chklovskii, 2001). Where smoothness is of critical importance, however, our results suggest that dedicated neural “hardware” that correlates the responses of immediately adjacent neurons may be the most effective approach.
What about areas like the inferotemporal cortex that contain many response types, but rather than random intermixing, show signs of local correlations both horizontally and vertically?
The inferotemporal (IT) cortex contains cells with a wide range of object-related response properties, and as would be expected for a multimap, it is not uncommon for nearby cells in IT to respond best to very different stimuli (for review, see Op de Beeck et al., 2008). In a departure from an idealized multimap, however, cells in IT tend to have related response properties moving both horizontally across the map (up to 400 μm), and vertically across the entire thickness of the cortex (Op de Beeck et al., 2008; Desimone & Gross, 1979; Fujita et al., 1992; Gochin, Miller, Gross, & Gerstein, 1991; Tanaka, 2003; Yamane, Tsunoda, Matsumoto, Phillips, & Tanifuji, 2006). One way to interpret these data is to assume that any given small neighborhood of IT cortex does not contain a full mixture of response types, as would be the case for an idealized multimap, but only cells representing the (potentially many) features associated with a particular object or group of related objects. In this more intricately structured type of map, nearby cells would tend to fire together, since they would tend to respond to the same object or object class, but would not necessarily have smoothly varying response properties from neuron to neighboring neuron as in an idealized polymap. Rather, immediately adjacent cells might respond best to different features of their common preferred object(s). While we did not study map formation using natural images as inputs, when development is driven by complex high-dimensional natural stimuli, a Hybrid rule could presumably encourage simultaneous differentiation of multiple cells within the LER, all moving off in different directions in feature space, so that ultimately small cortical neighborhoods tend to represent different features of the same objects. Further work will be needed to understand this more realistic developmental situation. In the mean time, the most definitive way to distinguish whether a cortical area has developed under the influence of a spatially-determined learning rate, as in Kohonen-style learning, versus a Hybrid rule of the kind studied here, is to measure whether the response properties of neurons can in general be predicted by interpolating the response properties of their immediately surrounding neighbors. Put another way, to the extent that immediately neighboring cells lie in a low-dimensional continuous parameter space of the same object feature, a map is more likely to be a polymap. To the extent that this continuous parameter space test fails at the finest special scale, it becomes more likely that neurons have been allowed to differentiate locally based on their activity levels, as occurs in classical competitive learning systems (Carpenter & Grossberg, 2010; Rumelhart & Zipser, 1985).
It is also possible that a cortical area is a multimap horizontally, but something closer to a polymap vertically. That is, minicolumns might differentiate as units (because the activity-dependent term in the hybrid learning rate might depend on activity of an entire column, and apply to all neurons within the column). Superimposed with this, there could be a smoothing mechanism that correlates learning rates of neighboring cells along the vertical dimension in the column. This could lead to the spreading out of cells in the vertical dimension to cover the continuous parameter space of the column's preferred object feature.
Why should multiple response types be comapped in the same cortical area?
Developing and maintaining multiple maps within the same cortical area could be advantageous for at least three reasons. First, in a hierarchical system, it is efficient from a wiring perspective to form multiple different compound features at Level n+1 if those features can be constructed from the same set of elemental features contained in Level n. For example, L-junctions and T-junctions are both composed of oriented edge elements, so that neurons representing those two compound features can be fed by the same set of axons from a preceding area. Second, from the converse (but still wiring) perspective, it is efficient to house distinct features at Level n+1 if those features form the common elements from which even higher-order features at Level n+2 will be constructed. Third, when features are mutually exclusive (such as an L-junction and a T-junction), only one of which can be present at a given image location at a time, colocalizing the cells in the cortex makes it convenient for local circuits to impose competitive (e.g., “winner take all” type) interactions between them to sharpen their selectivity.
What mechanism(s) could account for a flat learning-enablement profile within the LER?
A core feature of Hybrid learning is a mechanism that enables all of the neurons in the LER to learn, but within the LER, the learning rate is determined exclusively by a neuron's activity level rather than its location. This implements the Hebbian principle that postsynaptic neurons that fire more in response to a given input should learn more. Any structurally imposed tendency for neighboring neurons to have correlated learning rates, simply because they are neighbors in the cortex, pushes neighboring neurons towards similar developmental outcomes, and works against the fine-scale diversity required for multimap formation (Figure 12).
Effect of substituting a Gaussian learning eligibility function in the Hybrid rule in lieu of the default flat LER. (A) A four-feature multimap was learned with the default Hybrid rule, leading to an outcome similar to that shown in Figure 6. RF orientation preferences of only a single feature channel are shown. (B) Combined RF position/orientation map is shown for the same feature channel as in (A), with results qualitatively similar to Figure 6. (C) When a Gaussian spatial function was substituted for the flat LER function in the Hybrid rule, spatial correlations between neighboring neurons were introduced, leading to a smoothing of the mature orientation map. The effect was exaggerated as the number of feature types increased (data not shown). (D) The increase in map smoothness was accompanied by increased contortion, as in the Spatial rule shown in Figure 4.
A key prediction of our model is that during development of a higher order visual area (or any area of cortex with fine-scale mixing of response types), RF plasticity is gated by two nonlinear selection processes. First, learning should be enabled at any given time only in spatially restricted—heavily activated—areas of the map. Second, within those learning-enabled areas, a cell's learning rate should be determined only by its activity level, independent of its precise cortical location; and an equally or more active neuron outside the LER should show no plasticity.
What kinds of neural mechanisms could give rise to a flat learning enablement profile within a cortical region? One possibility is that when an input to a particular region reaches a certain threshold level of intensity, a subpopulation of interneurons activated by the stimulus transiently shuts down one or more other classes of interneurons in the region. For example, Somatostatin-expressing interneurons (SOM), which inhibit all other interneuron types, might be capable of producing such an effect (Pfeffer, Xue, He, Huang, & Scanziani, 2013). Along these lines, a recent review (Karnani, Agetsuma, & Yuste, 2014) discusses mechanisms supporting “blanket inhibition” in the cortex, but also points to a possible mechanism for producing “holes” in the blanket: activation of vasoactive intestinal polypeptide (VIP) interneurons that inhibit somatostatin (SOM) interneurons. A local silencing of SOM inhibition by VIP interneurons might transiently enable plasticity to a uniform degree across the affected region, producing a “flat” LER. Alternatively, a level playing field for synaptic plasticity might be generated within a strongly activated cortical region by a saturating concentration of a volume-transmitted modulatory factor that is necessary for the induction of plasticity (Zoli, Jansson, Syková, Agnati, & Fuxe, 1999). Though these mechanistic issues are complex and will require further study, it is likely to be most important for successful multimap formation in the brain that the learning rate have a flat profile at the center of the LER (rather than at the edges), especially early in the map formation process, before response type commitment has occurred. The flat center removes fine-scale location as a factor in determining learning rates, so that immediately neighboring neurons with different random initial conditions are free to diverge to represent different feature types. Once type commitment has occurred (Figure 9), and the learning activity threshold is set high, the hybrid rule functions similarly to a conventional Kohonen rule operating on cells of a single basic response type, since only cells of a single type will be able to cross the high threshold. From that point on, the requirement of a flat spatial learning profile is less certain.
We have focused on the formation of maps of “simple cells,” that is, cells containing only a single RF subunit representing a particular shape feature. In reality, many or most cells in visual cortex, beginning already in V1, are “complex” (Gilbert, 1977; Hubel & Wiesel, 1962). Complex cells pool over multiple spatially displaced RF subunits (Chen, Han, Poo, & Dan, 2007; Hubel & Wiesel, 1962; Mel, 1997), which is thought to be the main mechanism responsible for increasing spatial invariance from stage to stage in the ventral visual processing stream (Cadieu et al., 2007; Fukushima, 1980; Hubel & Wiesel, 1962; Lee, Grosse, Ranganath, & Ng, 2009; Mel, 1997; Nandy, Sharpee, Reynolds, & Mitchell, 2013; Ranzato, Huang, Boureau, & LeCun, 2007; Riesenhuber & Poggio, 1999; Rust & DiCarlo, 2010; Serre, Wolf, & Poggio, 2005; Sharpee, Kouh, & Reynolds, 2013; Ullman, Vidal-Naquet, & Sali, 2002; Wallis & Rolls, 1997). Our focus here on multimap formation involving simple rather than complex cells was justified by the assumption that the same two rules need to be followed whether the cells involved in the map formation process are simple or complex. In particular, a representational unit—whether the lone RF subunit of a simple cell, or one of the several subunits within a complex cell RF—should learn a stimulus only when (a) the subunit lies within a strongly activated cortical neighborhood, and (b) the subunit is itself strongly activated by the stimulus. What differs about training a map containing complex cells is that an additional mechanism must be included that ties together the multiple subunits contained within a complex cell's RF. The most commonly proposed mechanism to do this is a temporal trace rule (Berkes & Wiskott, 2005; Földiák, 1991; Wallis & Rolls, 1997). While this is clearly an important part of a full understanding of map formation in extrastriate visual areas, the process that encourages the multiple RF subunits of a complex cell to differentiate into distinct spatial variants of the same basic stimulus can occur after the process that differentiates cells into separate feature maps, and so, we argue, can be studied separately.
What is the relationship to multilayered feature hierarchy models?
A multimap is a structure containing a potentially large number of different topographic maps each with its own internal spatial parameters such as orientation, scale, etc. According to this definition, hierarchical object recognition models that have been studied for many years typically contain multimaps in each major layer of the hierarchy. However, the shared topographic organization in these systems has generally been explicitly imposed by the system designer, rather than developed from scratch (Fukushima, 1980; Le Cun et al., 1990; Lee et al., 2009; Ranzato et al., 2007; Riesenhuber & Poggio, 1999; Serre et al., 2005) though also see (Hyvärinen & Hoyer, 2001).
How many different feature maps does a higher cortical area contain?
In a hierarchical sensory system, where more complex features are built by conjoining simpler features, the number of different possible shape features can grow extremely rapidly. To illustrate, assuming we begin with just three different local contour elements (straight, convex and concave) that can be joined two or three at a time to form junctions, the three elements can be combined to create nine types of L-junctions, each with two orientation parameters, and 27 types each of T-junctions, forks and arrows, each with three orientation parameters, for a total of nearly 100 feature types, each with two or three orientation parameters. If just two of these intermediate-level features are conjoined to form higher order features, the number of types nominally jumps to ∼10,000, each with from four to six spatial parameters. This simple analysis ignores the fact that not all features are equally useful, so that a learning rule with access to natural image statistics indicating which features are most common, and a supervisory or reinforcement input indicating which discriminations are most important, can pare the system down to a smaller number of the most useful features, and thus escape the combinatorial explosion (see Mel & Fiser, 2000, for a treatment of this issue). Inescapably, however, a hierarchical system that builds complex features by conjoining simple features is bound to end up with a very large number of different features that must be separately mapped across the visual field (as in a convolutional neural network), each with their respective internal parameter variations (Dollar, Tu, & Belongie, 2006; Lee et al., 2009; Ranzato et al., 2007; Serre et al., 2005; Yamins, 2013).
It remains an open question how many independent feature maps coexist within any extrastriate visual area, but it is possible to estimate how many feature maps can in principle be accommodated within a particular area by estimating the number of neurons needed to form one map and comparing that to the total number of neurons available. Single unit recordings in macaque area V4 suggest that the number of distinguishable shape features represented by V4 neurons may number in the hundreds or more (Anzai et al., 2007; DiCarlo et al., 2012; Gallant et al., 1993; Kobatake & Tanaka, 1994; Pasupathy & Connor, 1999, 2001; Rust & DiCarlo, 2010; Sato et al., 2009). Are there enough neurons in V4 to map the entire visual field for such a number of features? A rough calculation suggests the answer is “yes.” The total surface area of V4 in macaque is about 540 mm2 (Felleman & Essen, 1991) with a cell density of ∼100,000/mm2 (as compared to ∼200,000 neurons under a square millimeter in the striate cortex, (O'Kusky & Colonnier, 1982). To be conservative, assuming only half of those neurons are available to participate in feature maps, this yields an estimate of 27 million available neurons. According to physiological data regarding RF sizes at different eccentricities in V4 (Freeman & Simoncelli, 2011), we estimate that just 50 cells are required to tile the visual field for a single feature type, assuming zero RF overlap. Assuming a 10-fold RF overlap factor (so that every point in every map is covered by 10 neurons), 500 neurons would be required per feature to provide retinotopic coverage. Features also have spatial parameters, however, such as orientation, scale, etc. Assuming each feature has two continuous parameters, each of which has 10 gradations, this consumes another factor of 100 neurons, thus, 50,000 neurons per feature map. Under these assumptions, V4 would have the neuronal capacity to hold 540 complete feature maps. Assuming all neurons in all maps are physically well mixed in three dimensions, a block of roughly 8 × 8 × 8 cell bodies would contain one cell of each response type, so that neurons of the same type would on average be separated by eight cells' bodies in any direction—well within the footprint of a pyramidal neuron's dendritic arbor (Elston & Rosa, 1998).
Other combinations of numbers of features and numbers of parameters can be contemplated. For example, V4 contains enough neurons to represent 50 feature maps each with three parameters, or 2,500 maps each with two parameters assuming coarser sampling of the parameter values, and so on. It is also worth noting that the spatial pooling of RF subunits by complex cells as discussed above may reduce the number of neurons needed to tile the visual field, thus easing the numerical requirement of neurons needed per feature map. In vivo imaging methods (Andermann, Kerlin, Roumis, Glickfeld, & Reid, 2011; Bonin, Histed, Yurgenson, & Reid, 2011; Ikezoe, Mori, Kitamura, Tamura, & Fujita, 2013; Nauhaus et al., 2012; Ohki et al., 2005; Roth, Helmchen, & Kampa, 2012; Smith & Häusser, 2010) seem the most promising avenue to yield solid quantitative descriptions of the features and their parameters represented within both developing and mature extrastriate cortical areas.
Thanks to Bosco Tjan, Irv Biederman, and the anonymous reviewers for many helpful comments.
Address: Neuroscience Graduate Program, University of Southern California, Los Angeles, CA, USA.
Andermann, M. L., Kerlin A. M., Roumis D. K., Glickfeld L. L., Reid R. C. (2011). Functional specialization of mouse higher visual cortical areas. Neuron, 72 (6), 1025–1039, doi:10.1016/j.neuron.2011.11.013.
Antolík J., Bednar J. A. (2011). Development of maps of simple and complex cells in the primary visual cortex. Frontiers in Computational Neuroscience, 5, 17, doi:10.3389/fncom.2011.00017.
Anzai A., Peng X., Essen D. C. V. (2007). Neurons in monkey visual area V2 encode combinations of orientations. Nature Neuroscience, 10 (10), 1313–1321, doi:10.1038/nn1975.
Bell A. J., Sejnowski T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37 (23), 3327–3338.
Bonin V., Histed M. H., Yurgenson S., Reid R. C. (2011). Local diversity and fine-scale organization of receptive fields in mouse visual cortex. Journal of Neuroscience, 31 (50), 18506–18521, doi:10.1523/JNEUROSCI.2974-11.2011.
Bosking W. H., Crowley J. C., Fitzpatrick D. (2002). Spatial coding of position and orientation in primary visual cortex. Nature Neuroscience, 5 (9), 874–882, doi:10.1038/nn908.
Cadieu C., Kouh M., Pasupathy A., Connor C. E., Riesenhuber M., Poggio T. (2007). A model of V4 shape selectivity and invariance. Journal of Neurophysiology, 98 (3), 1733–1750.
Cang J., Feldheim D. A. (2013). Developmental mechanisms of topographic map formation and alignment. Annual Review of Neuroscience, 36, 51–77.
Carpenter G. A., Grossberg S. (2010). Adaptive resonance theory. In Arbib M. A. (Ed.) The Handbook of Brain Theory and Neural Networks, 2nd Ed ( pp. 87–90). Cambridge, MA: MIT Press.
Carreira-Perpinán, M. A., Goodhill G. J. (2004). Influence of lateral connections on the structure of cortical maps. Journal of Neurophysiology, 92 (5), 2947–2959.
Chen X., Han F., Poo M., Dan Y. (2007). Excitatory and suppressive receptive field subunits in awake monkey primary visual cortex (V1). Proceedings of the National Academy of Sciences, USA, 104 (48), 19120–19125, doi:10.1073/pnas.0706938104.
Chklovskii D. B., Koulakov A. A. (2004). Maps in the brain: What can we learn from them? Annual Review of Neuroscience, 27 (1), 369–392, doi:10.1146/annurev.neuro.27.070203.144226.
Desimone R., Gross C. G. (1979). Visual areas in the temporal cortex of the macaque. Brain Research, 178 (2), 363–380.
DiCarlo J. J., Zoccolan D., Rust N. C. (2012). How does the brain solve visual object recognition? Neuron, 73 (3), 415–434, doi:10.1016/j.neuron.2012.01.010.
Durbin R., Mitchison G. (1990). A dimension reduction framework for understanding cortical maps. Nature, 343 (6259), 644–647.
Elston G. N., Rosa M. G. (1998). Morphological variation of layer III pyramidal neurones in the occipitotemporal pathway of the macaque monkey visual cortex. Cerebral Cortex, 8 (3), 278–294, doi:10.1093/cercor/8.3.278.
Espinosa J. S., Stryker M. P. (2012). Development and plasticity of the primary visual cortex. Neuron, 75 (2), 230–249.
Felleman D. J., Essen D. C. V. (1991). Distributed hierarchical processing in the primate. Cerebral Cortex, 1 (1), 1–47, doi:10.1093/cercor/1.1.1.
Földiák P. (1991). Learning invariance from transformation sequences. Neural Computation, 3 (2), 194–200, doi:10.1162/neco.1991.3.2.194.
Fujita I., Tanaka K., Ito M., Cheng K. (1992). Columns for visual features of objects in monkey inferotemporal cortex. Nature, 360 (6402), 343–346.
Fukushima K. (1980). Neocognitron: A self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36 (4), 193–202.
Gallant J. L., Braun J., Van Essen D. C. (1993). Selectivity for polar, hyperbolic, and cartesian gratings in macaque visual cortex. Science, 259 (5091), 100–103, doi:10.1126/science.8418487.
Gilbert C. D. (1977). Laminar differences in receptive field properties of cells in cat primary visual cortex. Journal of Physiology, 268 (2), 391–421.
Gochin P. M., Miller E. K., Gross C. G., Gerstein G. L. (1991). Functional interactions among neurons in inferior temporal cortex of the awake macaque. Experimental Brain Research, 84 (3), 505–516.
Goodhill G. J. (1993). Topography and ocular dominance: A model exploring positive correlations. Biological Cybernetics, 69 (2), 109–118.
Goodhill G. J. (2007). Contributions of theoretical modeling to the understanding of neural map development. Neuron, 56 (2), 301–311, doi:10.1016/j.neuron.2007.09.027.
Hansel D., van Vreeswijk C. (2012). The mechanism of orientation selectivity in primary visual cortex without a functional map. Journal of Neuroscience, 32 (12), 4049–4064, doi:10.1523/JNEUROSCI.6284-11.2012.
Hooser S. D. V. (2007). Similarity and diversity in visual cortex: Is there a unifying theory of cortical computation? Neuroscientist, 13 (6), 639–656, doi.org/10.1177/1073858407306597.
Hubel D. H., Wiesel T. N. (1968). Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology, 195 (1), 215–243.
Hübener M., Shoham D., Grinvald A., Bonhoeffer T. (1997). Spatial relationships among three columnar systems in cat area 17. Journal of Neuroscience, 17 (23), 9270–9284.
Hyvärinen A., Hoyer P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41 (18), 2413–2423.
Ikezoe K., Mori Y., Kitamura K., Tamura H., Fujita I. (2013). Relationship between the local structure of orientation map and the strength of orientation tuning of neurons in monkey V1: A 2-photon calcium imaging study. Journal of Neuroscience, 33 (42), 16818–16827.
Kaas J. H. (1997). Topographic maps are fundamental to sensory processing. Brain Research Bulletin, 44 (2), 107–112.
Karnani M. M., Agetsuma M., Yuste R. (2014). A blanket of inhibition: Functional inferences from dense inhibitory connectivity. Current Opinion in Neurobiology, 26, 96–102, doi:10.1016/j.conb.2013.12.015.
Kaschube M. (2014). Neural maps versus salt-and-pepper organization in visual cortex. Current Opinion in Neurobiology, 24, 95–102, doi:10.1016/j.conb.2013.08.017.
Kaschube M., Schnabel M., Löwel S., Coppola D. M., White L. E., Wolf F. (2010). Universality in the evolution of orientation columns in the visual cortex. Science, 330 (6007), 1113–1116, doi:10.1126/science.1194869.
Keil W., Wolf F. (2011). Coverage, continuity, and visual cortical architecture. Neural Systems & Circuits, 1 (1), 1–56, doi:10.1186/2042-1001-1-17.
Kobatake E., Tanaka K. (1994). Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. Journal of Neurophysiology, 71 (3), 856–867.
Kohonen T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43 (1), 59–69, doi:10.1007/BF00337288.
Kohonen T. (1990). The self-organizing map. Proceedings of the IEEE, 78 (9), 1464–1480, doi:10.1109/5.58325.
Koulakov A. A., Chklovskii D. B. (2001). Orientation preference patterns in mammalian visual cortex: A wire length minimization approach. Neuron, 29 (2), 519–527, doi:10.1016/S0896-6273(01)00223-9.
Le Cun B. B., Denker J. S., Henderson D., Howard R. E., Hubbard W., Jackel L. D. (1990). Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems ( pp. 396–404). Burlington, MA: Morgan Kaufmann.
Lennie P. (2003). Receptive fields. Current Biology, 13 (6), R216–R219.
Mel B. W. (1997). SEEMORE: Combining color, shape, and texture histogramming in a neurally inspired approach to visual object recognition. Neural Computation, 9 (4), 777–804, doi:10.1162/neco.1997.9.4.777.
Mel B. W., Fiser J. (2000). Minimizing binding errors using learned conjunctive features. Neural Computation, 12 (4), 731–762, doi:10.1162/089976600300015574.
Miller K. D. (1994). A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between ON-and OFF-center inputs. Journal of Neuroscience, 14, 409–409.
Mitchison G. J., Swindale N. V. (1999). Can hebbian volume learning explain discontinuities in cortical maps? Neural Computation, 11 (7), 1519–1526, doi:10.1162/089976699300016115.
Nandy A. S., Sharpee T. O., Reynolds J. H., Mitchell J. F. (2013). The fine structure of shape tuning in area V4. Neuron, 78 (6), 1102–1115, doi:10.1016/j.neuron.2013.04.016.
Obermayer K., Blasdel G. G., Schulten K. (1992). Statistical-mechanical analysis of self-organization and pattern formation during the development of visual maps. Physical Review A, 45 (10), 7568–7589, doi:10.1103/PhysRevA.45.7568.
Obermayer K., Ritter H., Schulten K. (1990). A principle for the formation of the spatial structure of cortical feature maps. Proceedings of the National Academy of Sciences, 87 (21), 8345–8349.
Ohki K., Chung S., Ch'ng Y. H., Kara P., Reid R. C. (2005). Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex. Nature, 433 (7026), 597–603, doi:10.1038/nature03274.
O'Kusky J., Colonnier M. (1982). A laminar analysis of the number of neurons, glia, and synapses in the adult cortex (area 17) of adult macaque monkeys. Journal of Comparative Neurology, 210 (3), 278–290, doi:10.1002/cne.902100307.
Olshausen B. A., Field D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37 (23), 3311–3325, doi:10.1016/S0042-6989(97)00169-7.
Op de Beeck H. P., DiCarlo J. J., Goense J. B. M., Grill-Spector K., Papnastassiou A., Tanifuji M., Tsao D. Y. (2008). Fine-scale spatial organization of face and object selectivity in the temporal lobe: Do functional magnetic resonance imaging, optical imaging, and electrophysiology agree? Journal of Neuroscience, 28 (46), 11796–11801, doi:10.1523/JNEUROSCI.3799-08.2008.
Paik S.-B., Ringach D. L. (2012). Link between orientation and retinotopic maps in primary visual cortex. Proceedings of the National Academy of Sciences, 109 (18), 7091–7096, doi:10.1073/pnas.1118926109.
Pasupathy A., Connor C. E. (1999). Responses to contour features in macaque area V4. Journal of Neurophysiology, 82 (5), 2490–2502.
Pasupathy A., Connor C. E. (2001). Shape representation in area V4: Position-specific tuning for boundary conformation. Journal of Neurophysiology, 86 (5), 2505–2519.
Pfeffer C. K., Xue M., He M., Huang Z. J., Scanziani M. (2013). Inhibition of inhibition in visual cortex: The logic of connections between molecularly distinct interneurons. Nature Neuroscience, 16 (8), 1068–1076.
Ranzato M., Huang F. J., Boureau Y.-L., LeCun Y. (2007). Unsupervised learning of invariant feature hierarchies with applications to object recognition. In IEEE 2007 Conference on Computer Vision and Pattern Recognition (pp. 1–8). Minneapolis, MN: IEEE.
Rathelot J.-A., Strick P. L. (2006). Muscle representation in the macaque motor cortex: An anatomical perspective. Proceedings of the National Academy of Sciences, 103 (21), 8257–8262, doi:10.1073/pnas.0602933103.
Riesenhuber M., Poggio T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2 (11), 1019–1025, doi:10.1038/14819.
Roth M. M., Helmchen F., Kampa B. M. (2012). Distinct functional properties of primary and posteromedial visual area of mouse neocortex. Journal of Neuroscience, 32 (28), 9716–9726.
Rothschild G., Mizrahi A. (2015). Global order and local disorder in brain maps. Annual Review of Neuroscience, 38, 247–268, doi:10.1146/annurev-neuro-071013-014038.
Rothschild G., Nelken I., Mizrahi A. (2010). Functional organization and population dynamics in the mouse primary auditory cortex. Nature Neuroscience, 13 (3), 353–360, doi:10.1038/nn.2484.
Rumelhart D. E., Zipser D. (1985). Feature discovery by competitive learning. Cognitive Science, 9 (1), 75–112, doi:10.1016/S0364-0213(85)80010-0.
Rust N. C., DiCarlo J. J. (2010). Selectivity and tolerance (“invariance”) both increase as visual information propagates from cortical area V4 to IT. Journal of Neuroscience, 30 (39), 12978–12995, doi:10.1523/JNEUROSCI.0179-10.2010.
Sato T., Uchida G., Tanifuji M. (2009). Cortical columnar organization is reconsidered in inferior temporal cortex. Cerebral Cortex, 19 (8), 1870–1888, doi:10.1093/cercor/bhn218.
Serre T., Wolf L., Poggio T. (2005). Object recognition with features inspired by visual cortex. In IEEE 2005 Computer Society Conference on Computer Vision and Pattern Recognition ( Vol. 2, pp. 994–1000). San Diego, CA: IEEE.
Sharpee T. O., Kouh M., Reynolds J. H. (2013). Trade-off between curvature tuning and position invariance in visual area V4. Proceedings of the National Academy of Sciences, USA, 110 (28), 11618–11623, doi:10.1073/pnas.1217479110.
Smith S. L., Häusser M. (2010). Parallel processing of visual space by neighboring neurons in mouse visual cortex. Nature Neuroscience, 13 (9), 1144–1149, doi:10.1038/nn.2620.
Swindale N. V. (1982). A model for the formation of orientation columns. Proceedings of the Royal Society of London, Series B: Biological Sciences, 215 (1199), 211–230, doi:10.1098/rspb.1982.0038.
Swindale N. V. (2000). How many maps are there in visual cortex? Cerebral Cortex, 10 (7), 633–643, doi:10.1093/cercor/10.7.633.
Swindale N. V., Bauer H. (1998). Application of Kohonen's self-organizing feature map algorithm to cortical maps of orientation and direction preference. Proceedings of the Royal Society of London B: Biological Sciences, 265 (1398), 827–838.
Swindale N. V., Shoham D., Grinvald A., Bonhoeffer T., Hübener M. (2000). Visual cortex maps are optimized for uniform coverage. Nature Neuroscience, 3 (8), 822–826.
Tanaka K. (2003). Columns for complex visual object features in the inferotemporal cortex: Clustering of cells with similar but slightly different stimulus selectivities. Cerebral Cortex, 13 (1), 90–99.
Ullman S., Vidal-Naquet M., Sali E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience, 5 (7), 682–687, doi:10.1038/nn870.
Wallis G., Rolls E. T. (1997). Invariant face and object recognition in the visual system. Progress in Neurobiology, 51 (2), 167–194, doi:10.1016/S0301-0082(96)00054-8.
Wolf F., Bauer H.-U., Geisel T. (1994). Formation of field discontinuities and islands in visual cortical maps. Biological Cybernetics, 70 (6), 525–531, doi:10.1007/BF00198805.
Yamane Y., Tsunoda K., Matsumoto M., Phillips A. N., Tanifuji M. (2006). Representation of the spatial relationship among object parts by neurons in macaque inferotemporal cortex. Journal of Neurophysiology, 96 (6), 3147–3156, doi:10.1152/jn.01224.2005.
Yamins D. L., Hong H., Cadieu C., DiCarlo J. J. (2013). Hierarchical modular optimization of convolutional networks achieves representations similar to macaque IT and human ventral stream. In Advances in neural information processing systems 26 ( pp. 3093–3101). NIPS.
Yen S.-C., Baker J., Gray C. M. (2007). Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. Journal of Neurophysiology, 97 (2), 1326–1341, doi:10.1152/jn.00747.2006.
Yoshimura Y., Dantzker J. L. M., Callaway E. M. (2005). Excitatory cortical neurons form fine-scale functional networks. Nature, 433 (7028), 868–873, doi:10.1038/nature03252.
Yu H., Farley B. J., Jin D. Z., Sur M. (2005). The Coordinated Mapping of Visual Space and Response Features in Visual Cortex. Neuron, 47 (2), 267–280, doi.org/10.1016/j.neuron.2005.06.011.
Zoli M., Jansson A., Syková E., Agnati L. F., Fuxe K. (1999). Volume transmission in the CNS and its relevance for neuropsychopharmacology. Trends in Pharmacological Sciences, 20 (4), 142–150.

References: V. 
 V. 
 V. 
 V. 
 V. 
 V.