Note on Development of Modularity in Simple Cortical Models
Authors: John E. Moody Alex Chernajvsky
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The existence of modularity in the organization of nervous systems  (e.g. cortical columns and olfactory glomeruli) is well known. We  show that localized activity patterns in a layer of cells, collective  excitations, can induce the formation of modular structures in the  anatomical connections via a Hebbian learning mechanism. The  networks are spatially homogeneous before learning, but the spon(cid:173) taneous emergence of localized collective excitations and subse(cid:173) quently modularity in the connection patterns breaks translational  symmetry. This spontaneous symmetry breaking phenomenon is  similar to those which drive pattern formation in reaction-diffusion  systems. We have identified requirements on the patterns of lateral  connections and on the gains of internal units which are essential  for the development of modularity. These essential requirements  will most likely remain operative when more complicated (and bi(cid:173) ologically realistic) models are considered. 
1 Present Address: Molecular and Cellular Physiology, Beckman Center, Stanford University, 
Stanford, CA 94305. 
2 Please address correspondence to John Moody. 
134 
Chernjavsky and Moody 
1 Modularity in Nervous Systems 
Modular organization exists throughout the nervous system on many different spa(cid:173) tial scales. On the very small scale, synapses appear to be clustered on dendrites.  On the very large scale, the brain as a whole is composed of many anatomically  and functionally distinct regions. At intermediate scales, the scales of networks and  maps, the brain exhibits columnar structures. 
The purpose of this work is to suggest possible mechanisms for the development  of modular structures at the intermediate scales of networks and maps. The best  known modular structure at this scale is the column. Many modality- specific  variations of columnar organization are known, for example orientation selective  columns, ocular dominance columns, color sensitive blobs, somatosensory barrels,  and olfactory glomeruli. In addition to these anatomically well-established struc(cid:173) tures, other more speculative modular anatomical structures may exist. These  include the frontal eye fields of association cortex whose modular structure is in(cid:173) ferred only from electrophysiology and the hypothetical existence of minicolumns  and possibly neuronal groups. 
Although a complete biophysical picture of the development of modular structures  is still unavailable, it is well established that electrical activity is crucial for the  development of certain modular structures such as complex synaptic zones and oc(cid:173) ular dominance columns (see Kalil 1989 and references therein). It is also generally  conjectured that a Hebb-like mechanism is operative in this development. These  observations form a basis for our operating hypothesis described below. 
2 Operating Hypothesis and Modeling Approach  Our hypothesis in this work is that localized activity patterns in a layer of cells  induce the development of modular anatomical structure within the layer. We  further hypothesize that the emergence of localized activity patterns in a layer is  due to the properties of the intrinsic network dynamics and does not necessarily  depend upon the system receiving localized patterns of afferent activity. 
Our work therefore has two parts. First, we show that localized patterns of ac(cid:173) tivity on a preferred spatial scale, collective excitations, spontaneously emerge in  homogeneous networks with appropriate lateral connectivity and cellular response  properties when driven with arbitrary stimulus (see Moody 1990). Secondly, we  show that these collective excitations induce the formation of modular structures  in the connectivity patterns when coupled to a Hebbian learning mechanism. 
The emergence of collective excitations at a preferred spatial scale in a homogeneous  network breaks translational symmetry and is an example of spontaneous symmetry  breaking. The Hebbian learning freezes the modular structure into the anatomy.  The time scale of collective excitations is short, while the Hebbian learning process  occurs over a longer time scale. The spontaneous symmetry breaking mechanism is  similar to that which drives pattern formation in reaction-diffusion systems (Turing  1952, Meinhardt 1982). Reaction-diffusion models have been applied to pattern for-
Note on Development or Modularity in Simple Cortical Models 
135