Supplemental Materials for “Equalization of ocular dominancecolumns induced by an activity-dependent learning rule andthe maturation of inhibition”
Taro Toyoizumi and Kenneth D. MillerFebruary 5, 2009

Contents
1 Analysis of the semi-linear Hebbian model
1.1 Analytical Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Stability of contra-dominated, equalized, and open-eye-dominated patterns under the covariance learning rule . . . . . . . . . . . . . . . . . . . . . . . . . .

S1S1

2 Parameter Dependence of Model Results

S4

3 Comparison of our Model to That of Swindale (1980)

S4

1

S3Analysis of the semi-linear Hebbian model

In this section, we mathematically analyze the properties of the Hebbian learning rule ofEq. 6, and illustrate, with a simple semi-linear model (linear except for hard upper and lower
bounds on weights) the stability or instability of contra-dominated, equalized, and open-eyedominated patterns under normal conditions and under MD.

1.1

Analytical Framework

hIwI (x) (x) =be the vector of synaptic weights and h(x) =Let w
be the input
hCwC (x)vector. To begin with, the output ﬁring rate of a neuron at cortical position x is describedbywa (x)ha + dx M (x − x )r(x )r(x) =


a=C,I


=

 (x),dx K(x − x )hT w(S1)

11and M̃ is the Fourier transformation of the function describdkeikx 1−M̃
where K(x) = 2π(k)ing intracortical connectivity. Note that the half rectiﬁcation function in Eq. 1 is omitted here
so that the equationwill 
be linear and amenable to analysis. We assume that the input mean
1has the form h =and the input covariance matrix has the form



1c
T(S2)
(h − h)(h − h)  = qc with covariance strength q > 0, MD factor 0 ≤  ≤ 1 and between-eye correlation 0 ≤ c ≤ 1.
Here,  = 1 corresponds to normal development while smaller values of  model monoculardeprivation.
S1


1 −1
Setting S =, the average changes of synaptic weights according to the−1 1
Hebbian learning rule with subtractive normalization is described by12 (x) = α(r(x) − ρr̄(x))Sh∆w(S3)= α(δr(x) + (1 − ρ)r̄(x))Sh (x )
= α dx K(x − x )S (h − h)(h − h)T  + (1 − ρ)hhT w
along with the saturation condition of synapses, 0 ≤ wa (x) ≤ wmax . Equation S3 becomes (x) = α dx K(x − x )Aw(x),(S4)∆w
uv=where we have deﬁned the matrix A = S−u −v
with u = q(1 − c)/2 + (1 − ρ)(1 − )/2 and v = −q(1 − c)/2 + (1 − ρ)(1 − )/2. The eigenvalue decomposition of this matrix is written as A = PΛP−1 where the columns of P are theeigenvectors of A and Λ is a diagonal matrix whose diagonal entries are the corresponding
eigenvalues of A. We ﬁnd that1−v 1(S5)
P=u −1λ1and(h − h)(h − h)T  + (1 − ρ)hhT
Λ=0 00 λ1

,

(S6)

where λ1 = u − v. Then, if e0 and e1 are the two eigenvectors of P, we can decompose
 (x) = z0 (x)e0 + z1 (x)e1 where z0 (x) and z1 (x) are given byw




1 1
wI (x)
z0 (x)−1
 (x) == P w
.
(S7)
u v
z1 (x)wC (x)
We can then reexpress Eq. S4 in terms of the dynamics of z0 and z1 :∆z0 (x) = 0,∆z1 (x) = αλ1

dx K(x − x )z1 (x).

(S8)

Note that z0 (x) = wI (x) + wC (x) does not change in time because of the imposed subtractive
normalization constraint, i.e, the sum of contra-eye and ispi-eye synapses should be constant.
The saturation condition of synapses is now described by
0 ≤ −vz0 (x) + z1 (x) ≤ λ1 wmax ,0 ≤ uz0 (x) − z1 (x) ≤ λ1 wmax .
The learning dynamics of Eq. S8 has an energy function 1dxdx z1 (x)K(x − x )z1 (x ).
E=−2S2(S9)(S10)

The direction of ﬂow at z = (z0 (x), z1 (x)) is described by the negative gradient of E.
We assume K(x) is symmetric about zero, K(x) = K(−x), so that its Fourier transform
is real. Because K̃(k) = 1/(1 − M̃ (k)) and we assume we are working in a regime in which
cortical dynamics are stable, meaning M̃ (k) < 1 for all k, the Fourier spectrum of K is always
positive, i.e., K̃(k) > 0.
From the positivity of the Fourier spectrum of K, we can show that all the stable ﬁxed
points of the dynamics satisfy z1 (x) = zmax or z1 (x) = zmin , where all the components ofwa (x) are saturated. Note that zmax = uwmax , zmin = vwmax if v < 0 and zmin = 0 if v > 0.
To show this, we show there cannot be a stable ﬁxed point z ∗ = (z0∗ (x), z1∗ (x)) of the dynamics
that has a non-saturated component. First, if we perturb z around that stable ﬁxed stable
point by δz = (δz0 (x), δz1 (x)), the energy function changes by 
1∗dxdx δz1 (x)K(x − x )δz1 (x ),(S11)δE = − dxδz1 (x)v1 (x) −2
where the negative gradient of E at z ∗ is described by v1∗ (x) = dx K(x − x )z1∗ (x ). Because
the ﬁxed point is stable, any allowed perturbation (meaning any perturbation that does
not carry synapses past their weight limits) must yield a positive δE. The second term ofEq. S11 is always negative because the Fourier spectrum of K is all positive (analogous toa positive deﬁnite matrix in a discrete case). Therefore, the ﬁrst term must be positive forany allowed perturbation. However, if at least one component of z1 (x) has not saturated, wecan always choose a non-zero δz1 (x) to be perpendicular to v1∗ (x), i.e., δz1 (x)v1∗ (x) = 0,
by perturbing only the non-saturated component of z1 (x). The gradient of E at z ∗ must be
zero in the direction of any nonsaturated component of z ∗ , because otherwise z ∗ would not
be a ﬁxed point. This is contradictory because, then, the energy function is decreased by theperturbation δz, so z ∗ is not stable. Therefore, all the components of z1 (x) are either zmax orzmin at stable ﬁxed points.

1.2

Stability of contra-dominated, equalized, and open-eye-dominated patterns under the covariance learning rule

In the main text, we explained that the initially contralaterally-dominated pattern stayed intact under the covariance learning rule with subtractive normalization until cortical inhibition
matured (Fig. 2C). This contralaterally-dominated pattern was destabilized with the maturation of inhibition that initiates the CP and the equalized pattern became stable (Fig. 2D).
Subsequently, if the contra-eye was closed, an MD shift to the open-eye-dominated patternoccurred in the presence of large noise (Fig. 2E), but the equalized pattern remained stableagainst MD without the noise. Here, we explain these results using the linear analysis described in section 1.1. In particular, the maturation of inhibition changes K from a more
low-pass to a more band-pass ﬁlter (as shown in Fig. S1A), and we show that this destabilizes
the contralaterally-dominated pattern. We also show that closing the contra-eye after theequalization of OD columns does not cause an MD shift without noise, because the equalized
pattern remains stable.
From the analysis in the section 1.1, the dynamics of the covariance learning under subtractive normalization are described by Eq. S8, and the z1 (x) are equal to either zmax orzmin at stable ﬁxed points. Hence, in order for z1 (x) to be a stable ﬁxed point, its ﬂow,
∆z1 (x) ∝ K(x − x )z1 (x )dx , should be positive at x where z1 (x) = zmax and negativeS3

at x where z1 (x) = −zmin . We simulated Eq. (S8), starting from the same initial condition describedin Fig. 2B. In Figure S1B, each panel shows z1 (red lines) and the corre
sponding K(x − x )z1 (x )dx (blue lines) under a diﬀerent condition. The top-left panel
pattern of z1 before the CP and the corresponding ﬂow,
shows the contralaterally-dominatedK(x − x )z1 (x )dx (blue lines). Before the CP, K is essentially a low-pass ﬁlter but with adelta-peak at the center. Hence, ∆z1 (x) is a superposition of z1 (x) itself and its low-passedor smoothed version. We can see from the ﬁgure that this z1 is stable, having two peaksof K. The
inherited from z1 (x); the stability of z1 (x) is essentially due to the delta-peak
top-right panel shows the same pattern z1 , along with the new ﬂow K(x − x )z1 (x )dx
induced after the maturation of inhibition that initiates the CP. Now, K is more a band-passﬁlter, and this creates another positive peak of ∆z1 in between the two peaks of z1 , making
this pattern unstable. This shows how the maturation of inhibition destabilizes the initial,contralaterally-dominated condition.
The bottom-leftpanel shows the equalized pattern of z1 that forms during the CP and thecorresponding ﬂow K(x−x )z1 (x )dx . This equalized pattern is stable given the maturationof inhibition. Under the subsequent MD condition, zmax and zmin change their values becauseof the change in the closed-eye input strength, . The bottom-right panel shows the equalized
pattern of z1 after MD – the same pattern as in the lower left panel, except that the values ofzmax and zmin are changed to reﬂect the MD – along with the corresponding ﬂow. The ﬂow
is barely changed, so that the equalized pattern remains stable under MD. This small changeof ∆z1 (x) after MD can be understood from the fact that the change in z1 is an almost equalincrease across all x, that is, it represents the addition of a DC (spatially invariant) component;
and K has a very small DC component, K̃(0), due to the strong cortical inhibition. Eventhough z1 is uniformly pushed toward more positive or less negative values, almost half ofcortical areas still have negative ∆z1 (x), preventing the MD from destabilizing the equalizedcondition.

2Parameter Dependence of Model Results

As described in the main text, both the Hebbian learning rule with subtractive weight normalization (“Hebbian rule”) and the Hebbian learning rule with homeostatic regulation (“homeostatic rule”) explained the equalization of OD columns by the maturation of inhibition andthe OD shift under subsequent MD. However, the outcome of the Hebbian rule was moresensitive to intracortical connectivity and noise levels than the homeostatic rule. While thehomeostatic rule robustly shows equalization and an MD shift with parameter set 1 as wellas with parameter set 2, the Hebbian rule does not equalize with parameter set 2 (Fig. S2).
As shown in Fig.7, the depression of closed-eye synapses is conditioned on the monoculardeprivation factor fM D . The precise fM D value at which closed-eye depression behavior
switches to closed-eye homeostatic potentiation depends on the decay coeﬃcient γ for active
synapses; small γ makes the closed-eye-potentiating domain larger (Fig. S3)

3

Comparison of our Model to That of Swindale (1980)

As we discuss brieﬂy in the main text, Swindale (1980, Appendix A) also addressed theproblem of equalization and MD. Here we further discuss how these models diﬀer.

S4

B

10
before CP
during CPM

50−5−1−0.5

00.5

1z1 , 2∆z1 /(αλ1 )∆x2
before CP
during CPK

10−1−1−0.5

00.5z1 , 2∆z1 /(αλ1 )A12051000−10−101

10−5−1010x14052000−5−10−10x

1−20−1∆xFigure S1: (A) The function describing intracortical connections, M (Top), and the resulting
linear ﬁlter K (Bottom). The amount of cortical inhibition is increased
af the onset of the CP.
(B) Each panel shows the non-trivial eigen mode z1 (red) and K(x − x )z1 (x )dx (blue),
which up to a proportionality constant is the derivative ∆z1 . Stability requires that regions
where z1 is positive have positive derivative, and negative regions have negative derivative.
Top-left: The contralaterally-dominated pattern is stable before the maturation of inhibition.
Top-right: The contralaterally-dominated pattern becomes unstable after the maturationof inhibition. Bottom-left: The equalized pattern becomes stable after the maturation ofinhibition. Bottom-right: The equalized pattern remains stable under MD.
We deﬁne a variable wD (x) = wC (x)−wI (x) representing the diﬀerence between contralateraleye (wC ) and ipsilateral-eye (wI ) input to cortical position x. Equalization means that∞w̃D (0) ∝ −∞ dx wD (x) = 0 where w̃D (0) is the Fourier transform of wD (x) at frequency
0, or the “DC” pattern of wD . An ocular dominance shift toward the ipsi eye due to MD of
the contra eye will manifest as w̃D (0) < 0. We also deﬁne a variable wS (x) = wC (x) + wI (x).
Due to our normalization constraint, this is equal to a constant, wS (x) = cs for all x and alltimes. Swindale (1980, Appendix A) made a similar assumption.
The analysis of equalization and MD in Swindale (1980, Appendix A) considered a linear
model of the dynamics of wD . Such a model applies when the dynamics have a stable ﬁxed
point, and deviations from that ﬁxed point are suﬃciently small. A linear version of ourmodel means that we are ignoring the nonlinearity represented by the []+ in Eq. 1, andignoring nonlinearities associated with saturation of weights at maximal or minimal allowed
values. When treated as linear equations, both our model and the model of Swindale (1980,Appendix A) yield an equation for w̃D (0) of the formdw̃D (0) = P̃ D (0)w̃D (0) + Zdt(S12)

In (Swindale 1980, Appendix A), P̃ D (0) is the Fourier transform at zero frequency of P D (x) =P CC (x)+P II (x) where pXY (x) is a phenomonological interaction between a synapse from eye
X (contra or ipsi) and a synapse of eye Y separated across cortex by a distance x, and Z isP II (x) ((which is proportionalcs times the Fourier transform at zero frequency of P CC (x) −D
CC+ C II − (C IC + C CI ) K̃(0),to the integral of this quantity). In our model, P̃ (0) = CS5AB−1−0.5xx

100.5

0.5
102

−1−0.5123010.510012x 10

10−0.5xx

2−1−0.5

0.50.512step35x 100x 10−11035500

10.5
1012step

305x 10Figure S2: Development of OD columns under the homeostatic plasticity rule with the parameter set 1 (A) and under the pure Hebbian plasticity rule with the parameter set 2 (B).
Panels plot synaptic strengths in color as a function of cortical position, x, and simulatedtimestep as in Fig. 2A or Fig. 4A. While the homeostatic learning rule show similar behavior
for the two parameter sets, the pure Hebbian plasticity rule is sensitive to parameter choices
and does not equalizes with the parameter set 2.
where C XY describes the correlation between the activities of inputs serving eye X and eye
to the integral of the function M (x)Y and K̃(0) = (1 − M̃ (0))−1 where M̃ (0) is proportionaldescribing intracortical connectivity, and Z = cs C CC − C II + (C CI − C IC ) K̃(0). In bothmodels Z = 0 if the two eyes have equal activities, while if the ipsi (contra) eye is deprived
Z becomes negative (positive).
The models diﬀer in the sign of P̃ D (0). Swindale assumed that this was negative. Thenw̃D (0) = Z/|P̃ D (0)| is a stable ﬁxed point. In our model, P̃ D (0) is always positive: increasing
the strength of inhibition makes M̃ (0) more and more negative but this makes P̃ D (0) smallerand smaller while keeping it positive. (Note that stability of the activity dynamics requiresthat M̃ (0) < 1, and in turn is required for the expression of the growth rate in terms ofK̃(0) to be valid, so K̃(0) can never be negative). The result is that w̃D (0) = −Z/P̃ D (0)is an unstable ﬁxed point, and a linear model cannot be suﬃcient to describe the dynamics.
Equalization occurs because a periodic pattern of wD grows suﬃciently faster than the DC
pattern in the linear regime that, in the nonlinear regime in which synapses are saturating,the DC pattern is suppressed.
In practice, the key diﬀerences between the models involve recovery from saturated initialconditions. In our model, both equalization and monocular deprivation begin from a condition in which most synapses are saturated, and as a result it can be diﬃcult for the eyewhose representation should expand (the open eye under monocular deprivation; the weaker
ipsilateral eye in the case of equalization) to activate the cells that they would have to takeover to expand. For example, in the case of monocular deprivation, as we discuss in the text,
the saturated condition when MD is initiated means that the open-eye synapses are both near
their maximal possible value in open-eye-dominated regions and near their minimal possible
value in closed-eye-dominated regions. This, along with mature inhibition, means that stimulation of the open eye largely or only inhibits the cells dominated by the closed eye. Asa result, the open eye has diﬃculty taking over the cells initially dominated by the closed
eye without an additional mechanism, e.g. homeostatic plasticity, to allow weak synapses toS6

1.3
contraipsi

1.2

mean(w)1.1
10.90.80.7
0.6
0.5
0.4
00.2

0.4

0.6

0.8

1fM DFigure S3: The precise fM D value that yields switching from closed-eye depression toclosed-eye homeostatic potentiation during MD depends on the decay coeﬃcient γ for active
synapses. With γ = 5Hz for ha > 1Hz, the switching behavior occurs at around fM D = 0.3(c.f. Fig. 7).
recover and compete. Similar considerations apply to the case of equalization. In contrast, ina linear model, at the stable ﬁxed point preceeding initiation of MD, ocular dominance varies
in a sinusoidal rather than square-wave pattern. Thus, the open eye can activate cells in theclosed-eye dominated regions near the borders of the regions, allowing the open eye to takeover these cells and expand its representation under MD.

S7