Biological Learning of Irreducible Representations of Commuting Transformations
Authors: Alexander Genkin
David Lipshutz
Siavash Golkar
Tiberiu Tesileanu
Dmitri Chklovskii
Venue: NeurIPS 2022 Accept (2022)
Keywords: learning
transformation
biologically plausible

Abstract: A longstanding challenge in neuroscience is to understand neural mechanisms underlying the brain’s remarkable ability to learn and detect transformations of objects due to motion. Translations and rotations of images can be viewed as orthogonal transformations in the space of pixel intensity vectors. Every orthogonal transformation can be decomposed into rotations within irreducible two-dimensional subspaces (or representations). For sets of commuting transformations, known as toroidal groups, Cohen and Welling proposed a mathematical framework for learning the irreducible representations. We explore the possibility that the brain also learns irreducible representations using a biologically plausible learning mechanism. The first is based on SVD of the anti-symmetrized outer product of the vectors representing consecutive images and is implemented by a single-layer neural network. The second is based on PCA of the difference between consecutive frames and is implemented in a two-layer network but with greater biological plausibility. Both networks learn image rotations (replicating Cohen and Welling’s results) as well as  translations. It would be interesting to search for the proposed networks in nascent connectomics and physiology datasets.

Biological Learning of Irreducible Representations of Commuting Transformations
Alexander Genkin'
David Lipshutz
Siavash Golkar'
Tiberiu Teşileanu
Dmitri B. Chklovskir
Neuroscience Institute, NYU Langone Medical School
Centerr for Computational Neuroscience, Flatiron Institute
alexander alexander. www.anderr.eennngggga...ooom
sgolaar, htt.laa.....aaaaa.....aaaaaaaaar
Abstract
A longstanding challenge in neuroscience to understand neural mechanisms un- derlying the brain's ' remarkable ability to learn and detect transformations of objects due to motion. Translations and rotations of images can be viewed as orthogonal transformations in the space of pixel intensity vectors. Every orthogonal trans- formation can be decomposed into rotations within irreducible -ro-dimensional subspaces (or representations). For sets of commuting transformations, known as toroidal groups, Cohen and Welling proposed a mathematical framework for learning the irreducible representations. We explore the possibility that the brain also learns irreducible representations using a biologically plausible learning mech- anism. The first based SVD of the anti-symmetrized outer product of the vectors representing consecutive images and is implemented by single-layer neural network. The second is based on PCA of the difference between consecutive frames and is implemented in a two-layer network but with greater biological plausibility. Both networks learn image rotations (replicating Cohen and Welling's s results) as well as translations. It would be interesting to search for the proposed networks in nascent connectomics and physiology datasets.
1 Introduction
The brains of humans and other animals identify objects, such as human faces, regardless of their location and orientation in the visual field. This requires learning prenettypressrving transformations such as translations, scaling and rotations. Guiding locomotion requires identifying the direction and speed of the optic flow. Flies, to maintain stable flight, detect relative translational and rotational motion of their surroundings.
Whereas real-life transformations can be a mixture of translations and rotations, for simplicity, we consider learning one kind of transformation and after transformation are streamed to an unsupervised algorithm. The algorithm learns these transformations and detects their magnitude, which corresponds to the speed times the time step.
time. A sequence of pairs of images before
Transformations such as rotations and translations any magnitude can be decomposed into infinites- imal transformation operators (called generators) and magnitudes. For these so called Lie groups generators can be learned from data and magnitudes of transformation determined for each pair of images, shown by Rao and Ruderman [14].
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Many transformations can be represented as rotations in the high-dimensional space of vectorized images. Such rotations can be decomposed into a set of commuting 2D rotations, each characterized by generator and a magnitude, and comprise toroidal groups. Learning toroidal groups reduces to finding an "irreducible representation" problem solved by Bethge et al. [3] (for fixed magnitudes) and Cohen and Welling [5] (for varying magnitudes).
An important problem in neuroscience to develop biologically plausible algorithms and neural circuits that learn to detect transformations and their magnitudes. By biological plausibility we understand the following requirements [4]: (i) the algorithm operates in the online setting where pairs of images are streamed sequentially while the transformations are learned and the magnitudes are determined in real time: (ii) previously seen data cannot be stored by the algorithm other than a highly compressed representation in the synaptic weights; (iii) synaptic weight update (learning) rules are local, meaning that the weight update a particular synapse depends only on the activity of the two neurons it connects.
The generator approach of [14] presents difficulties for biologically plausible neural circuit imple- mentations. Specifically, detecting transformation magnitude requires multiplying all possible pairs of pixels taken from the images before and after the transformation weighted by the generator. The difficulty was overcome in two biologically plausible neural circuits proposed in [1].
In this paper we propose two algorithms for learning irreducible representations of toroidal groups that exhibit greater biological plausibility than prior work. We construct biologically plausible imple- mentations that build upon the work from Cohen and Welling [5], which did not address biological plausibility. In contrast to the generator-base -based approach [14; 1J. using irreducible representations eliminates the need for the multiplication of inputs. The neural circuits propose can be searched for in the nascent connectomics and physiology datasets.
The first proposal is based on computing an average of anti-symmetrized outer products of vectorized images (bivectors, as [3]) followed by a singular value decomposition (SVD) to yield irreducible representations. To perform SVD online, adapt algorithm for the SVD of coss-covvri matrix. When only the top irreducible component is desired, we show how to implement this algorithm using biologically plausible model of a single neuron. For multiple irreducible representations the algorithm relies on deflation. The neural network implementation of the deflation step violates requirement (iii) of biological plausibility above, which motivates another approach.
Our second proposal is based on the PCA of time differences of vectorized images. The corresponding neural network relies solely on local learning rules, which makes it fully biologically plausible by our definition. The resulting two-layer network decomposes the transformations into their irreducible 2D rotations in the first layer, and outputs the angle of each 2D rotation in the second layer.
2 Setup
Consider a stream of consecutive video frames (Fig. 1) vectorized into d-dimensional pixel- intensity vectors [x:} satisfying linear relations
(1) We focus here on image transformations that maintain the norm of x, approximately constant. These include translations, rotations, and more generally, local deformations, which shuffle pix- els around without affecting their values signifi- cantly. This assumption implics that the transfor- mation matrix A, is orthogonal. In this case, can decompose A, into independent rotations within orthogonal 2D subspaces, follows
X = A,
24-1
Figure 1: Consecutive video frames with rotated
images are streamed to the learning algorithms.
A, T, r. = blockdiag (R(e)). ((()).....(((()))
R(O
R(0) := cos 0
sin 0
(2)
sin 0
cos 0
Here Q a d orthogonal matrix, F, is d x block block-diagonal matrix whose blocks are 2 x rotation matrices indexed by 0 i I..... d/2, and we have assumed, for simplicity, that d is even.

---

Pairs of consecutive column vectors (q.21-1, 91.2x) of Q define orthogonal 2D subspaces and 0; represents the angle of rotation within that 2D subspace. As desired, this canonically separates the transformation identity (the 2D subspace) from the velocity (the angle of rotation).
Following Cohen and Welling [5], consider the problem of learning a representation for the d //2-dimensional commutative subgroup of rotations of the form A Qr,Q' where Q is a fixed orthogonal matrix and T is any block-diagonal matrix of the form in equation (2). The subgroup is commonly referred to as a torvidal subgroup because it is parameterized by the d/2-dimensional torus {(0), ...... 04/2)= 0c E [0. 2 With this simplification, the d/2 mutually orthogonal 2D subspaces constitute an irreducible representation of the subgroup [5].
The focus of this work is to derive a neurally plausible algorithm for (A) learning the irreducible representation from stream of pairs of vectorized video frames {(x4-1, xx)), and (B) outputting the
angle of rotation 0 within the top k 2D subspaces, for some < S d/2.
3 Learning the irreducible representation from data
We consider learning the irreducible representations from data in the offline setting using two approaches. The first approach based on an SVD of the average bivector between consecutive frames. The second approach is based on PCA of the covariance of the difference between consecutive frames. The analysis here will be the starting point for the derivations of the online algorithms.
We assume the sequence {x,} } is centered and whitened so that (x,x,) ){ = Lq. Whitening of sensory inputs is hypothesized to be function of early sensory systems [2] so we believe it is reasonable to assume that the circuit receives pre-whitened inputs. Furthermore, a number of biologically plausible algorithms for input whitening have been proposed and experimentally observed in some cases [6; 11; 16].
In addition, assume that A, is independent of x₁ [14; 3]. which holds approximately in many situations. For instance, most objects can move either left, right or up, down, even though the exact probabilities of each kind of transformation might depend weakly object identity.
3.1 Learning on SVD of bivectors of consecutive frames
Given pair of consecutive frames (x, 1, x,), define the bivector
(3) where we have used the relation x; A,x The bivector is an anti-symmetric matrix, which is also commonly referred to the 'wedge product' or 'exterior product'. Under our assumptions that x, and A, are independent and {xr} is whitened (i.e., (xxx, ) I2), we have
B: :- x 1 A,x -
B = (B.): (A, AT): QGQ
where
GT blockdiag (GaddiiaagG........... (G....., Gk), G g
9i
9. == 2(sin 8)).
Note that gi can be zero if the distribution of Of symmetric about zero. Therefore, to use this approach, we assume that + 0 for i =1, l....,ke k.
To learn the column vectors of Q [q1 .q..... ... qu). it will be convenient to write the SVD of B in terms of Q To this end, we assume, without loss of generality, that g1 2 ... 2 94/2 2 0. Then we have the following lemma, whose proof is deferred to Supplement A. Lemma 1. The SVD of B can be written as B UEVT, where
Σ = diag(g) (01, 91 9d/2: 94/2), and (V2--1. V2.) = (q2x. - for
U: Q
V = [vi,..., val.
.......... d/2.
In other words, each pair of left- and right-singular vectors of B, (u, V:), spans one of the 2D mutually orthogonal subspaces. Therefore, to learn the irreducible representation, it suffices to learn pairs of left- and right-singular vectors of Moreover, the first pair of left- and right-singular vectors spans the same subspace as the second pair of left- and right-singular vectors, and same with the third and fourth pairs, and so on. Therefore sufficient to learn only subset such as all odd add-numbered pairs.
3

---

whose solution the pair of matrices of left- and right-singular vectors of B. To solve this online we 3.2 Learning on PCA of differences between consecutive frames
We now present an alternative method for learning the column vectors Q, by performing PCA on the covariance of the discrete time derivatives defined by x := that A, is independent of x, and the vectors x; are whitened, we have
Under our assumptions
(x,x)) 2 (A, + A, = Q (T, r Q = QŘQT,
(5)
where I diag(ni (1, ?l..... Yek 7h) and 7/ := 2(cos 0i)r. Importantly, the column vectors of Q are eigenvectors of (x+X; ) with associated eigenvalues "i and, assuming that 71
> 7d/2-
each ") corresponds to a unique 2D subspace. Therefore the transformations with the largest angles be found by performing PCA on (xx*, )- Note that under this approach, 7V 0 if and only if 0g identically zero.
It is worth noting that, under the whitening assumption Xt, performing PCA the discrete time sums X + X,-1 learns features that are 'temporally slow" [10], which is hypothesized to be useful for learning objects [17]. From this perspective, learning transformations can be viewed complement of learning objects.
Biological algorithms based on SVD of the average bivector
To compute the SVD of B, consider the following optimization problem:
max
Tr(U BV) s.t. -'U-VIN Id, U.V
(6)
adapt an algorithm for SVD of cross-covariance matrix [7].
4.1 Finding the subspace with largest rotation velocity
Start with optimization problem for finding the top pair of singular vectors:
max By s.t. Jull ||v|| =.
u.y
the online setting optimization is performed by projected stochastic gradient descent:
B,v
V B,u
I B,Vl''
|/ B,ul
where Th > 0 is the learning rate, B, is used as the current approximation of B, and we have used the anti-symmetric property B, B,
Normalization steps are hard to map onto a biological circuit, so following the idea from Oja's algorithm [13]: assuming small enough learning rates, update equations can be expanded using Taylor series in T)- In this case, the iteration takes the form:
u f- u + [B.v (u B.v)u),
(9a)
- v + ne B + (v B.u)v)
(9b)
convergence proof for this algorithm given Supplement B.
Since the matrix B is anti-symmetric, the singular vectors resulting from this optimization, u and v, are orthogonal, and define the 2D subspace with the largest average sine of the angle of rotation. Now each input vector X, can be projected on the 2D subspace, and we denote the projected coordinates as a, := u x,, b = v x,. Using the definition of B, from Eq. (3) and substituting a, b, into the objective yields:
u Bv - (B.).v (a.b, 0r i = (wx)
(10)
where 1/1 abi ar- This suggests two linear neurons with common input and weights and v generating output streams a, and b,, followed by a circuit known as a Hassenstein -ensenstein-Reichartt detector [15]; see Fig. 2(a). Now the angle of rotation O, can be recovered:
0, arcsin
V af
+ 12
a
+ 12_1

---

x²
u
HPF
LPF
HPF
LPF
C
3
(b)
Figure 2: Biologically plausible implementations of the SVD algorithm. (a) The Hassenstein- Reichardt circuit calculates bivectors of consecutive inputs. Neuron a (b) projects input vectors onto the synaptic weight vector (v). Then the delayed and non-delayed signals from a and b are multiplied and the products are ultimately subtracted, Eq. (10). (b) neuron learning the subspace with largest rotation velocity. Left (right) dendritic branches receive synapses projecting the input onto the vector u (v) and perform high- (low-) pass filtering (HPF and LPF in the figure) resulting in dendritic current (3). The two dendritic currents are divided in the soma.
4.2 Full dimension: finding multiple subspaces
Subsequent singular vector pairs can be found by deflation. Recall the identities for mymeetymmttiic
SVD: 12:-1 V2i; Ugi -2 02i-11
02: 9: in Lemma 1. We can thus rewrite SVD
using odd indexes only:
B
Σ Tou.v. = [(,v
Vyu 19411)/2 = E(ju, + +vyyJ)B
(11)
3:odd
food
Then u, V, for odd, which is all we need, can be found as solution of an optimization problem similar to Eq. (7), but for the "deflated" matrix:
maxu a.V
(I
Σ joodd.jec
(u,u) + -vvz)) Bv
s.t.
|
|v|
=1.
(12)
Algorithm 1: The SVD algorithm with deflation
for i = 1 to & do
u, V: +- random, |uu|| || [Initialize]
end for
for 1,2,... do
B. <- X,X,-1
x,-Ix,
D f- 0 (current deflation estimate)
for to & do
u; +- u, + +u
[B.v - u -
V. < v. m.l l.BBBu v.V, B,u, DB,u,
D D i v.v. (update deflation estimate)
a u, Xt: v.
- Vat2+082 Voi_ 2 2+
0: arcsin
end for end for
5

---

A straightforward deflation algorithm would find singular vector pairs one by one, corresponding decreasing singular values. The goal however is to find all of them parallel, an online and neurally plausible manner. On every time step, the algorithm will perform one step of the iterations from Eqns. (9a) and (9b) in a loop on sequentially deflated matrices, using current estimates of singular vectors. Using again B, as the current approximation of B, this yields Algorithm 1. Proof of convergence can found the Supplement B.
4.3 Biological SVD
The circuit in Fig. 2(a) above is not realizable inside a neuron because of the crossing pathways. also cannot be realized with two neurons, as the original Hassenstein-Reichaal detector, because then the synaptic weight updates in one neuron would require information only available in the other neuron, violating the principle of locality. To overcome these difficulties we propose modification of the bivector expression:
B. = (x, - xx,1)) +x:-1)
(13)
This is similar to what was used in [1]. but with important difference: the expectation under assumptions is the same as Eq. (4) (though individual matrices are no longer anti-symmetric). Projected on the 2D subspace of interest are now the difference and sum of vectors: ax := u (x:
xt-1) and Be = v (x+ + xt- Substituting into the objective,
rewrite: u J'B.v 04ßt-
This suggests a neuron with two dendritic branches both connecting to the same of inputs, Fig. 2(b). The first has synapses with weights that also calculate temporal differences (perform low-pass filtering): the second one has synapses with weights that also perform high-pass filtering. The neuron performs division in the soma and outputs y/: ar. Br tan O., from which the angle can be computed. We assume the activity is suppressed when 3g is too small for reliable calculation of the ratio. That means the neuron has "blind spot" when (x, xx-1) is close to orthogonal to
The neural dynamics are described by the equations:
f- u (xi x(-1)
3. v [x(xxx=>> x(-1)
(14)
y/ + az BB/Br,
and synaptic plasticity rules are derived using a Taylor approximation in the learning rate, as in Eqs. (9b)-
11 f- u1 + 7/ (B:(x, xt-1) - (0.1))-,,..11))
(15)
v - v + The ar (x: + x:-1) + an3iv)
Biological algorithm based on PCA of time differences
In this section, derive two-layer network that outputs the angles 0¹ ...., 6² The first layer of our network projects the inputs x ..... , onto the top k 2D subspaces defined by the pairs of column vectors span(qr q1, 92)...., span )2x), and the second layer computes the angles of rotation within each 2D subspace.
5.1 Derivation of the first layer
Recall that for each i = 1...., k the vectors (q2i-1, qus) are eigenvectors the covariance matrix (x+x,) ) associated with eigenvector %. To project the inputs x; onto the 2D subspaces, we adopt the similarity matching objective introduced in [12]. We assume the first layer of the network has 2k neurons whose activities at time t are encoded in the vector y; E 2 Define the data matrices
X = xx..... xr). Y = [y....., Yr). X := [ xr] and Y = [ý1,.... Yr), where recall that the dot represents a discrete time derivative, e.g., X; =x, X Let X = QEP denote the
SVD of the data matrix. We start with the PCA objective function [12]
min -2 TT XY T + TT A ILY -Y)
(16)

---

Σ
W
y
y
y
2
M
A
Figure 3: A two-layer network for learning transformations using the PCA algorithm. In the first layer, inputs are projected onto multiple subspaces. In the second layer, the rotation angles ,,...., 0 computed according Algorithm 2.
where A = diag(A, A1 . dR. AR is any fixed diagonal matrix with A >> ... > A/. Suppose that
the diagonal elements "/s of the matrix I' (see Eq. (5)) obey 72 7k- Then the optimal solutions to (16) are given by
op ARQ X,
(17)
where R is any block diagonal matrix with 2 2 blocks that are orthogonal matrices, see [12, Lemma IJ. In particular, the optimal solution is the projection of the data matrix X onto the 2D subspaces defined by the pairs of column vectors of Q.
Directly optimizing (16) will not result in an online algorithm due to the sample covariance terms #XY and # YY! Therefore, to obtain an online algorithm, we encode the sample covariances in the matrices W E R2kxd and M E 222x2 by substituting in with the following Legendre transforms
T TXX I XY WERlExd
(2 TT T Wx) - T((WT)) T
min
1
2 TrYY MÝ) THAMTAM)) AMD
Tr(YYA- -Y -Ý)
min ME
7
to obtain
min YERUEX? WERdexd McR
min
max
L(W, Y).
(18)
where
4 L(W,M, Y) - - TrYY wX) - - +
MÝ) 2 THY THAM AM). (19)
After interchanging the order of optimization, we arrive at the objective
min max min
((WMM. Y).
(20)
We McR36x24 YERLExT
In the offline setting, we first optimize over Y and find that Ý M M or equivalently.
Y = M- W.. We then take a gradient descent-ascent step with respect to (W, M):
W+ Y
[ Y AMA
M+++7
(21)
T
In the network the matrix W plays the role of feedforward synaptic weights and the matrix M encodes the recurrent lateral synaptic weights. In the online setting, at each time step f, the network receives an input x, and computes the output y. by running the recurrent neural dynamics equilibrium:
dy) dy
Wx, My.(7)
M
Wx,
(22)
The network then computes y, y: V and the synaptic weights are updated according to the local learning rules
W W + 7()x W).
M M + - AMA).
(23)

---

5.2 Derivation of the second layer
The first layer of the network consists of 2k neurons and transforms the inputs so that the 2D subspaces are represented by pairs of neurons, The second layer of the network consists of neurons with each neuron computing an angle 8 The ith neuron of the second layer receives inputs (v--1"; : 2 1. from the (2i 1) and 21th neurons of the first layer, which satisfy the relation
21-17 1/
yr
=
cos 0f
sin 0j
sin 0j cos 0g
yr Vi-1
(24)
Therefore,
21 2:
Vr 2i
21-1
02 arctan
V/ 1 Vr
10
(25)
Vi- yi
+ Vi 196
which we define to be the output of the 7th neuron in the second layer. For each i 1, ...... k, the computation -²-1"y² y/- 2 can be implemented in circuit that resembles the Hassenstein- Reichardt detector, Figure 2(a).
Algorithm 2: The PCA algorithm
initialize W, M positive definite, Xo 0. 0
for 1,2 do
Yr M M- Wxx
W W + m((()) Y1-1 xt-1) -WW
M M ((; - Yt- ly: yt-1) -MMA)
for = I..... do
21-1 21 - 4 18
26-1
0j arctan
-1- % 28th +u 19
end for end for
Simulations
Simulations here are intended to demonstrate the following properties of our algorithms. The first property is their ability to learn arbitrary toroidal groups from data and estimate the speed of transformations. The second property is the ability to generate filters that match theoretical predictions. Finally, we verify that the network output is indeed informative of the transformation by performing reconstruction of the transformed image.
For an arbitrary toroidal group the algorithm must correctly recover 2D subspaces and estimate rotation angles. We created random toroidal groups with three 2D subspaces in 10-dimensional space. An orthogonal matrix Q defining the group was generated randomly, then rotations angles in each subspace were generated with independent normal distributions with means 0.3, 0.2, 0.1, and corresponding standard deviations 0.4, 0.3, 0.2. These rotations were applied vectors generated from standard normal distribution in 10 dimensions. In total, 10 samples and rotations were generated and given as input to both algorithms. Learning rates were manually selected to be 5- 10-1 for both algorithms. This was repeated 10 times with random initialization. Simulation results are shown in Fig. 4. Subspace fit loss is evaluated as one minus cosine the angle between the true and estimated planes. Angle estimates are depicted for the last 1000 iterations and pooled randomly from all 10 runs. This experiment took 14 minutes total on a MacBook Pro with 3.5 GHz Dual-Cor Intel Core i7 processor.
To study filters, i.e., pairs of vectors for each 2D subspace generated by our algorithms, we experiment with image rotations and translations. As opposed to rotations of abstract vectors above, here 2D subspace angles are correlated. We used natural images from the Van Hateren database [8] and randomly selected 106 patches of size 16 x 16 from image frames. The resulting images were then rotated each by random angles around the center using bi-linear approximation and ignoring pixels outside the central circle. The obtained image pairs were used as inputs to SVD algorithm, seeking
8

---

9/b
9/B
18
Subspoce ft bxss
Subspoce ftt toxs
10
Iterations
Iterations
(b)
(c)
(d)
Figure 4: Learning of toroidal group in vector space. (a) and (b): Convergence of the subspace loss number of iterations. Blended area shows the range between maxima and minima across all 10 runs; dense line shows mean across runs, median-filtere over 500 iterations. (c) and (d): Angle estimation for each subspace.
MAMMA
80
(a)
(b)
(C)
(d)
Figure 5: Filters learned by our algorithms. (a) Filters obtained by the SVD algorithm for rotations of 2D images; left (right) singular vectors are shown in the top (bottom) half. (b) Angular power spectrum each filter (a). (c) Filters obtained by the PCA algorithm for translations of 1D images. (d) The power spectrum of each quadrature pair from (c).
20 top pairs of filters. The resulting filter images are presented in Fig. 5(a): they are similar those obtained from theory in [3], also to those obtained from random images in [5].
Next provide quantitative evaluation of the obtained filters versus theoretical predictions. Ideally, in polar coordinates the angular part of Fourier spectrum would have sharp peak. The estimated angular parts of the spectrum for each filter are shown in Fig. 5(b). Details of this estimation and also additional experiments with rotation of images are presented in Supplement C.
Sub-pixel translations of 1D images image vector. We generated 25 x 105 pi-pi 1D images from the standard normal distribution. Top 4 filter pairs obtained by PCA algorithm shown in Fig. 5(b); they look like Fourier pairs, as expected, and power spectrum of each pair in Fig. 5(c) confirms that. Execution took 5 minutes.
be approximated by cyclic shifts, which rotations of the
We next demonstrate that the transformations learned by our network on one set of images can be used to transform another set of images. To this end, we use the data from the image rotation experiment above and, using multi-layer perceptron, fit a function that would predict transformed image X; given the source image X,-1 and the output θ, of Algorithm 1. This function is then applied to completely different images: digits from the MNIST dataset [9]. We compare digit images transformed this way directly rotating images with bilinear interpolation of pixel values (the ground truth), Figure 6. The details this experiment and additional results are presented Supplement D.
All code for these experiments is included in the Supplementary material.

---

Figure 6: Top row: digit rotations using the transformation learned by the SVD algorithm; Bottom row: digit transformations using image rotation (ground truth).
Contributions and limitations
The main contribution this work the derivation of two biologically plausible online algorithms that can learn toroidal groups of transformations, detect transformations and measure their velocity. By assuming an irreducible representation, as used in [3; 5], we show that a toroidal group can be learned using SVD of the average bivector between vectorized framee before and after transformation, or alternatively using PCA of differences between these vectorized frames. Direct computation of bivector covariance matrices would involve multiplication of inputs, which is biologically implausible. This is avoided here by representing and learning singular vectors and PCA weight vectors synaptic weights.
The SVD approach involves modification of previously known online algorithm for SVD of cross-covariance matrix, and for the case of the irreducible component we devise biologically plausible implementation. The approach based on PCA applies a previously developed algorithm for PCA to new context. The two-layer network that is obtained fully adheres to the principles of biological plausibility outlined [4]. The ability algorithms to learn subspaces, measure speed of transformations, generate filters approximating theoretical predictions, and produce output sufficiently informative to generate transformed images, validated by simulations.
Our work is limited in that we only consider transformations that can be seen rotations of vectors. This covers important cases of image rotations and sub-pixel shifts but omits several other types. Another limitation comes from the assumption that all transformations observed by our networks come from one commutative group (toroid). We intend to overcome these limitations in our continuing work.
Our work generates experimentally testable hypotheses that can be searched for in nascent connec- tomics and physiological datasets from various species and brain regions. The relation of our model actual neurobiological circuits in living organisms is only conjectural and there remain hurdles in turning our algorithms into biologically realistic models. However, our approach can potentially prove beneficial for neuromorphic hardware.
Acknowledgments
We grateful to Y. Bahroun and J. Moore for discussion and helpful suggestions, to P. Gunn for editing, and to L. Seading-Ikkkande for help with figures.
10

---

References
[1] Yanis Bahroun, Dmitri Chklovskii, and Anirvan Sengupta. A similarity- network trained on transformed images recapitulates salient features of the fly motion detection circuit. In Advances in neural information processing systems, pages 14201-14211 2019.
[2] HB Barlow and Foldiak. Adaptation and decorrelation in the cortex. miall c, durbin m,, mitchison g). eds. the computing neuron, 1989.
[3] Matthias Bethge, Sebastian Gerwinn, and Jakob H. Macke. Unsupervised learning of steerable basis for invariant image representations. Human Vision and Electronic Imaging XII. Edited by Rogowitz,
2007. doi: 10.1117/12.711119.
[4] Dmitri Chklovskii. The search for biologically plausible neural computation: The conventional approach, 2016. URL htttp://wwx http///ww..fooooeeeeeooooooooett httg/20///11////MM//a//////
[5] Taco Cohen and Max Welling. Learning the irreducible representations of commutative lie groups. In International Conference Machine Learning, pages 1755-1763. PMLR, 2014.
[6] Yang Dan, Joseph J Atick, and R Clay Reid. Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. Journal of neuroscience, 16 (10):3351-3362, 1996.
[7] Konstantinos I Diamantaras and Sun- Yuan Kung. Cross-correlation neural network models. IEFE Transactions on Signal Processing, 42(11):3218-33 3223, 1994.
[8] H. van Hateren and A. van der Schaaf. Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings: Biological Sciences, 265 (1394):359-366, Mar 1998.
[9] Y LeCun, Cortes, and Burges. The mnist dataset of handwritten digits (images). NYU: New York, NY, USA, 1999.
[10] David Lipshutz, Charles Windolf, Siavash Golkar, and Dmitri Chklovskii. biologically plausible neural network for slow feature analysis. Advances in neural information processing systems, 33: 33:14986-14996, 2020.
[11] David Lipshutz, Cengiz Pehlevan, and Dmitri Chklovskii. Interncurons accelerate learning dynamics in recurrent neural networks for statistical whitening, arxiv preprint arxiv:2209.10634 2022.
[12] Victor Minden, Cengiz Pehlevan, and Dmitri B Chklovskii. Biologically plausible online principal component analysis without recurrent neural dynamics. In 2018 52nd Asilomar Conference on Signals, Systems, and Computers, pages 104-111. IEEE, 2018
[13] Erkki Oja. Simplified neuron model as a principal component analyzer. Journal of Mathematical Biology, 15:267-273, 1982.
[14] Rajesh P. N. Rao and Daniel L. Ruderman. Learning lie groups for invariant visual perception.
In Advances in neural information processing systems, pages 810-816. 1999.
[15] Werner Reichardt. Autocorrelation, a principle for evaluation of sensory information by the central nervous system. In Symposium on Principles of Sensory Communication 1959, pages 303-317. MIT press, 1961.
[16] Adrian A Wanner and Rainer W Friedrich. Whitening of odor representations by the wiring diagram of the olfactory bulb. Nature neuroscience, 23(3):433-444 2020.
[17] Laurenz Wiskott and Terrence Sejnowski. Slow Feature Analysis: Unsupervised learning of invariance. Neural Computation, 14(4):715-777 2002. doi: 10. 10.1162089977602311118888

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] See Supplement
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes] See Section Simulations
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See Section Simulations
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section Simulations
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'le rou're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
12


Reviews:
[{"review_id": "ygpIDpFsm8k", "reviewer": "Reviewer_pueb", "summary": "The authors depict two novel biologically plausible algorithms for online estimation of transformations using irreducible representations. By building on prior work, existing algorithms making use of commutative Lie groups for decomposing transformations are altered for biological plausibility. Possible neural implementations are depicted which can be searched for in connectomics. The results are evaluated experimentally on two small examples.", "questions": "How do the non-biologically plausible SVD-based algorithm and the altered version compare in terms of accuracy?", "limitations": "Limitations, such as biological implausibility of the original SVD based algorithm, are stated clearly.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Originality: The general idea of representing Rotations by toroidal groups is not new but nonetheless interesting and coming up with actual biologically plausible implementations is invaluable.\n- The text is accessible: starting with a general introduction and good explanation of the problem at hand. \n- All Proofs including proof of convergence for \"cross-power iteration\" are readily available in the appendix\n\n\nWeaknesses:\n- Not clear which parts are contributions, i.e. labeled as \"Theorem\"\n- Sadly no code provided\n- Quantitative comparison of the experimental results with \"biologically implausible\" counterparts (i.e. Cohen and Welling) would've been interesting\n\nSuspected errors:\n- Line 73: canonical (ly)\n- Line 227: canter", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "qrGsgQnB3cj", "reviewer": "Reviewer_nuRj", "summary": "The present paper studied the neural implementation of learning irreducible representation of commuting group transformations. Specifically, using the 2d rotation group as an example, the paper studied how the neural network learns the rotation groups by implementing different algorithms, including singular value decomposition and principal component analysis, and different algorithms eventually lead to different network architecture.", "questions": "- The Introduction (lines 44-45) mentions that an advantage of learning irreducible representation solves the need for multiplicative units and only a linear number of synaptic connections will be needed per neuron. However later in the Result section, I don’t see a detailed explanation of how this problem is solved in the proposed algorithms. Some elaborations are needed.\n\n- Both models (Fig. 2 and Fig. 3) explicitly output the rotation angles $\\theta$. In contrast, there are solid neuroscience studies that the angle is represented in a neural population code and can be read out by a linear decoder such as population vector. I am wondering how much the proposed network model and learning rules will be changed if we use a neural population code to represent the rotation angle.\n", "limitations": "- Both models considered in this paper only passively estimate the rotation angle. It would be more interesting to synthesize the transformed images, as in an autoencoder framework.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "It is a fundamental question that how neural circuits learn the commutative group transformations. This study provides two novel solutions which rely on single neuron mechanism (implementing SVD) and network mechanism (implementing PCA) respectively to learn the group transformations. The network model implementing PCA (similarity matching specifically) to learn transformations is based on a recent study (ref. 1) so it is not clear how much difference of this part compared with the mechanism presented in ref. 1. The whole paper is structure-wise but some presentation is not clear and the comparison with earlier studies was not done systematically (see details below).\n\n### Writing\n- Eq. 3: the index t should be t-1 if I understood correctly.\n\n- Line 141: should the $\\sigma$ be the g in Lemma 1? If so, please use consistent notations across the manuscript. \n\n- It is not clear the derivation in Eq. 18 based on Eq. 16 and some motivations are missing. Also, it is not clear the meaning of matrices W and M in Eq. 18 and I need to guess whether they correspond to feedforward and recurrent weight matrices respectively.\n\n- Fig. 5a: not clear which algorithm leads to the presented filters. That is, are those filters learnt from the SVD (representing the u and v), or from the network implementation of PCA (W and M)?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "joEuus4fSr", "reviewer": "Reviewer_YACa", "summary": "This manuscript proposes biologically plausible algorithms for learning group transformations from sequences of observations. Two algorithms are proposed, one based on SVD and the other on PCA. The performance of these algorithms is evaluated in synthetic experiments.\n", "questions": "The assumption of line 89-90 seems fundamental to be able to the whole paper, but are not discussed. What do they entail?\nOne particular concern is the independence between $A_t$ and $x_t$, as it is likely that in many practical settings, transformations are performed depending on the previous state.\nMoreover, assuming the the pixel intensity vector is white is also very unusual and unlikely in practice.\n\nIn addition, the experiment seem to assume that rotations in each subspace are also chosen independently, which is not necessarily the case in practice, and may play a key role in identifying the subspaces.", "limitations": "Limitations are not discussed. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- the authors address a very interesting and challenging question: biological plausibility of the learning of transformation representations,\n\nWeaknesses:\n- the setting is quite restrictive, reducing the problem to previously proposed biologically plausible algorithms for PCA and SVD.\n- experiments are very toy-like, with very simplified assumptions whose influence is not tested,\n- relation to previous work on biologically inspired PCA and other such algorithms is elusive. It is difficult to figure out what is the originality of the proposed approaches, and why previous approaches could not directly be applied. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "jNwRq6fspyu", "reviewer": "Reviewer_vnKM", "summary": "The authors present two online biologically plausible algorithms for the irreducible representation learning method proposed by Cohen and Welling (2014). \nThe first algorithm uses SVD to extract the subspace of maximal average rotation. As part of this algorithm, the authors’ present a novel algorithm for online SVD: cross-power iteration.\nThe second is based on PCA of time difference vectors and uses local learning rules to preserve biological plausibility.\nThe authors propose that these models may serve as hypotheses to guide future connectomics and neurophysiology research\n\nResults. \nThe authors find that both the SVD and PCA methods are able to recover the irreducible representations of SO(2) and S1 on synthetic datasets\nThese synthetic datasets are sets of random images or vectors sampled from the standard normal distribution, followed by the application of the transformations of interest (translation or rotation).\n", "questions": "see above", "limitations": "see above", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper contributes two novel and interesting methods for learning the irreducible representations of toroidal groups with biologically plausible mechanisms. This is a valuable contribution for both computational neuroscience (as a hypothesis for neural circuitry) and machine learning (as a novel algorithm for learning and modeling transformation structure from data). \n\nWeaknesses:\n- It would be nice to see a quantitative analysis on the learned filters, explicitly comparing to the standard translational and rotational Fourier bases, and reporting how well the learned weights approximate these bases.\n- It would be useful to see how well the model trains on natural datasets.\n\nGeneral comments:\n- the paper is framed as a neurobiological model, but it really seems more \"neuromorphic\" as the emphasis is on how to learn all these things with local learning rules in a neural circuit, as opposed to specific neurobiological substrates in the visual system.  There are issues such as how to compute arctan, and many others, that would need to be discussed or elaborated more to think of this in neurobiological terms.\n- it will be interesting to see how these results could be extended to real-world applications, such as estimating optical flow from natural videos.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept