  In this paper we study the spontaneous development of symmetries in the early
layers of a Convolutional Neural Network (CNN) during learning on natural
images. Our architecture is built in such a way to mimic the early stages of
biological visual systems. In particular, it contains a pre-filtering step
$\ell^0$ defined in analogy with the Lateral Geniculate Nucleus (LGN).
Moreover, the first convolutional layer is equipped with lateral connections
defined as a propagation driven by a learned connectivity kernel, in analogy
with the horizontal connectivity of the primary visual cortex (V1). The layer
$\ell^0$ shows a rotational symmetric pattern well approximated by a Laplacian
of Gaussian (LoG), which is a well-known model of the receptive profiles of LGN
cells. The convolutional filters in the first layer can be approximated by
Gabor functions, in agreement with well-established models for the profiles of
simple cells in V1. We study the learned lateral connectivity kernel of this
layer, showing the emergence of orientation selectivity w.r.t. the learned
filters. We also examine the association fields induced by the learned kernel,
and show qualitative and quantitative comparisons with known group-based models
of V1 horizontal connectivity. These geometric properties arise spontaneously
during the training of the CNN architecture, analogously to the emergence of
symmetries in visual systems thanks to brain plasticity driven by external
stimuli.
