  We explore whether Neural Networks (NNs) can {\it discover} the presence of
symmetries as they learn to perform a task. For this, we train hundreds of NNs
on a {\it decoy task} based on well-controlled Physics templates, where no
information on symmetry is provided. We use the output from the last hidden
layer of all these NNs, projected to fewer dimensions, as the input for a
symmetry classification task, and show that information on symmetry had indeed
been identified by the original NN without guidance. As an interdisciplinary
application of this procedure, we identify the presence and level of symmetry
in artistic paintings from different styles such as those of Picasso, Pollock
and Van Gogh.
