  Artificial Neural Networks (ANNs) became popular due to their successful
application difficult problems such image and speech recognition. However, when
practitioners want to design an ANN they need to undergo laborious process of
selecting a set of parameters and topology. Currently, there are several
state-of-the art methods that allow for the automatic selection of some of
these aspects. Learning Rate optimizers are a set of such techniques that
search for good values of learning rates. Whilst these techniques are effective
and have yielded good results over the years, they are general solution i.e.
they do not consider the characteristics of a specific network.
  We propose a framework called AutoLR to automatically design Learning Rate
Optimizers. Two versions of the system are detailed. The first one, Dynamic
AutoLR, evolves static and dynamic learning rate optimizers based on the
current epoch and the previous learning rate. The second version, Adaptive
AutoLR, evolves adaptive optimizers that can fine tune the learning rate for
each network eeight which makes them generally more effective. The results are
competitive with the best state of the art methods, even outperforming them in
some scenarios. Furthermore, the system evolved a classifier, ADES, that
appears to be novel and innovative since, to the best of our knowledge, it has
a structure that differs from state of the art methods.
