  This paper proposes a meta-learning approach to evolving a parametrized loss
function, which is called Meta-Loss Network (MLN), for training the image
classification learning on small datasets. In our approach, the MLN is embedded
in the framework of classification learning as a differentiable objective
function. The MLN is evolved with the Evolutionary Strategy algorithm (ES) to
an optimized loss function, such that a classifier, which optimized to minimize
this loss, will achieve a good generalization effect. A classifier learns on a
small training dataset to minimize MLN with Stochastic Gradient Descent (SGD),
and then the MLN is evolved with the precision of the small-dataset-updated
classifier on a large validation dataset. In order to evaluate our approach,
the MLN is trained with a large number of small sample learning tasks sampled
from FashionMNIST and tested on validation tasks sampled from FashionMNIST and
CIFAR10. Experiment results demonstrate that the MLN effectively improved
generalization compared to classical cross-entropy error and mean squared
error.
