LEAP: Learning Embeddings for Adaptive Pace
Determining the optimal order in which data examples are presented to Deep Neural Networks during training is a non-trivial problem.However, choosing a non-trivial scheduling method may drastically improve convergence.In this paper, we propose a Self-Paced Learning-fused Deep Metric Learning framework, which we call Learning Embeddings for Adaptive Pace.Our method parameterizes mini-batches dynamically based on the and of the sample within a salient feature representation space.In LEAP, we train an Convolutional Neural Network to learn an expressive representation space by adaptive density discrimination using the Magnet Loss.The CNN classifier dynamically selects samples to form a mini-batch based on the from cross-entropy losses and of examples from the representation space sculpted by the CNN.We evaluate LEAP using deep CNN architectures for the task of supervised image classification on MNIST, FashionMNIST, CIFAR-10, CIFAR-100, and SVHN.We show that the LEAP framework converges faster with respect to the number of mini-batch updates required to achieve a comparable or better test performance on each of the datasets.
LEAP combines the strength of adaptive sampling with that of mini-batch online learning and adaptive representation learning to formulate a representative self-paced strategy in an end-to-end DNN training protocol. ', "Introduces a method for creating mini batches for a student network by using a second learned representation space to dynamically select examples by their 'easiness and true diverseness'.", 'Experiments the classification accuracy on MNIST, FashionMNIST, and CIFAR-10 datasets to learn a representation with curriculum learning style minibatch selection in an end-to-end framework.