: Current labeling principles impose challenges for machine learning models. We introduce the Label Refinery, an iterative procedure to update ground truth labels using a visual model trained on the entire dataset. The Label Refinery produces soft, multicategory, dynamically-generated labels consistent with the visual signal. The training image shown is labelled with the single category "burrito". After a few iterations of label refining, the labels from which the final model is trained are informative, unambiguous, and smooth. This results in major improvements in the model accuracy during successive stages of refinement as well as improved model generalization. These plots show that as models proceed through successive stages of refinement, the gaps between train and test results and approach ideal generalization. Abstract. Among the three main components (data, labels, and models) of any supervised learning system, data and models have been the main subjects of active research. However, studying labels and their properties has received very little attention. Current principles and paradigms of labeling impose several challenges to machine learning algorithms. Labels are often incomplete, ambiguous, and redundant. In this paper we study the effects of various properties of labels and introduce the Label Refinery: an iterative procedure that updates the ground truth labels after examining the entire dataset. We show significant gain using refined labels across a wide range of models. Using a Label Refinery improves the state-of-the-art top-1 accuracy of (1) AlexNet from 59.3 to 67.2, (2) MobileNet 1 from 70.6 to 73.39, (3) MobileNet 0.25 from 50.6 to 55.59, (4) VGG19 from 72.7 to 75.46, and (5) Darknet19 from 72.9 to 74.47.
Bagherinezhad et al. REF studied the effects of various properties of labels and introduce the Label Refinery method that iteratively updated the ground truth labels after examining the entire dataset with the teacherstudent learning paradigm.
Label Refinery: Improving ImageNet Classification through Label Progression
ArXiv
ArXiv
Computer Science