title: Efficiently correcting machine learning: considering the role of example ordering in human-in-the-loop training of image classification models

authors: Holmes, Geoffrey

subjects: computer science

abstract: Arguably the most popular application task in artificial intelligence is image classification using transfer learning. Transfer learning enables models pre-trained on general classes of images, available in large numbers, to be refined for a specific application. This enables domain experts with their own—generally, substantially smaller—collections of images to build deep learning models. The good performance of such models poses the question of whether it is possible to further reduce the effort required to label training data by adopting a human-in-the-loop interface that presents the expert with the current predictions of the model on a new batch of data and only requires correction of these predictions—rather than de novo labelling by the expert—before retraining the model on the extended data. This paper looks at how to order the data in this iterative training scheme to achieve the highest model performance while minimising the effort needed to correct misclassified examples. Experiments are conducted involving five methods of ordering, using four image classification datasets, and three popular pre-trained models. Two of the methods we consider order the examples a priori whereas the other three employ an active learning approach where the ordering is updated iteratively after each new batch of data and retraining of the model. The main finding is that it is important to consider accuracy of the model in relation to the number of corrections that are required: using accuracy in relation to the number of labelled training examples—as is common practice in the literature—can be misleading. More specifically, active methods require more cumulative corrections than a priori methods for a given level of accuracy. Within their groups, active and a priori methods perform similarly. Preliminary evidence is provided that suggests that for “simple” problems, i.e., those involving fewer examples and classes, no method improves upon random selection of examples. For more complex problems, an a priori strategy based on a greedy sample selection method known as “kernel herding” performs best.

text: Efficiently correcting machine learning: considering the role of example ordering in human-in-the-loop training of image classification models Arguably the most popular application task in artificial intelligence is image classification using transfer learning. Transfer learning enables models pre-trained on general classes of images, available in large numbers, to be refined for a specific application. This enables domain experts with their own—generally, substantially smaller—collections of images to build deep learning models. The good performance of such models poses the question of whether it is possible to further reduce the effort required to label training data by adopting a human-in-the-loop interface that presents the expert with the current predictions of the model on a new batch of data and only requires correction of these predictions—rather than de novo labelling by the expert—before retraining the model on the extended data. This paper looks at how to order the data in this iterative training scheme to achieve the highest model performance while minimising the effort needed to correct misclassified examples. Experiments are conducted involving five methods of ordering, using four image classification datasets, and three popular pre-trained models. Two of the methods we consider order the examples a priori whereas the other three employ an active learning approach where the ordering is updated iteratively after each new batch of data and retraining of the model. The main finding is that it is important to consider accuracy of the model in relation to the number of corrections that are required: using accuracy in relation to the number of labelled training examples—as is common practice in the literature—can be misleading. More specifically, active methods require more cumulative corrections than a priori methods for a given level of accuracy. Within their groups, active and a priori methods perform similarly. Preliminary evidence is provided that suggests that for “simple” problems, i.e., those involving fewer examples and classes, no method improves upon random selection of examples. For more complex problems, an a priori strategy based on a greedy sample selection method known as “kernel herding” performs best.

year: 2022

title: Direct determination of vibrational density of states change on ligand binding to a protein

authors: Balog, Erika

subjects: protein

abstract: The change in the vibrational density of states of a protein (dihydrofolate reductase) on binding a ligand (methotrexate) is determined using inelastic neutron scattering. The vibrations of the complex soften significantly relative to the unbound protein. The resulting free-energy change, which is directly determined by the density of states change, is found to contribute significantly to the binding equilibrium.

text: Direct determination of vibrational density of states change on ligand binding to a protein The change in the vibrational density of states of a protein (dihydrofolate reductase) on binding a ligand (methotrexate) is determined using inelastic neutron scattering. The vibrations of the complex soften significantly relative to the unbound protein. The resulting free-energy change, which is directly determined by the density of states change, is found to contribute significantly to the binding equilibrium.

year: 2004

title: Research and development absorptive capacity: a Māori perspective

authors: Ruckstuhl, Katharina

subjects: computer science

abstract: This paper presents a view of research and development absorptive capacity from a Māori perspective. The assessment is part of a case study of a longitudinal programme – Science for Technological Innovation: Kia kotahi mai – Te Ao Putaiao me te Ao Hanagarau that aims to increase Aotearoa New Zealand’s capacity to use sci-tech for economic benefit. The paper finds that while Aotearoa New Zealand’s macro policy and meso institutional levels have become more responsive to Māori research and development demands, at the micro level of the individual or the firm there are still constraints given the small numbers of Māori in science, technology, engineering and mathematics. In response to this, a novel model of Māori sci-tech capacity is under development that considers not only research and development technical capacities, but also the human and relational capacities required to accelerate absorptive capacity to respond to Māori social and economic aspirations.

text: Research and development absorptive capacity: a Māori perspective This paper presents a view of research and development absorptive capacity from a Māori perspective. The assessment is part of a case study of a longitudinal programme – Science for Technological Innovation: Kia kotahi mai – Te Ao Putaiao me te Ao Hanagarau that aims to increase Aotearoa New Zealand’s capacity to use sci-tech for economic benefit. The paper finds that while Aotearoa New Zealand’s macro policy and meso institutional levels have become more responsive to Māori research and development demands, at the micro level of the individual or the firm there are still constraints given the small numbers of Māori in science, technology, engineering and mathematics. In response to this, a novel model of Māori sci-tech capacity is under development that considers not only research and development technical capacities, but also the human and relational capacities required to accelerate absorptive capacity to respond to Māori social and economic aspirations.

year: 2019