title: High-throughput machine learning algorithms

authors: Mitchell, Rory

subjects: thesis with publication

abstract: The field of machine learning has become strongly compute driven, such that emerging research and applications require larger amounts of specialised hardware or smarter algorithms to advance beyond the state-of-the-art. This thesis develops specialised techniques and algorithms for a subset of computationally difficult machine learning problems. The applications under investigation are quantile approximation in the limited-memory data streaming setting, interpretability of decision tree ensembles, efficient sampling methods in the space of permutations, and the generation of large numbers of pseudorandom permutations. These specific applications are investigated as they represent significant bottlenecks in real-world machine learning pipelines, where improvements to throughput have significant impact on the outcomes of machine learning projects in both industry and research. To address these bottlenecks, we discuss both theoretical improvements, such as improved convergence rates, and hardware/software related improvements, such as optimised algorithm design for high throughput hardware accelerators. 

Some contributions include: the evaluation of bin-packing methods for efficiently scheduling small batches of dependent computations to GPU hardware execution units, numerically stable reduction operators for higher-order statistical moments, and memory bandwidth optimisation for GPU shuffling. Additionally, we apply theory of the symmetric group of permutations in reproducing kernel Hilbert spaces, resulting in improved analysis of Monte Carlo methods for Shapley value estimation and new, computationally more efficient algorithms based on kernel herding and Bayesian quadrature. We also utilise reproducing kernels over permutations to develop a novel statistical test for the hypothesis that a sample of permutations is drawn from a uniform distribution.

The techniques discussed lie at the intersection of machine learning, high-performance computing, and applied mathematics. Much of the above work resulted in open source software used in real applications, including the GPUTreeShap library [38], shuffling primitives for the Thrust parallel computing library [2], extensions to the Shap package [31], and extensions to the XGBoost library [6].

text: High-throughput machine learning algorithms The field of machine learning has become strongly compute driven, such that emerging research and applications require larger amounts of specialised hardware or smarter algorithms to advance beyond the state-of-the-art. This thesis develops specialised techniques and algorithms for a subset of computationally difficult machine learning problems. The applications under investigation are quantile approximation in the limited-memory data streaming setting, interpretability of decision tree ensembles, efficient sampling methods in the space of permutations, and the generation of large numbers of pseudorandom permutations. These specific applications are investigated as they represent significant bottlenecks in real-world machine learning pipelines, where improvements to throughput have significant impact on the outcomes of machine learning projects in both industry and research. To address these bottlenecks, we discuss both theoretical improvements, such as improved convergence rates, and hardware/software related improvements, such as optimised algorithm design for high throughput hardware accelerators. 

Some contributions include: the evaluation of bin-packing methods for efficiently scheduling small batches of dependent computations to GPU hardware execution units, numerically stable reduction operators for higher-order statistical moments, and memory bandwidth optimisation for GPU shuffling. Additionally, we apply theory of the symmetric group of permutations in reproducing kernel Hilbert spaces, resulting in improved analysis of Monte Carlo methods for Shapley value estimation and new, computationally more efficient algorithms based on kernel herding and Bayesian quadrature. We also utilise reproducing kernels over permutations to develop a novel statistical test for the hypothesis that a sample of permutations is drawn from a uniform distribution.

The techniques discussed lie at the intersection of machine learning, high-performance computing, and applied mathematics. Much of the above work resulted in open source software used in real applications, including the GPUTreeShap library [38], shuffling primitives for the Thrust parallel computing library [2], extensions to the Shap package [31], and extensions to the XGBoost library [6].

year: 2021

title: Diorama and young people’s exploration of sustainable communities

authors: Taylor, Simon

subjects: Diorama, scientific play, agency.

abstract: This paper reports on a study using diorama constructions made by 13–14-year-olds who have been studying the topic of climate science. A diorama is a model representing a scene with three-dimensional structures, usually in miniature. The building and interaction of diorama is not the norm for students to represent their thinking of sustainable communities, however there is growing interest in representational work and multimodal reasoning which provide a springboard for scientific play, dialogue, and agency. Using a micro-ethnographic approach informed by socio-semiotic perspectives with the research of Lemke (2004), this study aimed to explore how diorama construction and the associated interaction can contribute to student reasoning and learning. Data were generated using audio recorders and multiple cameras, tracking groups of students negotiating their visual representations. The analysis proceeded through active and iterative viewing of audio and video recordings, followed by identification of themes to establish possible relationships between construction and reasoning. Through this process, the study (a) identifies multiple, necessary conditions and varied opportunities for students to enact and enable reasoning, and (b) extend current understanding of how the affordances of this visual mode interact with these conditions to contribute to student learning of climate science.

text: Diorama and young people’s exploration of sustainable communities This paper reports on a study using diorama constructions made by 13–14-year-olds who have been studying the topic of climate science. A diorama is a model representing a scene with three-dimensional structures, usually in miniature. The building and interaction of diorama is not the norm for students to represent their thinking of sustainable communities, however there is growing interest in representational work and multimodal reasoning which provide a springboard for scientific play, dialogue, and agency. Using a micro-ethnographic approach informed by socio-semiotic perspectives with the research of Lemke (2004), this study aimed to explore how diorama construction and the associated interaction can contribute to student reasoning and learning. Data were generated using audio recorders and multiple cameras, tracking groups of students negotiating their visual representations. The analysis proceeded through active and iterative viewing of audio and video recordings, followed by identification of themes to establish possible relationships between construction and reasoning. Through this process, the study (a) identifies multiple, necessary conditions and varied opportunities for students to enact and enable reasoning, and (b) extend current understanding of how the affordances of this visual mode interact with these conditions to contribute to student learning of climate science.

year: 2023-11-21

title: Maori children and death: Views from parents

authors: Jacob, Juanita

subjects: Maori

abstract: Research about Maori children's experiences and perceptions of death and tangi (Maori death rituals) is sparse. What is available tends to be generalised and stems from Western paradigms of knowledge. In this study we explore Maori children's experiences of death and tangi through the eyes of Maori parents. Through semi-structured interviews with 17 Maori parents, five areas were explored: a) the childhood experiences of parents and how they learned about death and an afterlife; b) what their adult beliefs about these matters are; c) how they have communicated the death concept to their children; and d) whether their children are likely to do the same in the future. From this study we learn that death was not hidden from children, that parents talked with their children in very open and age relevant ways, and considered their children's participation in tangi an important way to grieve and ensure continuity with kinship networks and support. This study suggests that the challenge now is to ensure that these practices continue to persist between parents and their children, and future generations.

text: Maori children and death: Views from parents Research about Maori children's experiences and perceptions of death and tangi (Maori death rituals) is sparse. What is available tends to be generalised and stems from Western paradigms of knowledge. In this study we explore Maori children's experiences of death and tangi through the eyes of Maori parents. Through semi-structured interviews with 17 Maori parents, five areas were explored: a) the childhood experiences of parents and how they learned about death and an afterlife; b) what their adult beliefs about these matters are; c) how they have communicated the death concept to their children; and d) whether their children are likely to do the same in the future. From this study we learn that death was not hidden from children, that parents talked with their children in very open and age relevant ways, and considered their children's participation in tangi an important way to grieve and ensure continuity with kinship networks and support. This study suggests that the challenge now is to ensure that these practices continue to persist between parents and their children, and future generations.

year: 2011-12