title: High-throughput machine learning algorithms

authors: Mitchell, Rory

subjects: thesis with publication

abstract: The field of machine learning has become strongly compute driven, such that emerging research and applications require larger amounts of specialised hardware or smarter algorithms to advance beyond the state-of-the-art. This thesis develops specialised techniques and algorithms for a subset of computationally difficult machine learning problems. The applications under investigation are quantile approximation in the limited-memory data streaming setting, interpretability of decision tree ensembles, efficient sampling methods in the space of permutations, and the generation of large numbers of pseudorandom permutations. These specific applications are investigated as they represent significant bottlenecks in real-world machine learning pipelines, where improvements to throughput have significant impact on the outcomes of machine learning projects in both industry and research. To address these bottlenecks, we discuss both theoretical improvements, such as improved convergence rates, and hardware/software related improvements, such as optimised algorithm design for high throughput hardware accelerators. 

Some contributions include: the evaluation of bin-packing methods for efficiently scheduling small batches of dependent computations to GPU hardware execution units, numerically stable reduction operators for higher-order statistical moments, and memory bandwidth optimisation for GPU shuffling. Additionally, we apply theory of the symmetric group of permutations in reproducing kernel Hilbert spaces, resulting in improved analysis of Monte Carlo methods for Shapley value estimation and new, computationally more efficient algorithms based on kernel herding and Bayesian quadrature. We also utilise reproducing kernels over permutations to develop a novel statistical test for the hypothesis that a sample of permutations is drawn from a uniform distribution.

The techniques discussed lie at the intersection of machine learning, high-performance computing, and applied mathematics. Much of the above work resulted in open source software used in real applications, including the GPUTreeShap library [38], shuffling primitives for the Thrust parallel computing library [2], extensions to the Shap package [31], and extensions to the XGBoost library [6].

text: High-throughput machine learning algorithms The field of machine learning has become strongly compute driven, such that emerging research and applications require larger amounts of specialised hardware or smarter algorithms to advance beyond the state-of-the-art. This thesis develops specialised techniques and algorithms for a subset of computationally difficult machine learning problems. The applications under investigation are quantile approximation in the limited-memory data streaming setting, interpretability of decision tree ensembles, efficient sampling methods in the space of permutations, and the generation of large numbers of pseudorandom permutations. These specific applications are investigated as they represent significant bottlenecks in real-world machine learning pipelines, where improvements to throughput have significant impact on the outcomes of machine learning projects in both industry and research. To address these bottlenecks, we discuss both theoretical improvements, such as improved convergence rates, and hardware/software related improvements, such as optimised algorithm design for high throughput hardware accelerators. 

Some contributions include: the evaluation of bin-packing methods for efficiently scheduling small batches of dependent computations to GPU hardware execution units, numerically stable reduction operators for higher-order statistical moments, and memory bandwidth optimisation for GPU shuffling. Additionally, we apply theory of the symmetric group of permutations in reproducing kernel Hilbert spaces, resulting in improved analysis of Monte Carlo methods for Shapley value estimation and new, computationally more efficient algorithms based on kernel herding and Bayesian quadrature. We also utilise reproducing kernels over permutations to develop a novel statistical test for the hypothesis that a sample of permutations is drawn from a uniform distribution.

The techniques discussed lie at the intersection of machine learning, high-performance computing, and applied mathematics. Much of the above work resulted in open source software used in real applications, including the GPUTreeShap library [38], shuffling primitives for the Thrust parallel computing library [2], extensions to the Shap package [31], and extensions to the XGBoost library [6].

year: 2021

title: Phylogeography and Ecology of New Zealand Freshwater Amphipoda (Paracalliope, Paraleptamphopus, and Phreatogammarus)

authors: Sutherland, Darin Lee

subjects: Genetics

abstract: This thesis examines phylogenetic patterns in three New Zealand amphipod taxa in relation to current geographic distributions and historical climatic (e.g. glaciation, marine inundation) and geological (e.g. mountain building) events using DNA sequencing and distributional data. It also examines how recognition behaviour can be used to delineate potential species boundaries and to assess the role of sexual selection.
The endemic genus Phreatogammarus has been found in only a limited number of sites and is not very abundant. An analysis of the genetic variation of two species within the genus using allozyme electrophoresis revealed high levels of genetic differentiation among populations but low levels within populations. This suggested that limited dispersal occurred among habitats with one population possibly representing a cryptic species.
The endemic freshwater genus Paraleptamphopus is thought to contain a large number of undescribed species with a number of these existing in small waterbodies such as seepages. Examination of the phylogeographic patterns using both mtDNA (CO1) and nuclear DNA (28S) showed that a number of distinct genetic lineages exist, with CO1 revealing 21 haplotypes with genetic distance of over 20%. Using a molecular clock rate of 2.4%, most haplotypes diverged approximately 8-12 million years ago during the Miocene era, possibly as a result of greater land availability increasing habitat diversity or by allopatric speciation. Morphological and genetic differences were not congruent, with morphologically similar taxa appearing among highly genetically distinct lineages, and some morphologically distinct forms appearing within single lineages.
The distribution and habitat variables of 419 sites were analysed to determine what was affecting the presence or absence of Paraleptamphopus. The presence of native vegetation in catchments had a positive affect on Paraleptamphopus distribution suggesting that large anthropogenic changes in catchment vegetation could have a negative effect on their abundance. I found smaller waterbodies to be more important than larger ones highlighting the need to study such sites as rare taxa may be ignored. A better understanding is needed on the role of small waterbodies in promoting overall species diversity in catchments. 
Examination of Paracalliope fluviatilis phylogenetic patterns using the mtDNA gene CO1 showed that a number of separate clades existed suggesting long term isolation and limited dispersal among catchments. Due to the large genetic divergences among some populations there was the possibility that cryptic species might exist. Species recognition experiments were conducted on seven populations to help determine whether cryptic species were present. For the three most genetically divergent crosses there was bias against inter-population pairings, suggesting that there were between two or three separate species.
Using a combined field and laboratory approach, size assortative mating was examined in Paracalliope fluviatilis. The field study showed positive size assortative mating and that larger females carried more eggs, suggesting they were more fecund. A series of laboratory experiments examining four existing theories explaining the phenomenon found that none adequately explained positive size assortative mating in P. fluviatilis. I therefore presented two new explanations to explain size assortative mating: a combination of female resistance and size-related variation in a male's capacity to amplex larger females or a form of indirect intra-sexual competition.

text: Phylogeography and Ecology of New Zealand Freshwater Amphipoda (Paracalliope, Paraleptamphopus, and Phreatogammarus) This thesis examines phylogenetic patterns in three New Zealand amphipod taxa in relation to current geographic distributions and historical climatic (e.g. glaciation, marine inundation) and geological (e.g. mountain building) events using DNA sequencing and distributional data. It also examines how recognition behaviour can be used to delineate potential species boundaries and to assess the role of sexual selection.
The endemic genus Phreatogammarus has been found in only a limited number of sites and is not very abundant. An analysis of the genetic variation of two species within the genus using allozyme electrophoresis revealed high levels of genetic differentiation among populations but low levels within populations. This suggested that limited dispersal occurred among habitats with one population possibly representing a cryptic species.
The endemic freshwater genus Paraleptamphopus is thought to contain a large number of undescribed species with a number of these existing in small waterbodies such as seepages. Examination of the phylogeographic patterns using both mtDNA (CO1) and nuclear DNA (28S) showed that a number of distinct genetic lineages exist, with CO1 revealing 21 haplotypes with genetic distance of over 20%. Using a molecular clock rate of 2.4%, most haplotypes diverged approximately 8-12 million years ago during the Miocene era, possibly as a result of greater land availability increasing habitat diversity or by allopatric speciation. Morphological and genetic differences were not congruent, with morphologically similar taxa appearing among highly genetically distinct lineages, and some morphologically distinct forms appearing within single lineages.
The distribution and habitat variables of 419 sites were analysed to determine what was affecting the presence or absence of Paraleptamphopus. The presence of native vegetation in catchments had a positive affect on Paraleptamphopus distribution suggesting that large anthropogenic changes in catchment vegetation could have a negative effect on their abundance. I found smaller waterbodies to be more important than larger ones highlighting the need to study such sites as rare taxa may be ignored. A better understanding is needed on the role of small waterbodies in promoting overall species diversity in catchments. 
Examination of Paracalliope fluviatilis phylogenetic patterns using the mtDNA gene CO1 showed that a number of separate clades existed suggesting long term isolation and limited dispersal among catchments. Due to the large genetic divergences among some populations there was the possibility that cryptic species might exist. Species recognition experiments were conducted on seven populations to help determine whether cryptic species were present. For the three most genetically divergent crosses there was bias against inter-population pairings, suggesting that there were between two or three separate species.
Using a combined field and laboratory approach, size assortative mating was examined in Paracalliope fluviatilis. The field study showed positive size assortative mating and that larger females carried more eggs, suggesting they were more fecund. A series of laboratory experiments examining four existing theories explaining the phenomenon found that none adequately explained positive size assortative mating in P. fluviatilis. I therefore presented two new explanations to explain size assortative mating: a combination of female resistance and size-related variation in a male's capacity to amplex larger females or a form of indirect intra-sexual competition.

year: 2006

title: What is the difference? What factors contribute to the variation of achievement for Year 7 and 8 Māori students in high decile, full primary schools?

authors: McLocklan, Sue

subjects: Maori student achievement

abstract: The educational disparity between indigenous Māori students and those of the majority continue to be a major issue in New Zealand. Attempting to bridge the gap for Māori students has generated focus on Māori student engagement in the classroom, the actions and understanding of the teacher and the relationship and interactions between the teacher and students in their learning environment. Elements of culturally responsive pedagogy have been identified as positive contributing factors to accelerating academic achievement for secondary Māori mainstream students. 
This small-scale cross-cultural exploratory qualitative research project seeks to identify the factors that contribute to student achievement variation for Year 7 and 8 students in high decile full primary schools and the integration of teacher’s cultural competencies in high quality learning opportunities. Within a kaupapa Māori research process, this study uses empirical data collected from a purposive sample through a semi-structured individual interview and a for self-assessment. Voluntary research participants were twelve students and six teachers in three schools in the Bay of Plenty, Waikato and Wellington regions. This research examines the engagement of student and teacher in pedagogical practice, learning variables, cultural knowledge and practices attributable to positive academic achievement for Māori. 
The findings reveal that when teachers place themselves in an agentic position; adopt a professional commitment and willingness to engage in effective relations, interactions and reciprocal practices to support Māori learners, high levels of relational trust and expectations prevail through mutual respect and interdependence to attain successful educational outcomes.
Further teacher professional learning is essential to enlighten and deepen teachers' understanding of culturally responsive pedagogy to enable Māori learners to reach their potential to participate and contribute in their world of choice.

text: What is the difference? What factors contribute to the variation of achievement for Year 7 and 8 Māori students in high decile, full primary schools? The educational disparity between indigenous Māori students and those of the majority continue to be a major issue in New Zealand. Attempting to bridge the gap for Māori students has generated focus on Māori student engagement in the classroom, the actions and understanding of the teacher and the relationship and interactions between the teacher and students in their learning environment. Elements of culturally responsive pedagogy have been identified as positive contributing factors to accelerating academic achievement for secondary Māori mainstream students. 
This small-scale cross-cultural exploratory qualitative research project seeks to identify the factors that contribute to student achievement variation for Year 7 and 8 students in high decile full primary schools and the integration of teacher’s cultural competencies in high quality learning opportunities. Within a kaupapa Māori research process, this study uses empirical data collected from a purposive sample through a semi-structured individual interview and a for self-assessment. Voluntary research participants were twelve students and six teachers in three schools in the Bay of Plenty, Waikato and Wellington regions. This research examines the engagement of student and teacher in pedagogical practice, learning variables, cultural knowledge and practices attributable to positive academic achievement for Māori. 
The findings reveal that when teachers place themselves in an agentic position; adopt a professional commitment and willingness to engage in effective relations, interactions and reciprocal practices to support Māori learners, high levels of relational trust and expectations prevail through mutual respect and interdependence to attain successful educational outcomes.
Further teacher professional learning is essential to enlighten and deepen teachers' understanding of culturally responsive pedagogy to enable Māori learners to reach their potential to participate and contribute in their world of choice.

year: 2013