title: Ensemble learning of high dimension datasets

authors: Lim, Nick Jin Sean

subjects: Ensemble learning

abstract: Ensemble learning, an approach in Machine Learning, makes decisions based on the collective decision of a committee of learners to solve complex tasks with minimal human intervention. Advances in computing technology have enabled researchers build datasets with the number of features in the order of thousands and enabled building more accurate predictive models. Unfortunately, high dimensional datasets are especially challenging for machine learning due to the phenomenon dubbed as the "curse of dimensionality". One approach to overcoming this challenge is ensemble learning using Random Subspace (RS) method, which has been shown to perform very well empirically however with few theoretical explanations to said effectiveness for classification tasks.

In this thesis, we aim to provide theoretical insights into RS ensemble classifiers to give a more in-depth understanding of the theoretical foundations of other ensemble classifiers. We investigate the conditions for norm-preservations in RS projections. Insights into this provide us with the theoretical basis for RS in algorithms that are based on the geometry of the data (i.e. clustering, nearest-neighbour). We then investigate the guarantees for the dot products of two random vectors after RS projection. This guarantee is useful to capture the geometric structure of a classification problem. We will then investigate the accuracy of a majority vote ensemble using a generalized Polya-Urn model, and how the parameters of the model are derived from diversity measures. We will discuss the practical implications of the model, explore the noise tolerance of ensembles, and give a plausible explanation for the effectiveness of ensembles.

We will provide empirical corroboration for our main results with both synthetic and real-world high-dimensional data. We will also discuss the implications of our theory on other applications (i.e. compressive sensing). Based on our results, we will propose a method of building ensembles for Deep Neural Network image classifications using RS projections without needing to retrain the neural network, which showed improved accuracy and very good robustness to adversarial examples. Ultimately, we hope that the insights gained in this thesis would make in-roads towards the answer to a key open question for ensemble classifiers, "When will an ensemble of weak learners outperform a single carefully tuned learner?"

text: Ensemble learning of high dimension datasets Ensemble learning, an approach in Machine Learning, makes decisions based on the collective decision of a committee of learners to solve complex tasks with minimal human intervention. Advances in computing technology have enabled researchers build datasets with the number of features in the order of thousands and enabled building more accurate predictive models. Unfortunately, high dimensional datasets are especially challenging for machine learning due to the phenomenon dubbed as the "curse of dimensionality". One approach to overcoming this challenge is ensemble learning using Random Subspace (RS) method, which has been shown to perform very well empirically however with few theoretical explanations to said effectiveness for classification tasks.

In this thesis, we aim to provide theoretical insights into RS ensemble classifiers to give a more in-depth understanding of the theoretical foundations of other ensemble classifiers. We investigate the conditions for norm-preservations in RS projections. Insights into this provide us with the theoretical basis for RS in algorithms that are based on the geometry of the data (i.e. clustering, nearest-neighbour). We then investigate the guarantees for the dot products of two random vectors after RS projection. This guarantee is useful to capture the geometric structure of a classification problem. We will then investigate the accuracy of a majority vote ensemble using a generalized Polya-Urn model, and how the parameters of the model are derived from diversity measures. We will discuss the practical implications of the model, explore the noise tolerance of ensembles, and give a plausible explanation for the effectiveness of ensembles.

We will provide empirical corroboration for our main results with both synthetic and real-world high-dimensional data. We will also discuss the implications of our theory on other applications (i.e. compressive sensing). Based on our results, we will propose a method of building ensembles for Deep Neural Network image classifications using RS projections without needing to retrain the neural network, which showed improved accuracy and very good robustness to adversarial examples. Ultimately, we hope that the insights gained in this thesis would make in-roads towards the answer to a key open question for ensemble classifiers, "When will an ensemble of weak learners outperform a single carefully tuned learner?"

year: 2020

title: Helping students value cultural diversity through research-based teaching

authors: Deakins, Eric

subjects: diversity (institutional)

abstract: Although international students studying in New Zealand desire and expect contact with their domestic peers, the level of cross-national interactions remains generally low. This paper describes an initiative to promote more and better intercultural understanding within a target group of students having similar needs and interests in a higher education setting. A research-based teaching approach progressively increased student engagement with higher order cognitive skills and both topic and process were aligned in such a way that training opportunities in intercultural competence were explored while also providing a process that offered further training in intercultural competence. Enduring appreciation of cultural diversity issues was achieved via deep styles of teaching and learning that raised awareness, changed attitudes and behaviour and ultimately impacted classroom culture. Initiative design, evaluation and results are described and limitations noted. The findings should be of interest to teachers of multicultural students and to academics studying cultural diversity issues.

text: Helping students value cultural diversity through research-based teaching Although international students studying in New Zealand desire and expect contact with their domestic peers, the level of cross-national interactions remains generally low. This paper describes an initiative to promote more and better intercultural understanding within a target group of students having similar needs and interests in a higher education setting. A research-based teaching approach progressively increased student engagement with higher order cognitive skills and both topic and process were aligned in such a way that training opportunities in intercultural competence were explored while also providing a process that offered further training in intercultural competence. Enduring appreciation of cultural diversity issues was achieved via deep styles of teaching and learning that raised awareness, changed attitudes and behaviour and ultimately impacted classroom culture. Initiative design, evaluation and results are described and limitations noted. The findings should be of interest to teachers of multicultural students and to academics studying cultural diversity issues.

year: 2009

title: 1999 Professorial Address: Nau te rourou, naku te rourou… Māori education: setting an agenda.

authors: Bishop, Russell

subjects: educational outcomes

abstract: Current educational policies and practices in Aotearoa/New Zealand were developed and continue to be developed within a framework of power imbalances, which effects Maori the greatest. An alternative model that seeks to address indigenous Māori aspirations and Treaty of Waitangi guarantees for self determination is presented here. This model suggests how a tertiary teacher education institution might create learning contexts wherein power-sharing images, principles and practices will facilitate successful participation by Māori students in mainstream classrooms. This model constitutes the classroom as a place where young people's sense-making processes (cultures) are incorporated and enhanced, where the existing knowledges of young people are seen as "acceptable" and "official" and where the teacher interacts with students in such a way that new knowledge is co-created. Such a classroom will generate totally different interaction and participation patterns and educational outcomes from a classroom where knowledge is seen as something that the teacher makes sense of and then passes on to students.

text: 1999 Professorial Address: Nau te rourou, naku te rourou… Māori education: setting an agenda. Current educational policies and practices in Aotearoa/New Zealand were developed and continue to be developed within a framework of power imbalances, which effects Maori the greatest. An alternative model that seeks to address indigenous Māori aspirations and Treaty of Waitangi guarantees for self determination is presented here. This model suggests how a tertiary teacher education institution might create learning contexts wherein power-sharing images, principles and practices will facilitate successful participation by Māori students in mainstream classrooms. This model constitutes the classroom as a place where young people's sense-making processes (cultures) are incorporated and enhanced, where the existing knowledges of young people are seen as "acceptable" and "official" and where the teacher interacts with students in such a way that new knowledge is co-created. Such a classroom will generate totally different interaction and participation patterns and educational outcomes from a classroom where knowledge is seen as something that the teacher makes sense of and then passes on to students.

year: 2000