title: Contextualised approaches to embedding word senses

authors: Ansell, Alan John

subjects: Natural Language Processing

abstract: Vector representations of text are an essential tool for modern Natural Language Processing (NLP), and there has been much work devoted to finding effective methods for obtaining such representations. Most previously proposed methods derive vector representations for individual words, known as word embeddings. While word embeddings have enabled considerable advances in NLP, they have a significant theoretical drawback: many words have several, often completely unrelated meanings; it seems dubious to conflate these multiple meanings into a single point in semantic space.

This drawback has inspired an alternative, "multi-sense" approach to representing words. In this approach, rather than learning a single vector for each word, multiple vectors, or "sense embeddings," are learned corresponding to the individual meanings of the word. While this approach has not in general surpassed the word embedding approach, it has proved beneficial for a number of tasks such as word similarity estimation and word sense induction.

One of the most significant recent advances in NLP has been the development of "contextualised" word embedding models. Whereas word embeddings model the semantic properties of words in isolation, contextualised models represent the meanings of words in context. This enables them to capture some of the vast array of linguistic phenomena that occur above the word level.

I propose a number of new methods for learning sense embeddings which exploit contextualised techniques, based on the underlying hypothesis that the probability of a word occurring in a given context is equal to the sum of the probabilities of its individual senses occurring in the context. I first validate this hypothesis by using it to derive a simple method for learning sense embeddings inspired by the Skip-gram model. I then present a method for extracting sense embeddings from a contextualised word embedding model. Finally I propose an end-to-end model for learning sense embeddings, and show that it comprehensively outperforms previous sense embedding models on the task of word sense induction, a standard task for evaluation of such models. To demonstrate the model's flexibility I apply it to some other word-sense related tasks with good results.

text: Contextualised approaches to embedding word senses Vector representations of text are an essential tool for modern Natural Language Processing (NLP), and there has been much work devoted to finding effective methods for obtaining such representations. Most previously proposed methods derive vector representations for individual words, known as word embeddings. While word embeddings have enabled considerable advances in NLP, they have a significant theoretical drawback: many words have several, often completely unrelated meanings; it seems dubious to conflate these multiple meanings into a single point in semantic space.

This drawback has inspired an alternative, "multi-sense" approach to representing words. In this approach, rather than learning a single vector for each word, multiple vectors, or "sense embeddings," are learned corresponding to the individual meanings of the word. While this approach has not in general surpassed the word embedding approach, it has proved beneficial for a number of tasks such as word similarity estimation and word sense induction.

One of the most significant recent advances in NLP has been the development of "contextualised" word embedding models. Whereas word embeddings model the semantic properties of words in isolation, contextualised models represent the meanings of words in context. This enables them to capture some of the vast array of linguistic phenomena that occur above the word level.

I propose a number of new methods for learning sense embeddings which exploit contextualised techniques, based on the underlying hypothesis that the probability of a word occurring in a given context is equal to the sum of the probabilities of its individual senses occurring in the context. I first validate this hypothesis by using it to derive a simple method for learning sense embeddings inspired by the Skip-gram model. I then present a method for extracting sense embeddings from a contextualised word embedding model. Finally I propose an end-to-end model for learning sense embeddings, and show that it comprehensively outperforms previous sense embedding models on the task of word sense induction, a standard task for evaluation of such models. To demonstrate the model's flexibility I apply it to some other word-sense related tasks with good results.

year: 2020

title: Rapid, large-scale and inexpensive genotype differentiation of Mus musculus castaneus and domesticus sub-species

authors: McCormick, Helen Margaret

subjects: ecological genetics

abstract: A robust rapid method is described for differentiating Mus musculus castaneus and domesticus sub-species based on the analysis of restriction fragment length polymorphisms in three regions of the mtDNA genome.

text: Rapid, large-scale and inexpensive genotype differentiation of Mus musculus castaneus and domesticus sub-species A robust rapid method is described for differentiating Mus musculus castaneus and domesticus sub-species based on the analysis of restriction fragment length polymorphisms in three regions of the mtDNA genome.

year: 2010

title: KAUMĀTUATANGA  Supporting School Leaders To Develop Cultural Values While Resisting The Dominance of Colonialism

authors: Nuri, Ngahuia

subjects: Kaumātua

abstract: In an education system that is asserting to the importance of Māori (indigenous people of Aotearoa) language, culture, identity and the history of Aotearoa  (New Zealand) into the curriculum, there is direction from the Ministry of Education (MOE) for schools to reach out to iwi. What this looks like and how this can be achieved is not an easy task. This thesis follows the journey of three kaumātua (respected, knowledgeable elders, both female and male) working alongside the Senior Leadership Team (SLT) of a decile 10 kura auraki (mainstream primary school) where ākonga Māori (Māori students) were in the minority. The research examines what role kaumātua might have in guiding other schools to help tamariki Māori (Māori children) enjoy and achieve education success as Māori. It highlights the coming together of leaders from two different worldviews with this vision as a common purpose. 

Kaumātua in te ao Māori (the Māori world) are respected elders. Kaumātua are leaders, respected for their wisdom, knowledge and models of behaviour among other attributes. The Māui narratives are one example of demonstrating the characteristics of kaumātua. It was through his respected elders that Maui achieved so much. It was his grandfather who rescued him and taught him all he knew. It was his grandmother Mahuika who gave him fire and it was his kuia (female elder) Murirangiwhenua from whom he received the magic jawbone. The wisdom of Māori kaumātua has been recognised for centuries by Māori and yet kaumātua are often an untapped resource in the mainstream sector of education. In this thesis the kaumātua and members of the SLT share their experiences of working together and the learning and unlearning that has taken place over a period of four years from 2016 through 2019. They share their vulnerability, humility and strength in the goals they have worked towards over this period of time.

text: KAUMĀTUATANGA  Supporting School Leaders To Develop Cultural Values While Resisting The Dominance of Colonialism In an education system that is asserting to the importance of Māori (indigenous people of Aotearoa) language, culture, identity and the history of Aotearoa  (New Zealand) into the curriculum, there is direction from the Ministry of Education (MOE) for schools to reach out to iwi. What this looks like and how this can be achieved is not an easy task. This thesis follows the journey of three kaumātua (respected, knowledgeable elders, both female and male) working alongside the Senior Leadership Team (SLT) of a decile 10 kura auraki (mainstream primary school) where ākonga Māori (Māori students) were in the minority. The research examines what role kaumātua might have in guiding other schools to help tamariki Māori (Māori children) enjoy and achieve education success as Māori. It highlights the coming together of leaders from two different worldviews with this vision as a common purpose. 

Kaumātua in te ao Māori (the Māori world) are respected elders. Kaumātua are leaders, respected for their wisdom, knowledge and models of behaviour among other attributes. The Māui narratives are one example of demonstrating the characteristics of kaumātua. It was through his respected elders that Maui achieved so much. It was his grandfather who rescued him and taught him all he knew. It was his grandmother Mahuika who gave him fire and it was his kuia (female elder) Murirangiwhenua from whom he received the magic jawbone. The wisdom of Māori kaumātua has been recognised for centuries by Māori and yet kaumātua are often an untapped resource in the mainstream sector of education. In this thesis the kaumātua and members of the SLT share their experiences of working together and the learning and unlearning that has taken place over a period of four years from 2016 through 2019. They share their vulnerability, humility and strength in the goals they have worked towards over this period of time.

year: 2024