title: Contextualised approaches to embedding word senses

authors: Ansell, Alan John

subjects: Natural Language Processing

abstract: Vector representations of text are an essential tool for modern Natural Language Processing (NLP), and there has been much work devoted to finding effective methods for obtaining such representations. Most previously proposed methods derive vector representations for individual words, known as word embeddings. While word embeddings have enabled considerable advances in NLP, they have a significant theoretical drawback: many words have several, often completely unrelated meanings; it seems dubious to conflate these multiple meanings into a single point in semantic space.

This drawback has inspired an alternative, "multi-sense" approach to representing words. In this approach, rather than learning a single vector for each word, multiple vectors, or "sense embeddings," are learned corresponding to the individual meanings of the word. While this approach has not in general surpassed the word embedding approach, it has proved beneficial for a number of tasks such as word similarity estimation and word sense induction.

One of the most significant recent advances in NLP has been the development of "contextualised" word embedding models. Whereas word embeddings model the semantic properties of words in isolation, contextualised models represent the meanings of words in context. This enables them to capture some of the vast array of linguistic phenomena that occur above the word level.

I propose a number of new methods for learning sense embeddings which exploit contextualised techniques, based on the underlying hypothesis that the probability of a word occurring in a given context is equal to the sum of the probabilities of its individual senses occurring in the context. I first validate this hypothesis by using it to derive a simple method for learning sense embeddings inspired by the Skip-gram model. I then present a method for extracting sense embeddings from a contextualised word embedding model. Finally I propose an end-to-end model for learning sense embeddings, and show that it comprehensively outperforms previous sense embedding models on the task of word sense induction, a standard task for evaluation of such models. To demonstrate the model's flexibility I apply it to some other word-sense related tasks with good results.

text: Contextualised approaches to embedding word senses Vector representations of text are an essential tool for modern Natural Language Processing (NLP), and there has been much work devoted to finding effective methods for obtaining such representations. Most previously proposed methods derive vector representations for individual words, known as word embeddings. While word embeddings have enabled considerable advances in NLP, they have a significant theoretical drawback: many words have several, often completely unrelated meanings; it seems dubious to conflate these multiple meanings into a single point in semantic space.

This drawback has inspired an alternative, "multi-sense" approach to representing words. In this approach, rather than learning a single vector for each word, multiple vectors, or "sense embeddings," are learned corresponding to the individual meanings of the word. While this approach has not in general surpassed the word embedding approach, it has proved beneficial for a number of tasks such as word similarity estimation and word sense induction.

One of the most significant recent advances in NLP has been the development of "contextualised" word embedding models. Whereas word embeddings model the semantic properties of words in isolation, contextualised models represent the meanings of words in context. This enables them to capture some of the vast array of linguistic phenomena that occur above the word level.

I propose a number of new methods for learning sense embeddings which exploit contextualised techniques, based on the underlying hypothesis that the probability of a word occurring in a given context is equal to the sum of the probabilities of its individual senses occurring in the context. I first validate this hypothesis by using it to derive a simple method for learning sense embeddings inspired by the Skip-gram model. I then present a method for extracting sense embeddings from a contextualised word embedding model. Finally I propose an end-to-end model for learning sense embeddings, and show that it comprehensively outperforms previous sense embedding models on the task of word sense induction, a standard task for evaluation of such models. To demonstrate the model's flexibility I apply it to some other word-sense related tasks with good results.

year: 2020

title: Investigating the lignocellulolytic gut microbiome of huhu grubs

authors: Payyakkal Viswam, Jayadev

subjects: Gut microbiome

abstract: New Zealand’s endemic longhorned beetle (Prionoplus reticularis) larvae, huhu grub, are xylophagous and feed on a broad range of dead trees. Since insects are not believed to have the ability to fully degrade lignocellulose, it is likely that microbes present in the gut assist in degrading lignocellulose to release energy rich sugars. This process is considered the rate limiting step in the utilisation of woody material in biofuel production. To better understand the gut consortium present while feeding on the lignocellulose and cellulose components, huhu grubs were initially collected and reared on non-degraded pine blocks (lignocellulose) or cotton (cellulose). The gut fungal and bacterial community of huhu grubs was investigated using high-throughput sequencing and analysis techniques. Furthermore, the effects of a dietary switch on both fungal and bacterial communities was explored by switching huhu grubs initially reared on pine to cotton, and vice versa. Cellulose- and lignocellulose-reared huhu grubs were expected to host a wide range of bacteria and fungi in their gut, some of which produce lignocellulose degrading enzymes. Assessment of the gut fungal community composition confirmed that cellulose-reared huhu grubs were significantly distinct from lignocellulose-reared grubs. The fungal community of lignocellulose-fed huhu grubs was more functionally diverse, with several yeasts, soft-rot fungi and potential hemicellulose and cellulose degraders present. Whereas, the fungal community of cellulose-reared grubs was mostly abundant in yeasts and potential cellulose degraders. When cotton-reared grubs where switched to pine (lignocellulose), lignocellulose was observed to have a ‘bottleneck effect’ or selective pressure on the fungal gut community. The gut bacterial community of huhu grubs reared on cellulose or lignocellulose diets seemed to be unaffected by the selective pressure of the diet, and no significant change in the bacterial community was observed following dietary switch. Therefore it was concluded that the bacterial community while still important is playing a secondary role to the fungi. Finally, bioprospecting studies conducted using the frass (excretia) of these huhu grubs resulted in the isolation of a bacterium, Acinetobacter H23, capable of producing an extracellular laccase. Overall, the increased understanding of structure and composition of the gut microbiome of huhu grubs reared on cellulose or lignocellulose diets, along with isolation of novel lignocellulolytic isolates could prove to be significant for the biofuel industry.

text: Investigating the lignocellulolytic gut microbiome of huhu grubs New Zealand’s endemic longhorned beetle (Prionoplus reticularis) larvae, huhu grub, are xylophagous and feed on a broad range of dead trees. Since insects are not believed to have the ability to fully degrade lignocellulose, it is likely that microbes present in the gut assist in degrading lignocellulose to release energy rich sugars. This process is considered the rate limiting step in the utilisation of woody material in biofuel production. To better understand the gut consortium present while feeding on the lignocellulose and cellulose components, huhu grubs were initially collected and reared on non-degraded pine blocks (lignocellulose) or cotton (cellulose). The gut fungal and bacterial community of huhu grubs was investigated using high-throughput sequencing and analysis techniques. Furthermore, the effects of a dietary switch on both fungal and bacterial communities was explored by switching huhu grubs initially reared on pine to cotton, and vice versa. Cellulose- and lignocellulose-reared huhu grubs were expected to host a wide range of bacteria and fungi in their gut, some of which produce lignocellulose degrading enzymes. Assessment of the gut fungal community composition confirmed that cellulose-reared huhu grubs were significantly distinct from lignocellulose-reared grubs. The fungal community of lignocellulose-fed huhu grubs was more functionally diverse, with several yeasts, soft-rot fungi and potential hemicellulose and cellulose degraders present. Whereas, the fungal community of cellulose-reared grubs was mostly abundant in yeasts and potential cellulose degraders. When cotton-reared grubs where switched to pine (lignocellulose), lignocellulose was observed to have a ‘bottleneck effect’ or selective pressure on the fungal gut community. The gut bacterial community of huhu grubs reared on cellulose or lignocellulose diets seemed to be unaffected by the selective pressure of the diet, and no significant change in the bacterial community was observed following dietary switch. Therefore it was concluded that the bacterial community while still important is playing a secondary role to the fungi. Finally, bioprospecting studies conducted using the frass (excretia) of these huhu grubs resulted in the isolation of a bacterium, Acinetobacter H23, capable of producing an extracellular laccase. Overall, the increased understanding of structure and composition of the gut microbiome of huhu grubs reared on cellulose or lignocellulose diets, along with isolation of novel lignocellulolytic isolates could prove to be significant for the biofuel industry.

year: 2016

title: Assessment of fish populations in Lake Horowhenua, Levin

authors: Tempero, Grant Wayne

subjects: Lake Horowhenua

abstract: Lake Horowhenua (Waipunahau) is of substantial historical, cultural and recreational value to the people of the Horowhenua region. However, water quality and biodiversity within the lake has been in decline for a number of years. As part of lake restoration efforts by Horizons Regional Council and the Lake Horowhenua Trustees, a survey of fish species in Lake Horowhenua was conducted by the University of Waikato using boat electrofishing and fyke netting. A lake restoration plan had previously identified invasive fish species such as koi carp (Cyprinus carpio) and European perch (Perca fluviatilis) as being potential barriers to rehabilitation of the lake. The purpose of this survey was to determine the abundance and diversity of fish species within the lake and to ascertain if pest fish species were present at biomasses high enough to be negatively impacting on lake ecology. Recommendations would then be made as to the potential methods and necessity for pest fish removal.

text: Assessment of fish populations in Lake Horowhenua, Levin Lake Horowhenua (Waipunahau) is of substantial historical, cultural and recreational value to the people of the Horowhenua region. However, water quality and biodiversity within the lake has been in decline for a number of years. As part of lake restoration efforts by Horizons Regional Council and the Lake Horowhenua Trustees, a survey of fish species in Lake Horowhenua was conducted by the University of Waikato using boat electrofishing and fyke netting. A lake restoration plan had previously identified invasive fish species such as koi carp (Cyprinus carpio) and European perch (Perca fluviatilis) as being potential barriers to rehabilitation of the lake. The purpose of this survey was to determine the abundance and diversity of fish species within the lake and to ascertain if pest fish species were present at biomasses high enough to be negatively impacting on lake ecology. Recommendations would then be made as to the potential methods and necessity for pest fish removal.

year: 2013