title: Diacritic Restoration and the Development of a Part-of-Speech Tagset for the Māori Language

authors: Cocks, John

subjects: Māori

abstract: This thesis investigates two fundamental problems in natural language processing: diacritic restoration and part-of-speech tagging. Over the past three decades, statistical approaches to diacritic restoration and part-of-speech tagging have grown in interest as a consequence of the increasing availability of manually annotated training data in major languages such as English and French. However, these approaches are not practical for most minority languages, where appropriate training data is either non-existent or not publically available. Furthermore, before developing a part-of-speech tagging system, a suitable tagset is required for that language. In this thesis, we make the following contributions to bridge this gap:

Firstly, we propose a method for diacritic restoration based on naive Bayes classifiers that act at word-level. Classifications are based on a rich set of features, extracted automatically from training data in the form of diacritically marked text. This method requires no additional resources, which makes it language independent. The algorithm was evaluated on one language, namely Māori, and an accuracy exceeding 99% was observed.

Secondly, we present our work on creating one of the necessary resources for the development of a part-of-speech tagging system in Māori, that of a suitable tagset. The tagset described was developed in accordance with the EAGLES guidelines for morphosyntactic annotation of corpora, and was the result of in-depth analysis of the Māori grammar.

text: Diacritic Restoration and the Development of a Part-of-Speech Tagset for the Māori Language This thesis investigates two fundamental problems in natural language processing: diacritic restoration and part-of-speech tagging. Over the past three decades, statistical approaches to diacritic restoration and part-of-speech tagging have grown in interest as a consequence of the increasing availability of manually annotated training data in major languages such as English and French. However, these approaches are not practical for most minority languages, where appropriate training data is either non-existent or not publically available. Furthermore, before developing a part-of-speech tagging system, a suitable tagset is required for that language. In this thesis, we make the following contributions to bridge this gap:

Firstly, we propose a method for diacritic restoration based on naive Bayes classifiers that act at word-level. Classifications are based on a rich set of features, extracted automatically from training data in the form of diacritically marked text. This method requires no additional resources, which makes it language independent. The algorithm was evaluated on one language, namely Māori, and an accuracy exceeding 99% was observed.

Secondly, we present our work on creating one of the necessary resources for the development of a part-of-speech tagging system in Māori, that of a suitable tagset. The tagset described was developed in accordance with the EAGLES guidelines for morphosyntactic annotation of corpora, and was the result of in-depth analysis of the Māori grammar.

year: 2012

title: Māori & Psychology Research Unit annual report 2010

authors: Rua, Mohi

subjects: Māori and Psychology Research Unit

abstract: Annual report of the Māori and Psychology Research Unit (MPRU) 2010. The unit was established in August of 1997. The unit is designed to provide a catalyst and support network for enhancing research concerning the psychological needs, aspirations, and priorities of Maori people. The MPRU is well situated to draw together skilled and experienced interdisciplinary research groups by networking and establishing working relationships with staff and students within the Faculty of Arts and Social Sciences, the University, and the wider community.

text: Māori & Psychology Research Unit annual report 2010 Annual report of the Māori and Psychology Research Unit (MPRU) 2010. The unit was established in August of 1997. The unit is designed to provide a catalyst and support network for enhancing research concerning the psychological needs, aspirations, and priorities of Maori people. The MPRU is well situated to draw together skilled and experienced interdisciplinary research groups by networking and establishing working relationships with staff and students within the Faculty of Arts and Social Sciences, the University, and the wider community.

year: 2011

title: Scaling up semi-supervised learning: An efficient and effective LLGC variant

authors: Pfahringer, Bernhard

subjects: computer science

abstract: Domains like text classification can easily supply large amounts of unlabeled data, but labeling itself is expensive. Semi- supervised learning tries to exploit this abundance of unlabeled training data to improve classification. Unfortunately most of the theoretically well-founded algorithms that have been described in recent years are cubic or worse in the total number of both labeled and unlabeled training examples. In this paper we apply modifications to the standard LLGC algorithm to improve efficiency to a point where we can handle datasets with hundreds of thousands of training data. The modifications are priming of the unlabeled data, and most importantly, sparsification of the similarity matrix. We report promising results on large text classification problems.

text: Scaling up semi-supervised learning: An efficient and effective LLGC variant Domains like text classification can easily supply large amounts of unlabeled data, but labeling itself is expensive. Semi- supervised learning tries to exploit this abundance of unlabeled training data to improve classification. Unfortunately most of the theoretically well-founded algorithms that have been described in recent years are cubic or worse in the total number of both labeled and unlabeled training examples. In this paper we apply modifications to the standard LLGC algorithm to improve efficiency to a point where we can handle datasets with hundreds of thousands of training data. The modifications are priming of the unlabeled data, and most importantly, sparsification of the similarity matrix. We report promising results on large text classification problems.

year: 2007