title: Scalable Text Mining with Sparse Generative Models

authors: Puurula, Antti

subjects: text mining

abstract: The information age has brought a deluge of data. Much of this is in text form, insurmountable in scope for humans and incomprehensible in structure for computers. Text mining is an expanding field of research that seeks to utilize the information contained in vast document collections. General data mining methods based on machine learning face challenges with the scale of text data, posing a need for scalable text mining methods.

This thesis proposes a solution to scalable text mining: generative models combined with sparse computation. A unifying formalization for generative text models is defined, bringing together research traditions that have used formally equivalent models, but ignored parallel developments. This framework allows the use of methods developed in different processing tasks such as retrieval and classification, yielding effective solutions across different text mining tasks. Sparse computation using inverted indices is proposed for inference on probabilistic models. This reduces the computational complexity of the common text mining operations according to sparsity, yielding probabilistic models with the scalability of modern search engines.

The proposed combination provides sparse generative models: a solution for text mining that is general, effective, and scalable. Extensive experimentation on text classification and ranked retrieval datasets are conducted, showing that the proposed solution matches or outperforms the leading task-specific methods in effectiveness, with a order of magnitude decrease in classification times for Wikipedia article categorization with a million classes. The developed methods were further applied in two 2014 Kaggle data mining prize competitions with over a hundred competing teams, earning first and second places.

text: Scalable Text Mining with Sparse Generative Models The information age has brought a deluge of data. Much of this is in text form, insurmountable in scope for humans and incomprehensible in structure for computers. Text mining is an expanding field of research that seeks to utilize the information contained in vast document collections. General data mining methods based on machine learning face challenges with the scale of text data, posing a need for scalable text mining methods.

This thesis proposes a solution to scalable text mining: generative models combined with sparse computation. A unifying formalization for generative text models is defined, bringing together research traditions that have used formally equivalent models, but ignored parallel developments. This framework allows the use of methods developed in different processing tasks such as retrieval and classification, yielding effective solutions across different text mining tasks. Sparse computation using inverted indices is proposed for inference on probabilistic models. This reduces the computational complexity of the common text mining operations according to sparsity, yielding probabilistic models with the scalability of modern search engines.

The proposed combination provides sparse generative models: a solution for text mining that is general, effective, and scalable. Extensive experimentation on text classification and ranked retrieval datasets are conducted, showing that the proposed solution matches or outperforms the leading task-specific methods in effectiveness, with a order of magnitude decrease in classification times for Wikipedia article categorization with a million classes. The developed methods were further applied in two 2014 Kaggle data mining prize competitions with over a hundred competing teams, earning first and second places.

year: 2015

title: Continuous Typist Verification using Machine Learning

authors: Hempstalk, Kathryn

subjects: keystroke verification

abstract: A keyboard is a simple input device.  Its function is to send keystroke information to the computer (or other device) to which it is attached.  Normally this information is employed solely to produce text, but it can also be utilized as part of an authentication system.  Typist verification exploits a typist's patterns to check whether they are who they say they are, even after standard authentication schemes have confirmed their identity.  This thesis investigates whether typists behave in a sufficiently unique yet consistent manner to enable an effective level of verification based on their typing patterns.

Typist verification depends on more than the typist's behaviour.  The quality of the patterns and the algorithms used to compare them also determine how accurately verification is performed.  This thesis sheds light on all technical aspects of the problem, including data collection, feature identification and extraction, and sample classification.  

A dataset has been collected that is comparable in size, timing accuracy and content to others in the field, with one important exception: it is derived from real emails, rather than samples collected in an artificial setting.  This dataset is used to gain insight into what features distinguish typists from one another.  The features and dataset are used to train learning algorithms that make judgements on the origin of previously unseen typing samples.  These algorithms use 'one-class classification'; they make predictions for a particular user having been trained on only that user's patterns.

This thesis examines many one-class classification algorithms, including ones designed specifically for typist verification.  New algorithms and features are proposed to increase speed and accuracy.  The best method proposed performs at the state of the art in terms of classification accuracy, while decreasing the time taken for a prediction from minutes to seconds, and - more importantly - without requiring any negative data from other users.  Also, it is general: it applies not only to typist verification, but to any other one-class classification problem.  

Overall, this thesis concludes that typist verification can be considered a useful biometric technique.

text: Continuous Typist Verification using Machine Learning A keyboard is a simple input device.  Its function is to send keystroke information to the computer (or other device) to which it is attached.  Normally this information is employed solely to produce text, but it can also be utilized as part of an authentication system.  Typist verification exploits a typist's patterns to check whether they are who they say they are, even after standard authentication schemes have confirmed their identity.  This thesis investigates whether typists behave in a sufficiently unique yet consistent manner to enable an effective level of verification based on their typing patterns.

Typist verification depends on more than the typist's behaviour.  The quality of the patterns and the algorithms used to compare them also determine how accurately verification is performed.  This thesis sheds light on all technical aspects of the problem, including data collection, feature identification and extraction, and sample classification.  

A dataset has been collected that is comparable in size, timing accuracy and content to others in the field, with one important exception: it is derived from real emails, rather than samples collected in an artificial setting.  This dataset is used to gain insight into what features distinguish typists from one another.  The features and dataset are used to train learning algorithms that make judgements on the origin of previously unseen typing samples.  These algorithms use 'one-class classification'; they make predictions for a particular user having been trained on only that user's patterns.

This thesis examines many one-class classification algorithms, including ones designed specifically for typist verification.  New algorithms and features are proposed to increase speed and accuracy.  The best method proposed performs at the state of the art in terms of classification accuracy, while decreasing the time taken for a prediction from minutes to seconds, and - more importantly - without requiring any negative data from other users.  Also, it is general: it applies not only to typist verification, but to any other one-class classification problem.  

Overall, this thesis concludes that typist verification can be considered a useful biometric technique.

year: 2009

title: Indigenous aspirations and ecological integrity: Restoring and protecting the health and wellbeing of an ancestral river for future generations in Aotearoa New Zealand

authors: Te Aho, Linda

subjects: Maori

abstract: This paper seeks to provide a domestic example of the application of the environmental ethic of the indigenous Māori which ensures that humans are kaitiaki (guardians) of their surrounding environment by virtue of shared genealogy and how this ethic is being applied to restore and protect the health and wellbeing of the Waikato River, New Zealand’s longest. This particular river restoration project reflects many of the important aspects of ecological integrity.  It serves too. as a domestic example of the most sophisticated attempt at powersharing to date between Māori and the Crown in relation to natural resources, which may be of interest to other jurisdictions.

text: Indigenous aspirations and ecological integrity: Restoring and protecting the health and wellbeing of an ancestral river for future generations in Aotearoa New Zealand This paper seeks to provide a domestic example of the application of the environmental ethic of the indigenous Māori which ensures that humans are kaitiaki (guardians) of their surrounding environment by virtue of shared genealogy and how this ethic is being applied to restore and protect the health and wellbeing of the Waikato River, New Zealand’s longest. This particular river restoration project reflects many of the important aspects of ecological integrity.  It serves too. as a domestic example of the most sophisticated attempt at powersharing to date between Māori and the Crown in relation to natural resources, which may be of interest to other jurisdictions.

year: 2011