title: Extracting corpus specific knowledge bases from Wikipedia

authors: Milne, David N.

subjects: Wikipedia

abstract: Thesauri are useful knowledge structures for assisting information retrieval. Yet their production is labor-intensive, and few domains have comprehensive thesauri that cover domain-specific concepts and contemporary usage. One approach, which has been attempted without much success for decades, is to seek statistical natural language processing algorithms that work on free text. Instead, we propose to replace costly professional indexers with thousands of dedicated amateur volunteers--namely, those that are producing Wikipedia. This vast, open encyclopedia represents a rich tapestry of topics and semantics and a huge investment of human effort and judgment. We show how this can be directly exploited to provide WikiSauri: manually-defined yet inexpensive thesaurus structures that are specifically tailored to expose the topics, terminology and semantics of individual document collections. We also offer concrete evidence of the effectiveness of WikiSauri for assisting information retrieval.

text: Extracting corpus specific knowledge bases from Wikipedia Thesauri are useful knowledge structures for assisting information retrieval. Yet their production is labor-intensive, and few domains have comprehensive thesauri that cover domain-specific concepts and contemporary usage. One approach, which has been attempted without much success for decades, is to seek statistical natural language processing algorithms that work on free text. Instead, we propose to replace costly professional indexers with thousands of dedicated amateur volunteers--namely, those that are producing Wikipedia. This vast, open encyclopedia represents a rich tapestry of topics and semantics and a huge investment of human effort and judgment. We show how this can be directly exploited to provide WikiSauri: manually-defined yet inexpensive thesaurus structures that are specifically tailored to expose the topics, terminology and semantics of individual document collections. We also offer concrete evidence of the effectiveness of WikiSauri for assisting information retrieval.

year: 2007-06-01

title: Left restriction monoids from left E-completions

authors: Stokes, Tim E.

abstract: Given a monoid S with E any non-empty subset of its idempotents, we present a novel one-sided version of idempotent completion we call left E-completion. In general, the construction yields a one-sided variant of a small category called a constellation by Gould and Hollings. Under certain conditions, this constellation is inductive, meaning that its partial multiplication may be extended to give a left restriction semigroup, a type of unary semigroup whose unary operation models domain. We study the properties of those pairs  for which this happens, and characterise those left restriction semigroups that arise as such left E-completions of their submonoid of elements having domain 1. As first applications, we decompose the left restriction semigroup of partial functions on the set X and the right restriction semigroup of left total partitions on X as left and right E-completions respectively of the transformation semigroup  on X, and decompose the left restriction semigroup of binary relations on X under demonic composition as a left E-completion of the left-total binary relations. In many cases, including these three examples, the construction embeds in a semigroup Zappa-Szép product.

text: Left restriction monoids from left E-completions Given a monoid S with E any non-empty subset of its idempotents, we present a novel one-sided version of idempotent completion we call left E-completion. In general, the construction yields a one-sided variant of a small category called a constellation by Gould and Hollings. Under certain conditions, this constellation is inductive, meaning that its partial multiplication may be extended to give a left restriction semigroup, a type of unary semigroup whose unary operation models domain. We study the properties of those pairs  for which this happens, and characterise those left restriction semigroups that arise as such left E-completions of their submonoid of elements having domain 1. As first applications, we decompose the left restriction semigroup of partial functions on the set X and the right restriction semigroup of left total partitions on X as left and right E-completions respectively of the transformation semigroup  on X, and decompose the left restriction semigroup of binary relations on X under demonic composition as a left E-completion of the left-total binary relations. In many cases, including these three examples, the construction embeds in a semigroup Zappa-Szép product.

year: 2022-05

title: Machine translation for te reo Māori

authors: Keegan, Te Taka Adrian Gregory

abstract: What is Machine Translation? Machine Translation (MT) is a process where computer software is used to translate texts from one natural (or spoken) language to another. Early research centred on two distinct approaches: Rules Based Machine Translation (RBMT) and Statistical Machine Translation (SMT). In simple terms RBMT makes use of large sets of linguistic rules that define languages whereas SMT uses statistical techniques to build language models from large language corpora. Increases in computing power and the amount of language corpora available has meant that SMT had become the preferred option with recent advances in neural networks also being applied to improve the accuracy of SMT. For commercial reasons, this is an area of research that has generated a lot of interest and funding support from some major international computer companies, including Google¹,Microsoft² and Facebook³.
Why is Machine Translation important for te reo Māori? A te reo Māori purist may argue that it is not important to focus activities on having a machine undertake translations for te reo Māori; if people want to understand te reo Māori then they should put in the effort to learn the language. This line of reasoning is difficult to argue with. But, from a Māori language activist perspective the value of MT is not so much in the translation of te reo Māori to (say) English, but rather the translation of English to te reo Māori. If this translation can be done efficiently, with low costs, it will assist in the proliferation of te reo Māori into new contexts, new environments and will assist its normalisation in New Zealand's society.
At this time, two of the major international companies, Google and Microsoft, have invested significantly in MT for te reo Māori. This paper summarises their endeavours and reports on the quality of translations they have been able to generate.

text: Machine translation for te reo Māori What is Machine Translation? Machine Translation (MT) is a process where computer software is used to translate texts from one natural (or spoken) language to another. Early research centred on two distinct approaches: Rules Based Machine Translation (RBMT) and Statistical Machine Translation (SMT). In simple terms RBMT makes use of large sets of linguistic rules that define languages whereas SMT uses statistical techniques to build language models from large language corpora. Increases in computing power and the amount of language corpora available has meant that SMT had become the preferred option with recent advances in neural networks also being applied to improve the accuracy of SMT. For commercial reasons, this is an area of research that has generated a lot of interest and funding support from some major international computer companies, including Google¹,Microsoft² and Facebook³.
Why is Machine Translation important for te reo Māori? A te reo Māori purist may argue that it is not important to focus activities on having a machine undertake translations for te reo Māori; if people want to understand te reo Māori then they should put in the effort to learn the language. This line of reasoning is difficult to argue with. But, from a Māori language activist perspective the value of MT is not so much in the translation of te reo Māori to (say) English, but rather the translation of English to te reo Māori. If this translation can be done efficiently, with low costs, it will assist in the proliferation of te reo Māori into new contexts, new environments and will assist its normalisation in New Zealand's society.
At this time, two of the major international companies, Google and Microsoft, have invested significantly in MT for te reo Māori. This paper summarises their endeavours and reports on the quality of translations they have been able to generate.

year: 2017