Applying machine learning to Amharic text classification

Even though the last years have seen an increasing trend in investigating applying language processing methods to other languages than English, most of the work is still done on very few and mainly European and East-Asian languages. However, there is a need for people all over the World to be able to use their own language when using computers or accessing information on the Internet. This requires the existence of a variety of applications including local language spell-checkers, word processors, machine translation systems, search engines, etc. At the same time, the amount of work required to develop all aspects of natural language processing for a new language is huge. The main obstacles to progress in language processing for new languages are three-fold. Firstly, the peculiarities of a language itself might force new strategies to be developed. Secondly, the lack of already available resources and tools creates a vicious circle: having resources makes producing resources easier, but not having resources makes the creation and testing of new ones more difficult and time-consuming. Thirdly, there is commonly a lack of interest (and understanding) of the needs for people to be able to use their own language in computer applications — a lack of interest both in the surrounding world, but also sometimes even in the countries where a language is used. Until a critical mass of language processing resources have been made publicly available, it is not likely that natural language processing for any language can “take off” and reach a level higher than that of academic prototype systems. Making such resources publicly available will further boost activities in the area by allowing researchers to benefit from the work of others and thereby to build more advanced tools without having to invent the wheel over and over again. We believe this to be the best chance for any language with limited computational linguistic resources to break the vicious circle of always having to start from scratch with research in the area. Amharic is the language for country-wide communication in Ethiopia and has its own writing system which was incorporated into Unicode only in year 2000. It is quite dialectally diversified and probably representative of the languages of a continent which so far has received little attention within the language processing field. Amharic is a language for which very few computational linguistic tools or corpora (such as lexica, part-of-speech taggers, parsers or tree-banks) exist. 1 This problem, which is shared by a number of so called “under-resourced languages” has proven to be a bottleneck when it comes to promoting the use of computers and the Internet in the language. It is difficult to develop new linguistic resources without access to already existing ones. We are currently involved in project the primary goal of which is to investigate how well existing linguistic knowledge can be transferred between languages and to develop tools and techniques that can support such knowledge transfer. Another goal is to support the collection and publication of a large (monoand bilingual) corpus and other resources for Amharic.