From: kishore@argo.rice.edu (Ananda Kishore)
Subject: How rit works (rit example)
Organization: Rice University, Houston
Date: Thu, 5 Nov 1992 23:37:23 GMT



       How the Rice Inverse Transliterator (rit) works


The key to rit's flexibility and power is its ability to distinguish
Telugu words from English. rit looks up each word in an on-line
English dictionary (using Unix's spell program). If the word is in the
English dictionary, then rit treats the word as English and prints it
accordingly. If the word is not in the English dictionary, then it is
a Telugu word (for rit) and will be printed so. This procedure will
work if the on-line English dictionary contains all the words in the
English language, and if no Telugu word becomes a valid English word
when transliterated. 

In practice, the on-line English dictionary doesn't contain all the
English words but only a subset of them (eg, holomorphic doesn't pass
through Unix spell.) and there are a few Telugu words which when
transliterated become valid English words (like ani, mana). Another
practical problem is the spell programs are not perfect, and they
accept non-English words as English. (Unix spell passes talli, tanni,
diSaloe, for example.) Such conflicting words (Telugu words that
become English words when transliterated) are few, fortunately. More
important than their number is the likely occurrence of these
conflicting words as English and as Telugu. While "mana" is an English
word, it is unlikely that it will be used in English sense by rit
users: so we default mana to Telugu. Similarly, we default "are" to
English. A key feature of the Rice Transliteration Standard for Telugu
is that it is almost "orthogonal" to English: there are very few
conflicting words, and they rarely cause problems.

The on-line English dictionary doesn't contain Indian proper names.
So, we added them to a local (English) dictionary which is an addendum
to the standard dictionary that Unix spell uses. When rit finds such a
proper name in this local dictionary, it treats the name as English.
Also, words like dharma, karma, and yoga are found in the standard
dictionary, and hence are treated English, by default. By providing a
way to change such defaults from English to Telugu and vice versa, rit
takes care of automatic determination of Telugu from English. 

Once we determine if a word is English or Telugu, we can do several
things: we can determine if there is a Telugu word in the same
sentence before a given English word. If so, we say that the English
word occurs in a Telugu context. We can then provide a
translation/transliteration service on some English words. (We
maintain a small English-to-Telugu transliteration/translation
dictionary for this purpose.) Similarly, once we determine that a word
is Telugu, we can check its Telugu spelling to see if there are any 
unacceptable letter clusters. 

That is the basic idea, and the rest of it is good book-keeping. From
FAQ:

23. Isn't looking up every word in the dictionary slow?

We invoke the spell program only once, on the entire file. This part
of the program is no slower than running a spell-check on an English
document which many word-processors normally do.

Ananda Kishore
Rama Rao Kanneganti

% rit defaults can be overridden dynamically, thus:
%[english ani mana] originally these are Telugu; but now English
%[english eg FAQ], eg is not an English word, for example.
% we already took care of talli, tanni, etc.

% if eg or FAQ appears in Telugu, then perhaps you haven't
% registered as a user of rit. To register, send mail to
% rit-register@power.ee.ndsu.nodak.edu


Newsgroups: soc.culture.indian.telugu