Predictive and highly ambiguous typing for a severely speech and motion impaired user

Typing commands is the access of choice for many experienced computer users because of the speed and flexibili ty frequent interactions can take place and the ease of defining and using macros. One specific group of persons who can be counted as frequent computer users are speech impaired people relying on electronic devices for augmentative and alternative communication (AAC). But many speech impaired users are severely motor impaired, too. Therefore, the usage of a physical or on-screen keyboard is prohibitively costly for them with a standard keyboard layout. This paper presents a communication aid based on a highly ambiguous keyboard with word completion for communication, interaction and navigation. 1. Electronic communication for severely speech impaired users One outstanding feature of the information society nowadays is the importance of text as communication media in emails, SMS, newsgroups, and webpages. Even in synchronous situations like chatting electronic devices (computers, mobile phones, hand-helds) are used to transmit textual information . The success of ubiquituous communication has partly its reasons in the advantages of written messages via SMS: 1. they may be used for a synchronous or asynchronous dialogue as wanted; 2. they can be absorbed very fast and 3. they can be displayed in private (an important point for the use in classrooms). Likewise, chats, e-mails and newsgroups have become attractive alternative channels of synchronous or asynchronous communication for a varity of virtual communities (groups of interests). For the fraction of population affected by analphabetism, this development establishes a new barrier for participating in society. On the other hand, electronic communication could become an equally accessible communication channel for speech impared members of tomorrow’s information society. However, many strongly speech impaired people have severe motion impairments, too, because of the same causation: by birth (e.g. cerebral palsy), by accident or by disease (e.g. stroke, Lou Gehrig's or multiple sclerosis). For them, typing text with a standard keyboard or the usage of a graphical input device might be impossible. Often only the operation of a single or very few switch signals via buttons, joystick, eye tracking, EEG or other sensors is possible. There are several solutions available for substituting the physical keyboard of a computer by a virtual, on-screen keyboard with different layouts that can be operated with one or two physical switches by scanning: In the simple case of linear scanning, the user presses a switch A repeatedly to step from one key of the virtual keyboard to the next key of the scanning cycle until he reaches the desired key that is then activated by pressing a switch B. For rowcolumn scanning, the user first steps through rows of keys with switch A, selects the wanted row by switch B and then steps through the columns of keys in this row, finally activating the desired key again with switch B. In case only one physical switch is operated, one of the switches can be substituted by self-activation after a certain delay. Thus, the operation of a keyboard is simulated with one or two physical switches, but typing text letter by letter this way is cumbersome: in general, the typing of a word of six letters on a 32 keys keyboard by row-column scanning wil l take dozens of scanning steps. The achieved written communication rate ranges from less than 1 to 5 wpm (words per minute) — a typist's rate in comparison ranges from 10 to 20 wpm depending on her level of expertise (Darragh and Witten, 1992). 2. Ambiguous keyboards and word prediction Instead of reducing the number of required keystrokes for a word by using an enlarged keyboard with e.g. iconic codings, Kushler (1998) proposes the use of an ambiguous keyboard with multiple letter key assignments in the way phone keys are used for the so called vanity phone numbers: every phone digit key from 2 to 9 is additionally labeled with at least three letters. Instead of memorizing digit sequences for the phone number of telephone services, it suffices to dial the phone keys labeled with the letters of e.g. the name of the provider. If arbitrary words are coded this way many encodings will be the same for different words, i.e. the decoding function is not injective. Therefore, the user has to select the intended word in a list of word suggestions (ordererd e.g. by word frequencies) whenever the mapping between the encoding and the intended word is not one-to-one. Nevertheless, as Witten (1982) already pointed out, with nine letter keys only eight percent of 24,500 words actually are encoded ambiguously and thus need adisambiguation step. Following studies (Foulds et al., 1987) concentrated on ambiguous typing where only the next character is predicted based on n-grams and corrected if necessary, as the size of an electronic full word or even syllable dictionary would have been too limited (Arnott & Javed, 1992). The drawback of this character-based disambiguation is that the user has to pay attention to the predicted word prefix after each keystroke. Today, the storage of large vocabulary in main memory of a standard computer is not a serious problem anymore. The advantages of an ambiguous keyboard with eight keys and word disambiguation are enumerated by Kushler (1998) for users of alternative and augmentative communication devices as follows: 1. The efficiency of an ambiguous keyboard is near to one keystroke per letter. 2. Apart from literacy, no memorization of special encodings is required. 3. Attention to the display is required only after the word has been typed. 4. A keyboard with fewer keys can have larger keys for direct selection. 5. The average time to select a key by scanning is reduced considerably. 6. Simple, linear scanning can be used efficiently to select a key. 7. Fewer keys may allow direct selection with various input devices. Surely, the main arguments for a character-based text input with an ambiguous keyboard are that the keystroke per letter efficiency is very high and that there is no need for the user to learn any special encodings besides standard orthography. Further, if the user did a typing mistake, this error is corrected more easily as if the typing mistake is done via scanning on a 100+ keys keyboard. Finally, a character-based vocabulary can be extended very easily with new words, whereas an iconic language system has to depend on the user to memorize the icon word associations and reaches its limits, if we do not restrict ourselves to a base vocabulary of a few thousand words. The use of an ambiguous keyboard is compatible with another technique to accelerate typing input: the prediction and completion of user input. Actually, the completing prediction of commands and other user input (e.g. Darragh and Witten, 1992) to accelerate keyboard input for motion impaired users is a well-known and much used feature of standard software like word processors or command shells nowadays. The key idea is to present the user at every step of user input a ranked list of predictions what the user is going to type next based on a statistical model of the recent input. On top of reducing the typing costs there is another advantage of this input enhancing feature for users who do not always memorize exactly the set of admitted inputs in a certain context: The user can type the prefix of the intended input and request a list to be displayed for possible completion alternatives, thus reducing the cognitive load of memorization for orthography (Newell et al., 1995). 3. The UKO pilot study UKO is the abbreviation for German “Unbekanntes Kommunikationsobjekt” (i.e. “unidentified communication object” ) – the name given to our communication aid by its pilot user, a fifteen-year old girl with cerebral palsy who visits a regular school. This communication aid consists of an ambiguous keyboard with four letter keys plus an “enter” and a “delete” key. The keys are scanned cyclically. A second physical switch is used to activate the word completion with the word suggestions depicted on the right side (as in Figure 1). Figure 1. The UKO virtual keyboard (after selecting the first key four times) In his diploma thesis, Garbe (2000) followed an evolutionary optimisation approach to find an ambiguous letter key assignment for a keyboard with six keys that is to be used by cyclic scanning (cf. Levine and GoodenoughTrepagnier, 1990). The UKO system includes keyboard layouts for German and English and the corresponding lexica from the CELEX lexicon database with 360,000 resp. 160,000 word form entries (Baayen et al., 1995). Using the 360,000 German word form lexicon including word frequencies information, the ambiguous word form lists do not grow longer than 34 entries. This motivated us to look further for even smaller keyboards, and, actually, a keyboard with only three letter keys is used now, with a fourth key functioning as a quasimodal command key (Raskin, 2000) that is used in connection with one of the letter keys to invoke the “enter”, “delete” and “complete” function of the UKO system. These three plus one keys wil l be selected directly by our pilot user who formerly has used two-switch-scanning until now. Therefore, another considerable typing speed-up is expected. But even with linear scanning the six keys of the UKO keyboard as depicted in Figure 1, experimental results compared with an iconic communication system are encouraging: an time improvement of 18% could be achieved on a small test set of ten sentences with ten words in average, mainly since there were on average two unknown words per sentence using the iconic vocabulary, which then had to be spelled rather costly. Using the larger CELEX lexicon for the UKO system, only two of the 101 words in total were unknown for the ten sentences. The sentences with no unknown words have been entered in approximately the same time of 8.56 seconds