Developing a new grammar checker for English as a second language

In this paper we describe the prototype of a new grammar checker specifically geared to the needs of French speakers writing in Engl ish . Most c o m m e r c i a l g r a m m a r checkers on the market today are meant to be used by native speakers of a language who have good intui t ions about their own language competence. Non-native speakers of a language, however, have different intuitions and are very easily confused by false alarms, i.e. error messages given by the grammar checker when there is in fact no error in the text. In our project a imed at developing a complete writing tool for the non-nat ive speaker, we concent ra ted on building a grammar checker that keeps the rate of over-f lagging down and on developing a user-friendly writing environment which contains, among other things, a series of on-line helps. The grammar checking component, which is the focus of this paper, uses island processing (or chunking) rather than a full parse. This approach is both rapid and appropriate when a text contains many errors. We explain how we use automata to identify mul t i -word units, detect errors (which we first isolated in a corpus of errors) and interact with the user. We end with a short evaluation of our prototype and compare it to three cur ren t ly ava i lab le commercial grammar checkers. I In troduct ion Many word-processing systems today include grammar checkers which can be used to locate various grammatical problems in a text. These tools are clearly aimed at native speakers even if they can be of some help to non-na t ive speakers as wel l . However , non-native speakers make more errors than native speakers and their errors are quite different (Corder 1981). Because of this, they require grammar c h e c k e r s designed for their specific needs (Granger & Meunier 1994). The prototype we have developed is aimed at French native speakers writing in English. From the very start, we worked on the idea that our prototype would be commercialized. In order to find out users' real needs, we first conducted a survey among potential users and experts in the field concerning such issues as coverage and the interface. In addition, we studied which errors needed to be dealt with. To do this, we integrated the information on errors found in published lists of typical learner errors, e.g. Fitikides (1963), with our own corpus of errors obtained from English texts written by French native speakers. Some 27'000 words of text produced over 2'800 errors, which were classified and sorted. We have used this corpus to decide which errors to concentrate on and to evaluate the correction procedures developed. The fol lowing two tables give the percentages of errors found in our corpus, broken down by the major categories, followed by the subcategories pertaining to the verb. Error type percentage spelling & pronunciation 10.3 % adjectives 5.3 % adverbs 4.4 % nouns 19.6 % verbs 24.5 % word combinations 8.3 % sentence 27.6 % Table 1: Percentage of errors by major categories Error type percentage morphology 1.2 % agreement 1.4 % tenses 8.8 % lexicon 8.0 % phrase following the verb 5 . 1 % Table 2: Percentage of verb e r r o r s by subcategories The prototype includes a set of writ ing aids, a problem word highl ighter and a grammar checker. The writing aids include two monolingual and a bilingual dictionary (s imulated) , a verb conjugator , a s m a l l translating module for certain fixed expressions, and a c o m p r e h e n s i v e o n l i n e grammar. The problem word highlighter is used to show all the potential lexical errors in a text. While we hoped that the grammar checker would cover as many different types of errors as possible, it quickly became clear that certain errors could not be handled satisfactorily with the checker, e.g. using library (based on French librairie) instead of bookstore in a sentence like I need to go to the library in order to buy a present for my father. Instead of flagging every instance of library in a text, something other grammar checkers often do, we developed the problem word highlighter. It allows the user to view all the problematic words at one glance and it offers help, i.e. explanat ions and examples for each word, on request . Potential errors such as false friends, confusions, foreign loans, etc. are tackled by the highlighter. The heart of the prototype is the grammar checker, which we describe below. Further details about the writing aids can be found in Tschumi et al. (1996). 2 T h e g r a m m a r c h e c k e r Texts written by non-native speakers are more difficult to parse than texts written by native speakers because of the number and types of errors they contain. It is doubtful whether a complete parse of a sentence conraining these kinds of errors can be achieved with today's technology (Thurmair 1990). To get around this, we chose an approach using island processing. Such a m e t h o d , similar to the chunking approach described in Abney (1991), makes it possible to extract from the text most of the information needed to detect errors without wasting time and resources on trying to parse an u n g r a m matical sentence fully. Chunking can be seen as an intermediary step between tagging and parsing, but it can also be used to get around the problem of a full parse when deal ing with ill-formed text. Once the g rammar checker has been actived, the text which is to be checked goes through a number of stages. It is first segmented into sentences and words. The individual words are then looked up in the dictionary, which is an extract of CELEX (see Burnage 1990). It includes all the words that occur in our corpus of texts, with all their possible readings. For example, the word the is listed in our dictionary as a determiner and as an adverb; table has an entry both as a noun and as a verb. In this sense, our dictionary is not a simplified and scaled-down version of a full dictionary, but simply a shorter version. In the next stage, an algorithm based on neural networks (see Bodmer 1 9 9 4 ) d i s a m b i g u a t e s all words which belong to more than one syntactic category. Furthermore, some mul t i -word units are identified and labeled with a single syntactic category, e.g. a lot of(PRON) I. After this stage, island parsing can begin. A first step consists in identifying simple noun phrases. On the basis of these NPs, preproeessing automata assemble complex noun phrases and assign features to them whenever possible. In a second step, other preprocessing automata identify the verb group and assign tense, voice and aspect features to it. Finally, in a third step, error detection automata are run, some of which involve interacting with the user. Each of these three steps will be described below. 3 The preprocessing stage The noun phrase parser identifies simple n o n r e c u r s i v e noun phrases such as Det+Adj+N or N+N. The method used for this process involves an algorithm of the type described in Church (1988) which was trained on a manually marked part of our corpus. The module is thus geared to the particular type of second language text the checker needs to deal with. The resulting information is passed on to a preprocessing module consisting of a number of automata groups. The automata used here (as well as in subsequent modules) are f inite-state automata similar to those described in Allen (1987) or Silberztein (1993). This type of automata is wel l -known for its efficiency and versatility. In the preprocessing module, a first set of automata scan the text for noun phrases, identify the head of each NP and assign the features fo r person and number to it. Other sets of automata then mark temporal noun phrases, e.g. this past week or six months ago. In a similar fashion, some prepositional phrases are given specific features if they denote time or place, e.g. during that period, at the office. Still within the same preprocessing module, some recursive NPs are then assembled into more complex NPs, e.g. the arrival of the first group. Finally, human NPs are identified and given a special feature. This is illustrated in the following automaton: