## Sven Tarp*

## The usefulness of different types of articles in learner's dictionaries

Review of 'Ursula Wingate: The Effectiveness of Different Learner Dictionaries: An Investigation into the Use of Dictionaries for Reading Comprehension by Intermediate Learners of German . Tübingen: Niemeyer 2002.

0. Content
1. Presentation
2. First part
4. 2.1 The experiment
5. 2.2 The think-aloud study
6. 2.3 Some comments
3. New definitions
4. Second part
9. 4.1 The quantitative analysis
10. 4.2 The main think-aloud study: Qualitative analysis
11. 4.3 The bilingual dictionary
12. 4.4 The final experiment
5. Recommendations
6. Conclusion
7. Bibliography
16. 7.1 Dictionaries
17. 7.2 Other literature

* Sven Tarp Center for leksikografi Handelshøjskolen i Århus Fulesangs allé 4 DK-8210  Aarhus V

215

## 216

## 1. Presentation

Ursula Wingate's book The Effectiveness of Different Learner Dictionaries: An Investigation into the Use of Dictionaries for Reading Comprehension by Intermediate Learners of German is the result of a research work conducted by the author within the framework of a thesis concerned with the following main questions:

- Which type of dictionary, a monolingual or a bilingual one, provides most assistance to intermediate foreign-language learners in terms of text reception?
- What features make monolingual dictionary definitions more understandable for intermediate foreign-language learners?

The first of these two questions has the specific pedagogical aim to develop 'some evaluation criteria for dictionaries' in order to determine 'which dictionary can be recommended as the most effective one for intermediate learners' (p. 10). The second question aims, among other things, 'to contribute empirically-based principles for monolingual dictionary definitions to lexicographic theory' (p. 9).

As will be seen, the methodology chosen for the thesis makes it difficult to answer the first of the two questions in a convincing way, whereas the thesis provides a number of valuable indications concerning the second question.

The starting point for the thesis is both interesting and highly peculiar: It has been observed that Chinese students of German in Hong Kong have a strong preference for bilingual German-English dictionaries instead of - as it might be supposed - bilingual German-Chinese dictionaries or monolingual German learner's dictionaries.

The book consists of two main parts:

In the first part, the author wants to establish a) whether the above observation indeed is typical for Chinese foreign-language students, b) whether the German-English dictionary is really as effective as perceived by the students, and c) whether the monolingual German learner's dictionary is really so difficult as the students assume or whether their judgement is rather based on prejudices.

In the second part, the author makes a comparison between a 'traditional' definition - based on the principles of genus proximum and differentia specifica - and a so-called 'new definition' - especially developed on the basis of the philosophy of the natural-language definitions of the Collins Cobuild English Dictionary for Advanced Learners -  and  then  proceeds  to  determine which features  make the definition more effective for the intermediate learners of German.

## 2. First part

After a review of the theoretical literature and a profound introduction to the methodology chosen for the thesis, Ursula Wingate goes on to answer the first question, i.e.  whether  it  is  a  general  phenomenon  that  Chinese  foreign-language students prefer bilingual dictionaries with English as the target language. This is done by means of a questionnaire. As the number of intermediate students of German in Hong Kong in relatively limited, Ursula Wingate directs her questions not only to students of German, but also to intermediate students of French. The answers are unequivocal: The students don't use the monolingual German dictionary because they think that it is too difficult and that they can't understand the defining vocabulary. As regards a bilingual dictionary with Chinese as the target language, their main argument against the use of such a dictionary is that Chinese is too different from German/French and that translations are often incorrect. The main reasons why they prefer a dictionary with English are that it is more comfortable and convenient to use, that English and German/French are related languages and that translations, thus, are more accurate. Hence, the vast majority of the Chinese foreign-language students actually prefer a bilingual dictionary with English although their proficiency level in English is far from the corresponding one in their mother tongue, i.e. Chinese. The author concludes that the dictionary situation for foreign-language learners in Hong Kong is far from satisfactory.

At this point it must be noted that the above problematic is, indeed, very specific. It goes without saying that it constitutes a real problem that has to be solved by lexicography and that there might be other languages with similar problems. But above this level of similarity it is hardly possible to generalise the conclusions drawn by Ursula Wingate when she goes on to compare the usefulness of the monolingual German and the bilingual German-English dictionary. In this regard, the scope of her research is relatively limited, as she herself also admits.

## 2.1. The experiment

After analysing the results of the questionnaire, Ursula Wingate performs an experiment in order to determine whether the bilingual German-English dictionary is actually more helpful than the monolingual German dictionary. The subjects or informants were given a German text with a number of target words that they were supposed not to know. Half of them were allowed to consult a monolingual dictionary and the other half to consult a bilingual dictionary in order to verify the meanings of the unknown words. Their reading comprehension and incidental vocabulary learning were then tested be means of a) a supply-definition test where they had to supply a translation in English or a defini- tion for the target word, b) a multiple-choice test in which they had to tick the right answer from four choices and c) an immediate recall protocol where they had to write down everything they remembered immediately after reading the text.

There is no doubt at all that it is important to test the reading comprehension and incidental vocabulary learning in terms of the usefulness of learners' dictionaries. Nevertheless, there seems to be some theoretical confusion here. One of the most promising things about Ursula Wingate's book is its subtitle. Although lexicographic theory has been developed during the past decades, it is still rare to encounter a theoretical work that does not only refer to user needs in general, but also to a specific user group and a specific user situation where the sort of problem pops up that originally give rise to the whole lexicographic consultation process. It is the great merit of Ursula Wingate that she emphasises this particular aspect and does it already in the subtitle. The user group dealt with in the research project are (Chinese) intermediate learners of German and the user situation is reading comprehension (text reception). This relates to dictionaries whose lexicographic function is to assist Chinese intermediate learners of German solving the complex of problems they might have when reading German texts.

Incidental language learning is, however, not a component part of this function.  As the author rightly indicates in the theoretical part of her book, language learning is a continuum. It might take its starting point in text reception (passive confrontation/interaction with the language), proceed through word understanding and text comprehension and then reach the level of vocabulary learning and general assimilation of the language system (first passive and then active). It might, however, also start with text production (active confrontation/ interaction with the language) - directly in the foreign language or through the mother tongue - or even with a conscious study of the foreign-language system, i.e. lexis and grammar, and most frequently it takes place as a combination of all these processes (see Tarp 2003a, 2003b). After reading Ursula Wingate's book, it becomes evident that she is very much in debt to the British lexicographic tradition which is characterised by its lack of a general theory of lexicography and its disregard of the lexicographic functions when discussing any particular topic or data included in dictionaries. Vocabulary learning is not part of text comprehension but constitutes a further step in the overall foreign-language learning process. Ursula Wingate herself admits that 'a by-product of reading is incidental vocabulary learning' (p. 14). Text comprehension doesn't always lead to vocabulary learning and it shouldn't necessarily do so. Foreignlanguage learning implies the confrontation with a large amount of texts containing many unknown words that hamper the comprehension of the text. This is  exactly the kind of problem that can be solved by means of a dictionary. Some of the unknown words might be rare words that will pop up very few times or maybe only once, whereas others might be more frequent with multiple exposures. The human brain has to concentrate on the most important tasks and it is evident that it is easier - and more important - to remember (learn) the most frequent words, especially for learners at a beginner's or intermediate level. This is probably the reason why some learner's dictionaries, e.g. COBUILD and MACMILLAN, have indicated frequency by all lemmata. To conclude this escapade it should, therefore, be noted that vocabulary learning is not a criterion for whether or not a learner's dictionary for reading comprehension fulfils its function (the declared purpose of the thesis), but is a further criterion that, of course, has very much to do with the quality of the dictionary.

The results of the experiment carried out by Ursula Wingate in order to determine the effectiveness of monolingual and bilingual dictionaries in terms of both reading comprehension and incidental vocabulary learning are interesting, but not conclusive. There is a clear tendency that the bilingual GermanEnglish dictionary is more effective for learners - both at the lower and upper intermediate level - than the monolingual German one. But this tendency is not - according to the author - statistically significant (conclusive). On the other hand, this also means that the monolingual dictionary is not significantly less helpful than the bilingual one, a conclusion that contradicts the general opinion expressed by the Chinese intermediate learners of German.

## 2.2. The think-aloud study

Ursula Wingate then combines the above tests with a so-called 'think-aloud study' where the informants are encouraged to think aloud during the dictionary consultation process and in presence of the researcher in order to determine which features make the dictionaries more or less effective for the users. This study, however, is hampered and limited in its scope by the fact that there were only two informants for each type of dictionary and an average of 25,5 words looked up by each informant, a fact that provides even less statistic value to the results. There are, anyway, a number of interesting observations regarding successful  and  unsuccessful look-ups that have implications for the further research.

In the bilingual dictionary, there were a total of 17 unsuccessful look-ups due to the following categories of problems:

Table 1: Categories of unsuccessful look-ups in the bilingual dictionary

| Category                                                  |   Number of occurrences | Percentage   |
|-----------------------------------------------------------|-------------------------|--------------|
| 1. Reading only part/beginning of dictionary entry        |                       6 | 35.3%        |
| 2. Failing to find the right entry                        |                       3 | 17.6%        |
| 3. Ignoring or misunderstanding symbols and abbreviations |                       3 | 17.3%        |
| 4. Not knowing the English equivalent                     |                       3 | 17.6%        |
| 5. Insufficient dictionary information                    |                       2 | 11.8%        |

This table shows that 88.2 percent (categories 1, 2, 3 and 5) of the unsuccessful look-ups are due to 'technical' problems that can be interpreted both as a bad dictionary culture among the users and as a bad dictionary design in terms of lexicographic structures, symbols and abbreviations. Only 11.8 percent (category 4) have to do with the limitations related to the Chinese students' proficiency level in English. As for the reasons for a successful consultation of the bilingual dictionary, Ursula Wingate writes:

'With non-target words included, there were 37 words which were clearly identified from the think-aloud protocols as understood. One strong factor for success could be identified, i.e. the position of the appropriate word sense in the dictionary article. For 18 of the words which were successfully looked up, the suitable meaning was the first in the dictionary entry, and for another 5 words the suitable meaning was the second. For at further 8 words, the entries contained only one translation equivalent. That means that for 31 out of 37, or 83.8 percent of successful look-up actions, a major factor was that the subjects only had to read the beginning of the entries.' (p. 114)

This conclusion corresponds to the above information that 35.3 percent of the unsuccessful look-ups were due to the fact that the informants only read a part / the beginning of the dictionary article. This once more testifies to either the users' bad dictionary culture or an inappropriate search-field structure of the dictionary articles. It suggests at least (knowing the limited statistic evidence) that the bilingual dictionary could be improved if a more convenient searchfield structure was chosen together with a more appropriate access structure and use of symbols and abbreviations. The author, however, doesn't draw this conclusion with its subsequent consequences.

In the monolingual dictionary, there were a total of 24 unsuccessful lookups due to the following categories of problems:

Table 2: Categories of unsuccessful look-ups in the monolingual dictionary

| Category                                 |   Number of occurrences | Percentage   |
|------------------------------------------|-------------------------|--------------|
| 1. Difficulties with defining vocabulary |                      10 | 41.7%        |
| 2. Derivational definition               |                       3 | 12.5%        |
| 3. Definition structure                  |                       1 | 4.2%         |
| 4. Failing to find the right entry       |                       4 | 16.7%        |
| 5. Reading only beginning of entry       |                       1 | 4.2%         |
| 6. Others                                |                       5 | 20.8%        |

Table 2 shows that 58.2 percent (categories 1, 2 and 3) of the unsuccessful look-ups have to do with the definition and its defining vocabulary, whereas other 16.7 percent (category 4) have to do with the access structure. As for the successful consultation of the monolingual dictionary, Ursula Wingate writes that 'that there is also clearly one main factor':

'For 11 of the 18 words which were identified as understood from the think-aloud protocols, the definition contained a segment which was familiar to the subjects and could be substituted as an equivalent for the target word.' (p. 117)

## 2.3 . Some comments

On the basis of the above results of the experiment and the think-aloud study, Ursula Wingate concludes that an effort should be dedicated to the improvement of the definitions in the monolingual learner's dictionary. But her argumentation is developed in such a strange way that it invalidates her first main purpose with the book, i.e. to determine whether a monolingual or a bilingual dictionary is most effective for intermediate learners of German. Already in the beginning of the book, she contradicts this purpose:

'Despite a lack of research evidence for the superiority of the monolingual over the bilingual dictionary, in this research the position is taken that it is desirable to introduce the monolingual dictionary as early as possible in the language acquisition process, especially for reading comprehension.' (pp. 9-10)

Here, she somehow provides the answer before putting the question. After the experiment and the think-aloud test she maintains this line of argumentation:

'Giving the unwillingness of most students to use the German-Chinese bilingual dictionary and the fact that the German-English dictionary does not yield much better results than the monolingual one, perhaps (sic!) the use of the monolingual dictionary should be encouraged. As discussed in 2.4.1., the majority of educators believe in the superiority of the monolingual dictionary over the bilingual, albeit without much empirical evidence for that superiority. In this learning context, it can be argued that there is a sound reason for recommending the monolingual dictionary at least for reading. If the monolingual would replace the German-English bilingual, the unsatisfactory situation in which the students deal with three languages and possibly two translations in order to find the meanings of unknown words would be eliminated. In addition, if the cultural difference between Chinese and German is indeed so great that for many words there is no suitable translation equivalent, then this is an even stronger argument for the use of the monolingual dictionary which does not rely on translations, but explains words in their own cultural context. Because of these arguments, the focus in the second part of the research was entirely on monolingual dictionaries.' (p. 121)

Recommendations of this kind are sympathetic and quiet understandable from the point of view of language didactic. But one of the pedagogical aims of Ursula Wingate's thesis was exactly to provide research evidence for such recommendations. Hence, in this respect she hasn't met the expectations. If one analyses the above quotation, at least two other directions for the further research could be pointed out. First of all, the bilingual German-English dictionary could be improved and even adapted to the specific needs of the Chinese students. Secondly, a research could be done in the factual problems of the German-Chinese dictionary in order to determine whether the problems mentioned by the Chinese students are real and insurmountable problems or whether this type of dictionary could actually be improved as well. In this regard, it would be interesting to know their experiences with English-Chinese dictionaries when they were at a similar proficiency level as learners of English. In a critical review of the existing  Afrikaans learner's dictionaries, Gouws/Tarp 2003 have indicated that there are a least five basic ways of explaining foreign-language words in a learner's dictionary:

- -explanations in the foreign language
- -provision of text examples in the foreign language
- -illustrations
- -explanations in the mother tongue
- -a cumulative selection of equivalents in the mother tongue
- -or a combination of two or more of the above ways.

All of these possibilities should actually be explored in the research instead of only concentrating on the traditional apposition: explanations in the monolingual dictionary and equivalents in the bilingual one. Until this is done, it is hardly possible to draw any empirically-based conclusions whether bilingual or monolingual dictionaries should be recommended to Chinese intermediate learners of German. Ursula Wingate seems to recognise this limitation when she, at the end of the book, recommends that further investigation should be made in illustrations in learner's dictionaries and in 'the effectiveness of bilingual dictionaries, especially those where words are explained in the mother tongue' (p. 238).

Another thing that makes it difficult to close the discussion at this point is the fact that only two dictionaries - the bilingual Langenscheidts New College German Dictionary, German-English ,  and  the  monolingual Langenscheidts Großwörterbuch Deutsch als Fremdsprache (LGDAF) - were used in the above experiment. The use of only two dictionaries constitute a very weak basis for scientific conclusions, and this weakness is further enhanced by the long German tradition of compiling complex and condensed dictionaries that are very little  user-friendly.  Ursula  Wingate  comments  that  neither  the Pons Basiswörterbuch Deutsch als Fremdssprache nor the de Gruyter Wörterbuch Deutsch als Fremdsprache - both monolingual learner's dictionaries - were yet published at the time when the research was carried out and that LGDAF, thus, was the only monolingual German learner's dictionary available for the experiment. Apparently, the language spoken in the former German Democratic Republic was not German as there is no reference to the monolingual learner's  dictionary Grundwortschatz der deutschen Sprache ,  published  by Kosasras István in 1983. It could, however, have been interesting to include the latter in the research as it was conceived as a dictionary for text comprehension and contains simple definitions that are frequently accompanied by illustrations. Maybe this would have provided strong arguments for the monolingual track that the author follows in the second part of her book.

## 3. New definitions

Before proceeding to the second part of her research work, Ursula Wingate describes the process of elaborating new definitions (explanations) conceived for a monolingual German learner's dictionary. The changes concern both the definition style and the structures of the dictionary article (entry). With respect to  the  definitions,  Ursula Wingate is, as already mentioned, inspired by the principles developed and put into practice by COBUILD - and for good reasons. It must be admitted that it is nice for once to be able to say something positive about the British lexicographic tradition. Although the famous English learn- er's dictionaries suffer from fundamental problems regarding text production, they have all advanced a lot in terms of text reception and are, in this regard, among the very best in the world. This is, among other things, due to a rich and engaged theoretical discussion combined with the innovating practical work performed by the compilers of the dictionaries. COBUILD seems to be the favourite among most lexicographers although there are also quite a number of critical voices to be found. Its definitions are based on the principles of naturalspoken language, full sentences and a reduced defining vocabulary. It seems, thus, logical to adapt these principles to the German reality for the sake of the present research. On that basis, Ursula Wingate writes new definitions for 61 target words where 'the number and order of word meanings were taken over from LGDAF'. The reason for this was that 'since the aim of the research was to compare the effectiveness of the defining styles, the change of other information types could have affected the results' (p. 141). However, Ursula Wingate also introduces profound changes in the structures of the articles modelling them 'on the format of COBUILD's entries':

'Each meaning of polysemous words is presented in its own paragraph. Idiomatic expressions are listed and numbered as meanings of the headword. In the definition sentences, the headword appears in bold print. The definitions follow by the unchanged information from the LGDAF entries, i.e. collocations, examples, and compounds of the headword. However, grammatical and usage information is presented in an extra column on the right side of the main text.' (p. 141)

In this way, Ursula Wingate has not only changed the definition styles, but also introduced an advanced search-field structure and, thus, changed the internal access structure in the dictionary articles. Now the question is whether these changes are actually improvements. As mentioned, there have been a lot of arguments for and against COBUILD's defining style, but no field research has yet been performed in order to determine how the users, the sine qua non of all dictionaries, respond to the various defining styles and article structures. This is what Ursula Wingate pretends to do in the second part of her book.

## 4. Second part

In the second part of the book, Ursula Wingate submits the new definitions to various tests. This part of the book is by far the most interesting. It consists of the main think-aloud study followed by another, improved experiment. The most important topic is to compare the two different types of monolingual dictionary definitions - the new definitions and the traditional definitions based on genus proximum and differentia specifica - but the think-aloud study also includes informants using the bilingual dictionary in order to 'triangulate this study with the previous ones' (p.145). The author performs both a quantitative and a qualitative analysis of the results.

## 4.1. The quantitative analysis

The quantitative analysis is based on the written transcripts of the think-aloud study, a supply-definition test, a multiple-choice test and an immediate recall protocol and is sought to provide answers to the following questions:

1. Do subjects using the new definitions understand the meaning of more unknown words than those using the LGDAF or the bilingual dictionary?
2. Do subjects who use the new definitions learn more words incidentally than those who use the LGDAF or the bilingual dictionary?
3. Do subjects who use the new definitions comprehend the texts better than those who use the LGDAF or the bilingual dictionary?

The answers to the first question were as follows:

Table 3: Number and means of the percentage of the words successfully/unsuccessfully looked up

| Type of definition and total no. of look-ups   | Understood       | Not understood   |
|------------------------------------------------|------------------|------------------|
| New definitions: 133                           | 78 words: 57.87% | 37 words: 27.17% |
| LGDAF definitions: 124                         | 50 words: 40.27% | 67 words: 54.98% |
| Bilingual definitions: 127                     | 62 words: 48.48% | 49 words: 39.00% |

These findings give answer to the first question and show that the users of the new definitions did, in fact, understand the meaning of more unknown words than the users of the other two options. However, the numbers of informants six for the new definitions, six for the LGDAF and five for the bilingual dictionary - 'is too small to prove statistically that the differences are significant' (p. 146). On the other hand, the results of the vocabulary and reading comprehension tests showed a 'rather irregular pattern' and Ursula Wingate has to conclude that 'this suggests that vocabulary and reading comprehension tests may not be reliable measures of dictionary effectiveness' (p. 147). One of the reasons for this is, for instance, that the immediate recall protocols

'requires strategies quite opposite to those in which the learners have been trained in their foreign language classes. They are usually encouraged to understand the global meaning of texts and to avoid getting stuck with unknown words unless they are crucial for understand- ing. The recall protocol, however, requires attention to every unit, and the recall of as many details as remembered.' (p. 109)

All this meant that the results of the quantitative analysis didn't provide answers to the two last of the above questions and it also led to the changing of the testing method in the subsequent experiment.

## 4.2. The main think-aloud study: Qualitative analysis

After discussing the results of the quantitative analysis, Ursula Wingate goes on to present the results of the qualitative analysis of the main think-aloud study. The main purpose of this analysis is to provide answers to the following two questions:

- Which features make monolingual dictionary definitions effective for the intermediate learners?
- Which features make monolingual dictionary definitions ineffective for the intermediate learners?

Although the group of informants, as already mentioned, was very limited (6+6+5), the results are in any case highly interesting and instructive. The most important categories of findings in terms of successful look-ups in monolingual dictionary conditions were as follows:

1. The definition phrase contains a synonym or near-synonym (Ursula Wingate calls them equivalents) which is familiar to the users and can be substituted for the target word. These synonyms or near-synonyms can be either a word or a short phrase.
2. The dictionary article provides a synonym that is indicated explicitly and doesn't form part of the definition phrase. If the users are familiar with this synonym, they can substitute it for the target word.
3. Additional data, i.e. redundancies and examples, are integrated in the definition text. This makes the definitions context rich or 'imageable': 'A common feature in the think-aloud protocols for this Category was that the subjects either mentioned a redundancy or an example as helpful, or found the meaning of the target word immediately after having read the redundancy or example. In some cases when the target sense of a word was abstract or figurative, the subjects seemed to access the abstract or figurative concept through the more imageable concept of the direct sense' (p. 157).

Concerning the two first categories, it became clear that 'the learners search for a short phrase of word which can be substituted for the target word', but that 'if the surrounding text is linguistically too difficult, the effectiveness of equivalents and synonyms is lost' (p. 156). Although the new definitions, as a whole, were more effective than the LGDAF definitions, there were nevertheless some cases where the latter showed up to be more effective than the former. One of the reasons for this was that 'it seems to be easier to identify equivalents and synonyms in the structure of the LGDAF definitions' (p. 165). This finding corresponds with the argument put forward by Hausmann/Gorbahn 1989 and quoted by Ursula Wingate. They warned that COBUILD's 'natural-spoken language' style had the

'…disadvantage… that the essential parts of the definition…, by not being set off through print, tend to be lost, while in the authentic classroom situation they would be emphasised by the teacher's intonation.' (Hausmann/Gorbahn 1989:48)

This indicates that it might be advantageous if synonyms and quasi-synonyms within the definitions were typographically highlighted, but this is a topic for future research. After concluding the above analysis of the successful lookups, Ursula Wingate describes to the reasons for the unsuccessful look-ups in monolingual dictionary conditions. In this respect, the most important categories of findings are as follows:

- Difficulty to reduce full-sentence definitions into one-word equivalents. In this case, the synonyms for the target word could not be identified as easily in the full-sentences structure of the new definitions as in the traditional LGDAF definitions which frequently begin with a superordinate that might be familiar to the user. If the latter is the case, 'it gives them access to the semantic field of the word, and they can presumably understand more easily the other meaning components of the word which are provided in the differentia specifica' (p. 168).
- The definitions (in LGDAF) are linguistically complex in terms of a) a difficult vocabulary, b) a complex syntactic structure and c) a condensed text. The protocols showed mainly evidence for the first of these features, i.e. the definition contained one of several words unknown to the learners. However, in almost all the cases, 'where the subjects stated that they could not understand the meaning because of unknown words, the definitions also had difficult syntactic structures and condensed test' (p. 170).
- Derivational definitions where the root word is used as an important part of the definition. 'This type of definition is destined to frustrate learners at the intermediate level, because it is usually the root word which they do not know' (p. 174).
- LGDAF's unrestricted defining vocabulary is a major obstacle. Altogether, 40.3 percent of the unsuccessful look-ups in LGDAF were at least partly

due to unknown vocabulary. 'This is a clear indication that LGDAF's defining vocabulary is beyond the lexical competence of intermediate learners. The defining vocabulary of a learners' dictionary must be restricted to a certain extent if the dictionary is to be used by intermediate learners' (p. 178).

A part from these main categories, there were also a couple of examples where the informants couldn't recognise the grammatical form (e.g. they confused the noun Treten with the verb treten ). Another reason for unsuccessful look-ups had to do with the informants' look-up strategies, the most interesting of which was that the informants only read part of the dictionary article. As one of the principles of the new definitions was an improved search-field structure, 'it is not surprising that there were more incidents of LGDAF subjects giving up reading after the first meaning, as the run-on entries in the LGDAF are harder to read' (p. 179). By doing so, the informants also missed all further data on the word such as collocations and examples. Ursula Wingate comments:

'This is contrary to the expectation of the LGDAF's editor who assert in the 'Instructions for Users' that the 'real' meaning of the headwords is to be derived not from the definitions alone, but from the definitions in combination with the additional information… In reality, however, learners seem to make little or no use of the additional information. In this respect it is a positive feature of the new definitions that the examples are included in the definition text'. (p. 179)

Fundamentally, this problem is, of course, due to a bad dictionary culture but the interesting conclusion which can be drawn from the above analysis is that the dictionary actually can compensate for this bad culture. A final reason for unsuccessful look-ups, that is worth mentioning, had to do with LGDAF's presentation of derived nouns which were 'listed under the root word without their own meaning presented, even when the meaning of the noun cannot be derived from that root word so easily' (p. 181).

Although there were only six informants for each type of monolingual dictionary and the results might be modified by a larger group of informants, there is no doubt that Ursula Wingate, with the above results of the qualitative analysis of the think-aloud study, makes a relevant empirically based contribution to the theory and practice of definitions and article structures in learner's dictionaries. The theoretical debate has hitherto suffered from the fact that there was little or no empirical basis for the arguments and opinions put forward by the various lexicographers. The results of Ursula Wingate's qualitative analysis provide - within the statistic limits - evidence for some of the arguments and suspicions, dispute others and raise a number of new questions to which little or no attention has been paid so far.

## 4.3. The bilingual dictionary

As regards the bilingual dictionary, the results of the qualitative analysis of the think-aloud study - whose purpose was to 'triangulate this study with the previous ones' - confirmed the findings from its predecessors. The 'successful or unsuccessful consultation of the bilingual dictionary was, to a much larger extent, due to appropriate or inappropriate strategies than is the case for the monolingual dictionaries' (p. 183). The majority of the words that were successfully looked up had only one meaning, or the appropriate meaning was the first one listed in the article. On the other hand, the most important reasons for unsuccessful look-ups were a) that the informants only read the first meaning of the article, b) that the informants failed to find the right entry and c) that they didn't know the English equivalents. Apart from this, they were also due to some basic weaknesses of the dictionary in question (LNCGD). Ursula Wingate concludes:

'The examples show that even a bilingual learners' dictionary, which is usually regarded as the easiest option for users, presents a number of difficulties. Especially, by not providing all the meaning of words the lexicographers overestimate the skills of learners. It means that the learners have to extract a common concept or 'core of meaning' from the equivalents given, and try to integrate it into the source text. The results from the words 'Aufklärung' and 'Umstellung' demonstrate that this exercise is too demanding for intermediate learners. The list of equivalents should include all meanings of the word, and also the most common forms in which words appear.' (p. 188)

These conclusions are important, but it would, as already mentioned, have been more interesting if the author herself had taken the above recommendations into account and also improved the bilingual dictionary in order to provide a more convincing answer to the first main question of her book, i.e. whether the monolingual or the bilingual dictionary is more suitable for the learners at an intermediate level.

## 4.4. The final experiment

The above think-aloud study generated a number of hypotheses in terms of the effectiveness of the two different types of monolingual dictionary definitions. As both groups of informants consisted of only six students, these hypotheses had to be confirmed by evidence from a larger sample. Hence, in order to test them and measure the effectiveness of the definition types quantitatively, a final experiment was conducted by Ursula Wingate. This experiment was modified in the light of the experiences from the first experiment. It included a total of 42 informants for each type of dictionary definition, some of them from mainland China. A total of eight hypotheses were put forward, two of them of more general character and five of them related to specific features that had been revealed during the think-aloud test. The last hypothesis was an additional one. For all of them, except the two general ones, a number of target words - from one to four - were tested for each dictionary type. The hypotheses and results of the test are as follows:

Hypothesis 1: The new definitions are more effective for intermediate learners than the LGDAF definitions.

This important hypothesis was confirmed as the informants using the new definitions understood 'significantly more target words than the LGDAF users' (p. 198).

Hypothesis 2: The new definitions are particularly effective for intermediate learners of low verbal ability.

This hypothesis had to be rejected, as there was no significant difference between the users of the two definition types. The research 'revealed that the significant difference observed in the overall comparison was due to the high verbal ability group' (p. 199). However, not even in this group the 'subjects using the more user-friendly new definitions understood the meaning of half of the target words' (p. 209). From this fact Ursula Wingate draws a dramatic conclusion:

'High verbal ability students are on the threshold of being able to use a monolingual dictionary in the style of the new definitions, which is linguistically not as difficult as the LGDAF, while the weaker students at this proficiency level are not yet ready to use even this more userfriendly type of monolingual dictionary.' (p. 209-210)

This conclusion - if it proves to be right in future research - must, without any doubt, be considered one of the most important results of Ursula Wingate's research. However, here it should be noted that the term 'intermediate learner' is defined as a foreign-language student that has had more than 450 and less than 800 hours of instruction, before and after which he or she is considered a beginner or an advanced learner respectively. This mechanical way of defining an intermediate learner makes it difficult to draw comparisons between the Hong Kong learners and learners that are subject to another educational system. It is, for instance, not the same to be 4 or 40 students in the classroom, to have a lot or little homework, to follow a study program that is teacher-orientated or one that is case-orientated outside the classroom and without the presence of the teacher, etc. In this respect, Ursula Wingate's conclusions in terms of proficiency level probably need to be relativated.

As already mentioned, the next six hypotheses deal with special features in the definitions:

Hypothesis 3: Synonyms in definitions are effective for intermediate learners. They are most helpful if they are not embedded in a fullsentence structure.

The results of this category were contradictory. The hypothesis had to be rejected and a modified hypothesis put forward which emphasised that the synonyms should be placed in an initial position in the definition, even if they are part of a full-sentence structure. This new hypothesis was confirmed by the results for two of the four target words. This, however, doesn't invalidate the idea that if a synonym 'is offered in the second part of a compound sentence, it should be somehow highlighted for the learner, for instance be underlining it' (p. 211), as it was not tested in the experiment.

Hypothesis 4: A rich definition context including redundancies and examples is effective for intermediate learners.

This hypothesis could be confirmed, as the difference between the users of the two definition types was 'highly significant' (p. 202).

Hypothesis 5: Noun definitions with an if-structure are ineffective for the intermediate learners.

The results of this category were contradictory and the hypothesis had to be rejected.

Hypothesis 6: Condensed text in definitions is ineffective for intermediate learners.

This hypothesis had to be rejected as there was 'no significant difference between LGDAF and new definition subjects' (p. 204). This result, which was only based on two target words, 'came as a surprise' (p. 213) and Ursula Wingate argues that other features in the definitions could have influenced the result and that further research therefore has to be conducted on this question.

Hypothesis 7: Derivational definitions are ineffective for intermediate learners.

'Highly significant results support this hypothesis' (p. 204). This conclusion, however, was based on results from only one target word.

(Extra) hypothesis 8: A clear entry format is especially beneficial for learners when less frequent senses of polysemous words have to be located.

Also this hypothesis had to be rejected, as there was no significant difference between the two groups of users, but this conclusion, once more, was based on only one word.

There is no doubt that the above results are extremely important for lexicographic research in learner's dictionaries. But it should also be clear that the results are not conclusive and that the experiment as such suffers from various important limitations. In her summary, Ursula Wingate herself comments upon the methodological limitations of the testing individual factors. The greatest limitation was

'that the factors can never be completely isolated, but work in combination with others. Another drawback was the limited number of target words available to test the hypotheses. Therefore, the conclusions drawn from this research context have to be verified by further research.' (p. 217)

As a whole, the experiment provided quantitative evidence that the new definitions inspired by the COBUILD definition style are more useful for intermediate learners than the traditional definitions in LGDAF. In this way the experiment confirmed the previous think-aloud study.

There were, however, two basic features of the new definitions that could not be isolated for testing because they were inherent in all definitions. These were the restricted defining vocabulary and the use of full-sentence definitions. Ursula Wingate assumes that the significantly higher success rate of the new definitions reflects that 'the restricted defining vocabulary and full-sentence structure are effective features in definitions' (p. 217). Nevertheless, she points out that the fact that, in the think-aloud study, about 40 percent of unsuccessful look-ups in the LGDAF were due to vocabulary problems, is a direct evidence for the effectiveness of the controlled vocabulary of the new definitions, whereas 'in the present research format the effectiveness of full-sentences explanations remains a hypothesis' (p. 217).

## 5. Recommendations

On the basis of the results from the main think-aloud study and the above experiment, Ursula Wingate provides four so-called 'preliminary lexicographic recommendations… for the design of effective definitions for intermediate learners':

1. A restricted defining vocabulary should be used for the definitions.
2. Redundancies and examples should be integrated into the definitions.
3. No derivational definitions should be used in dictionaries for intermediate learners.
4. If a word can be explained by a synonym, this synonym should be stated in initial position.

Ursula Wingate points out that 'because of limitations in this study, future research is needed to confirm these recommendations' (p. 218). It is, however, important not to forget other recommendations such as to avoid a condensed definition language and to devise an appropriate search-field and access structure in the dictionary article. Although these two recommendations were not confirmed directly by the experiment, this could, as the author rightly points out, be due to the interference of other features and the limitations of her study.

## 6. Conclusion

Until now, nothing has been said whether Ursula Wingate's book is good or bad, whether it should be recommended or not. However, the space that this article has dedicated to comment upon it should speak for itself. Her book is most inspiring. Although there are weaknesses and limitations of both theoretical  and  methodological kind, there is, on the other hand, no doubt that she provides valuable material for the further research in learner's dictionaries. As she herself has demonstrated with the elaboration of the 'new definitions' in German, the future research must be 'interactive' in the sense that it should not only engage in the observation of lexicographic practice, but should also contribute with new ideas - and dictionaries. This is what lexicography is all about.

## 7. Bibliography

## 7.1. Dictionaries

- COBUILD = John Sinclair (ed.): Collins Cobuild English Dictionary . London: HarperCollins Publishers 1995.
- DE GRUYTER = Kempcke, Günter (ed.): de Gruyter Wörterbuch Deutsch als Fremdsprache . Berlin / New York: Walter de Gruyter 2000.
- GDDS = Kosasras István: Grundwortschatz der deutschen Sprache. Berlin: V olk und Wissen Volkseigener Verlag 1983.
- LGDAF = Dieter Götz / Günther Haensch / Hans Wellmann (eds.). In collaboration with Vincent J. Dochertz and Günther Jehle: Langenscheidts Großwörterbuch Deutsch als Fremdsprache. Das neue einsprachige Wörterbuch für Deutschlernende. Berlin / München: Langenscheidt 1993.
- LNCGD = Sonia Brough (ed.): Langenscheidts New College German Dictionary, German-English. Berlin / München: Langenscheidt 1995.
- MACMILLAN = Michael Rundell (ed.): English Dictionary for Advanced Learners . London: Macmillan 2002.
- PONS = Dörthe Hecht / Annette Schmollinger (eds .): Pons Basiswörterbuch Deutsch als Fremdssprache . Stuttgart: Klett 1999.

## 7.2. Other literature

- Gouws, Rufus H. / Tarp, Sven 2003: Wie leer wat uit  Afrikaanse (aan)leerderwoordeboeke. In Tydskrift vir geestes wetenskappe . Pretoria: South African Academy. (In preparation)
- Hausmann, Franz Josef / Gorbahn Adeline 1989: COBUILD and LDOCE II. A comparative review. In International Journal of Lexicography vol. 2, no. 1 , pp. 44-56.
- Herbst, T. / Popp, K. (eds.): The Perfect Learners' Dictionary (?) . Tübingen: Niemeyer 1999. (= Lexicographica Series Maior 95 )
- Tarp, Sven 2003a: Reflections on dictionaries designed to assist the users with text production in a foreign language. In Lexikos 13 . Stellenbosch: Buro van die WAT. (In preparation)
- Tarp, Sven 2003b: Basic problems of learner's lexicography. In Lexikos 13 . Stellenbosch: Buro van die WAT. (In preparation)
- Wiegand, Herbert Ernst (ed.): Perspektiven der pädagogischen Lexikographie des Deutschen. Untersuchungen anhand von «Langenscheidts Großwörterbuch Deutsch als Fremdsprache». Tübingen: Max Niemeyer Verlag 1998. (= Lexicographica Series Maior 86 )
- Wiegand, Herbert Ernst (ed.): Perspektiven der pädagogischen Lexikographie des Deutchen II. Untersuchungen anhand des »de Gruyter Wörterbuch Deutsch als Fremdsprache« . Tübingen: Max Niemeyer Verlag 2002. (= Lexicographica Series Maior 110)
- Wingate, Ursula: The Effectiveness of Different Learner Dictionaries: An Investigation into the Use of Dictionaries for Reading Comprehension by Intermediate Learners of German . Tübingen: Max Niemeyer Verlag 2002. (= Lexicographica Series Maior 112 )
- Zöfgen, E.: Lernerwörterbuch in Theorie und Praxis. Ein Beitrag zur Metalexikographie mit  besonderer  Berücksichtung  des  Französischen .  Tübingen:  Niemeyer  1994. (= Lexicographica Series Maior 59 )