Proceedings of the 41st Annual Meeting of the English Language Education Society of Japan
日本英語教育学会第 41 回年次研究集会論文集

Automatic Generation of High-Frequency Vocabulary Lists and Level-Adjusted
Exemplar Sentences for Non-Native Speakers of English
アントニ
早稲田大学理工学術院英語教育センター
E-mail:ローレンス〒169-8555 東京都新宿区大久保 3-4-1

anthony@waseda.jp

概要
過去研究によると第 2 言語学習者にとって高頻度の単語の Direct Learning が効果的である。また、高頻度の単語に関して語彙知識の深さ（Depth)も重要である。この結果から高頻度の単語の学習に当たって、単純な L2-L1 語彙リストは明らかに不十分な学習道具である。高頻度の単語を学習する際、語彙の語源、派生、活用などに直面すべきであり、意味を持つ典型文も目にする必要がある。また、その典型文に含まれている語彙が適切な語彙レベルであることも１つの条件である。この論文では新しい語彙リストと典型文作成ツールを提案する。このツールにより、語彙アイテム、発音ガイド、適切な語彙レベルでの典型文などを自動的に作成し、すぐに印刷し、学生に配布できる形にアウトプットする。ツールの機能と設定が柔軟であり、生成されたリストも既に日本における大型の理工系英語教育プログラムで使用され、効果的な語彙学習教材であることが証明されている。

Automatic Generation of High-Frequency Vocabulary Lists and Level-Adjusted
Exemplar Sentences for Non-Native Speakers of English
Laurence ANTHONY
Waseda University, Faculty of Science and Engineering, Center for English Language Education
3-4-1 Okubo, Shinjuku-ku, Tokyo 169-8555, Japan
E-mail: anthony@waseda.jp
Abstract

Previous research on vocabulary learning has shown that non-native speakers of English benefit greatly from the direct
learning of high-frequency vocabulary items. It is also known that increasing the depth of the learners' knowledge ofhigh-frequency items is important. These findings suggest that simple L2-L1 vocabulary lists are insufficient. Instead,learners should be exposed to high-frequency items in a variety of forms and contexts, together with meaningful
exemplar sentences that are also at an appropriate vocabulary level. In this paper, I present a novel vocabulary list andexemplar sentence generator that automatically creates an attractively formatted table of vocabulary learning items,
pronunciation guides, and exemplar sentences at an appropriate learner level. The generator is highly customizable andthe lists produced by the tool have already proved successful in a very large English program for students of science andengineering in Japan.
more about what vocabulary items are important to

1. Introduction

learn and how vocabulary items at different

Vocabulary instruction has traditionally been afrequency levels should be taught. On the othercentral component of English language courses. In

hand, implementing these ideas in a real-world

recentvocabulary

EFL program has proven to be more problematic.

knowledge and the creation of large corpora of

For example, although it is now clear that students

general and specialized English, our understandingcanof vocabulary and its importance in English as a

vocabulary items using a direct approach [1],Foreign Language (EFL) instruction has increased

providing a contextual link between such lists ofdramatically. Today, applied linguists know muchhigh-frequency words and other learner materials

years,

with

new

tests

of

be

successfullytaughthigh-frequency

アントニ ローレンス, “Automatic GenerationofHigh-FrequencyVocabulary Lists andLevel-AdjustedExemplar Sentences for Non-Native Speakers of English”
日本英語教育学会第 41 回年次研究集会論文集, pp. 7-14, 日本英語教育学会編集委員会編集, 早稲田大学情報教育研究所発行, 2012 年 3 月 31 日.
This proceedings compilation published by the Institute for Digital Enhancement of Cognitive Development, Waseda University.
Copyright ©2011-12 by Laurence ANTHONY.
All rights reserved.

can be challenging. Also, it can be assumed thatpossiblelearners require a greater depth of knowledge ofsuggested.

high-frequency words than they do for low(i.e., inflections and derivations) and contexts.
Again, however, providing suitable materials thatdevelop vocabulary depth can be difficult.
In this paper, I will first briefly review thecurrent theories on vocabulary learning and discuss
the importance of high-frequency vocabulary items.
Next, I will present a novel tool for creating
vocabulary materials that addresses the current
challenges facing EFL materials developers. The
system presented here takes as input a pre-defined
list of vocabulary items, an upper bound of theuser's general vocabulary level, and a set ofelectronically stored course texts that serve as a
exemplarsentences.

Usually,thepre-defined vocabulary lists will be composed ofhigh frequency general words that are selectedfrom traditional vocabulary resources or generated
dynamically using corpus tools. However, the lists
might also be more specialized, such as lists oftechnical vocabulary used in specialist disciplines.
The

upper

of

bound

of

the

user’sgeneralvocabulary, on the other hand, will be defined
based on known research estimates or measured
using a standard vocabulary level test.
Using this input, the tool then automatically
identifies which target items appear in the course
texts

and

formatted

presents
table

these

together

in
with

an

the

tool

will

be

Early studies on the vocabulary size of native
speakers suggested that adult native speakers haveat their disposal a huge amount of vocabulary thatwould be almost impossible for foreign language
learners to acquire (for a review of this research,see [2]). However, when researchers began
investigating the size of native speakers'
vocabulary knowledge more exactly, a surprising
result was found. Nation [1], for example, showedthat native speakers typically learn only 1000 wordfamilies each year of their life. In this work,
Nation defined a word family as a base word (e.g.
"organize") together with its inflections and close
derivations (e.g., "organizing," "organized,""organization"). Nation's estimate was also
supported by Beck and McKeown [3], who
calculated that children aged five to six know onlyaround 2500 to 5000 words. From these results,
Waring and Nation [4] estimated that a typicaluniversity graduate would know around 20,000
word families. This is, of course, much smaller
than the number of words in a typical dictionary.
Forexample,Webster’s
3rd
International
Dictionary contains around 54,000 word families[4].
Similar research has also been carried out todetermine the typical vocabulary size of non-native
speakers of English. In the same paper as above,
Nation & Waring [4] reported that many adultlearners of English as a foreign language know lessthan 5000 word families. A more recent study by
Chujo [5] also suggests a similar figure, withJapanese university students demonstrating aknowledge of only around 3000 to 5000 words. In
addition, researchers have also identified aproblem in the complexity level of vocabulary
items that EFL learners have typically acquired.
Rather than acquiring the most frequent words
used by native speakers, many non-native learnersof English have been shown to have acquired somevery low-frequency words that they are unlikely to
encounter in real-life situations [6]. One reason forthis is perhaps the wash-back effect on high school
education programs of university entrance exams

exposed to high-frequency words in varying formsofto

2. Review of Literature on Vocabulary Learning

frequency words. This implies that they aresourceimprovements

attractively

frequency-level

information, pronunciation guides, and carefully
selected exemplar sentences from the course
materials that are matched to the user's general
vocabulary level.
The tool has already been successfully appliedin the creation of vocabulary lists for a very large
English program for students of science andengineering in Japan. A brief overview of the
program will be given and the vocabulary lists
created by the tool will be discussed. Finally,8

found that a coverage of 95% of the running words
was needed. Later, research by Nation [1],suggested that this value may even be as high as
98%. Of course, not all the words resulting in a
95% or 98% coverage will have a high frequency;
highly specialized texts in science and engineering,for example, are likely to include many words thathave a low frequency in general English.
Nevertheless, the importance of exposing learners
to high-frequency words is clearly a beneficial firststep to achieving this 95% or 98% coverage level.
A related issue is the kind of information
learners should be expected to know about
vocabulary at different frequency levels. Nation[1:27] lists multiple facets of vocabularyknowledge that can be important. These are thesound of the word, its written form, its constituteword parts, its core meaning, the concepts it isassociated with, other words associated with it, thegrammar patterns that it is likely to appear in, thewords it is likely to co-occur with, and theregisters and genres that the word is likely to be
used in. When combined, these facets serve as ameasure of vocabulary depth (in contrast to thesheer number of words known, which is usuallyreferred to as breadth). Other researchers haveinvestigated the relationship between depth andbreadth and there is evidence to suggest that thetwo are highly correlated (e.g., [9], [10]).
Although several researchers (e.g., [11-13])
have argued that English has a monosemic bias, i.e.,words have a single, fundamental meaning, it isclear that the depth of knowledge needed to fully'know' a word will still be related to the word's
frequency level. High-frequency words, forexample, almost by definition, will occur in morevaried contexts, have more concepts associatedwith them, and be used in a greater number ofregisters and genres than low frequency words.
There
is
also
a
strongargumentthat
high-frequency words will exhibit multiple core
meanings. A simple example is the very
high-frequency word "bank," which can refer to an
establishment that stores peoples savings or theside of a river. It can appear as a noun or a verb,
and will be a frequent term in both academic andnon-academic texts. In contrast, the very low
frequency word "supermacroporous" is likely tothat tend to include very difficult English reading
passages [6].
The importance of high-frequency words over
low frequency words is clearly revealed when therelationship between frequency and coverage isplotted. Table 1 and Figure 1 show the coverage ofthe most frequent words in the Brown Corpus
plotted for the top five 1000-word levels [7]. The
results show that knowledge of the first 1000
words will give learners a coverage of 72% of all
the words in the corpus. Knowing an additional
1000 words, on the other hand, will only improvethat coverage by 7.7%, and a further 1000 words,
will only improve on that by 4.3%. Clearly, if a
non-native speaker hopes to understand English,knowledge of the most frequent words is essential.
Table 1. Relationship between Vocabulary Size andCoverage in the Brown Corpus
Brown Corpus (100 million words)
Vocabulary
Coverage
Cumulative
Level
(%)
Coverage (%)
1000
72
72
2000
7.7
80
3000
4.3
84
4000
2.8
87
5000
1.9
89

Figure 1. Relationship between Vocabulary Size
and Coverage in the Brown Corpus
What is not clear from Table 1 or Figure 1 is
how many frequent words a learner would need tocomprehend a text without support materials, suchas dictionaries, glossaries, or native-speaker
informants, assuming that some words could beunderstood through context. However, thisquestion was investigated by Laufer [8], who

9approaches whilst also providing suitable
information
toimprove
vocabularydepthknowledge acquired through both direct andindirect learning approaches. In the following
section, I introduce AntLister, a novel, automated,vocabulary list and exemplar sentence generator
that demonstrates that such a vocabulary list
creation tool can be developed with today's
computer hardware and software.

only occur as an adjective and appear only in the
narrow genre of academic research papers in thefield of materials science.
The
examplesof"bank"
and"supermacroporous" strongly demonstrate thatlearners will generally require a greater depth ofknowledge to master high-frequency words and agreater breadth of knowledge if they hope tomaster low frequency words of many varying
disciplines, genres, and registers.
A separate strand of research has focused on thebest way for learners to acquire both high- andlow-frequency vocabulary items. Contrary to
beliefs among some teachers, research has shownthat a direct approach can be successful, especiallywhen dealing with high-frequency words (e.g., [1],[16]). However, this does not imply that direct
approaches are the only approach that should be
used. On the contrary, Nation [1] clearly states thatboth direct and indirect approaches should be seen
as complimentary. Also, he argues that direct
approaches should only account for a small
percentage (25%) of the time allocated to the
vocabulary component of a language program.
Also, as the need to learn increasingly lessfrequent words, such as technical vocabulary inscience and engineering, arrives, the use ofincidental learning approaches becomes preferred.
The above review of literature suggests several
key features of a successful vocabulary programthat are relevant to the current study: 1) it should
provide students with materials to help them
master the most frequent words of English throughboth direct and incidental learning methods, 2) itshould provide learners with information todevelop
their
depth
of
knowledgeofhigh-frequency vocabulary items, and 3) the
materials should facilitate both direct andincidental learning approaches. One final point isthat vocabulary learning is only one component ofan effective language program. Therefore, the timenecessary to generate vocabulary lists should be
minimized to allow teachers to focus on otheraspects of teaching preparation.
This leads to the following question: Is it
possible to automate the creation of effective lists
for the learning of high-frequency vocabulary
items that lend themselves to direct teaching

3. Design and Development of AntLister
AntLister is a software tool designed toautomatically create vocabulary lists and exemplar
sentences. The software is built using the Perl 5.14
programming language and runs on the commandline of a Windows operating system. For a
specified target word, the tool is programmed togenerate the following information: a) a numberedindex used for in-class instruction, and review
purposes, b) the target word itself, c) an IPA
pronunciation guide, d) an L1 translation, e) the
position of the first occurrence of the word in a setof EFL course materials, f) the frequency level of
the word, g) one or more exemplar sentences fromthe source materials that demonstrate varying usesof the word, and g) one or more exemplar
sentences from an alternative reference source.
To automatically generate an appropriate IPA
pronunciation guide, the Carnegie Mellon
University Pronouncing Dictionary is utilized [17].
This is an electronic dictionary of 125,000 words,
each with an Arpabet phonetic transcription code[18]. Arpabet is a phonetic transcription codedeveloped by the Advanced Research Projects
Agency (ARPA) in the 1970s for use in speech
recognition systems, where each phoneme isrepresented by a distinct sequence of ASCII
characters, e.g., green (G R IY N), she (SH IY),
and yield (Y IY L D). By programming a simple
mapping from the Arpabet phonetic transcription
code to IPA symbols, an automated IPA
pronunciation guide could be created.
To provide an automated L1 (Japanese)translation of the target word, data from the EDICT(Japanese/English Dictionary Project) is utilized[19]. In cases where multiple L1 translations are
possible, the first word in the list dictionary entry
is provided. However, in some cases no direct L1

10high-pass filter that evaluates their vocabulary
difficulty level by comparison with a pre-defineduser vocabulary level. If the sentences contain lessthan a pre-defined coverage level of words
assumed to be known by the user (e.g., 95%), they
are removed. Finally, the remaining exemplar
sentences are randomized and a predefined numberof them added to the vocabulary list. If theremaining sentences contain words that highly
collocate with the target word, these collocatingwords are also marked.
To present the results to the user in aneasy-to-understandformat,theresultinginformation is combined into a dynamicallycreated HTML formatted documented (andassociated CSS style sheet) and output to a Web
browser display for viewing. For distribution tolearners, the HTML document can then be saved asa PDF file.

translation is available and in these cases the entry
is left blank.
To identify the first occurrence of the target
word in the EFL course materials, an electronic
version of the materials is stored on disk with each
unit of the materials saved as a separate file. Then,a simple software scan reveals in which file, andhence in which unit, the target word appears.
The creation of a set of exemplar sentences
highlighting
varying
inflections
andnear-derivations of the target word is achieved in apipe-line process. First, 16 baseword lists createdby Paul Nation [20] are inputted into the system.
These consist of the most frequent 16,000 wordfamilies in the British National Corpus grouped
under head words and composed of a set of 'family
members,' which correspond to inflections andnear-derivations of the head word. For each target
word, all baseword lists are searched and theassociated family members of the target word
extracted. Next, the system searches in the
supplied course materials to find the firstoccurrence of each family member together with
its surrounding context (i.e., the sentence in which
it occurred). If no occurrence is found, the family
member is deleted. Similarly, the system also
searches for ‘hit’ sentences in a reference corpus
from the specialist subject of the learners. This
results in the final exemplar sentences containing
useful specialist-subject words that can be learned
incidentally while learners examine the usage ofthe target words.
Next, all 'hit' sentences are subject to a

4. Results and Discussion
An example output from AntLister is shown in
Figure 2. Note that in this case, no L1 translation
has been included. Also, note that two sources of
exemplar sentences have been incorporated; Ex. 1
and Ex. 2 refer to examples from course materials,
whereas BNC3 and BNC4 refer to examples takenfrom a secondary source, i.e., the science sectionof the British National Corpus.
The table of vocabulary and exemplar sentences
created by AntLister is created in a completely
automated fashion. Not surprisingly, therefore,
some of the information in the table can be

Figure 2. Output from AntLister for the words "entertain" and "fact"

11

required to study courses in Communication
Strategies and Academic Listening Comprehension.
Both courses extend across two semesters and havean important vocabulary component. Generally,both courses require that students develop their
vocabulary knowledge outside of the classroom aspart of a self-study component. On the other hand,some teachers of the courses include explicit
vocabulary instruction in class. Therefore, thevocabulary materials needed to be flexible enoughto facilitate both teacher-led and self-study
learning approaches. Due to the extensive time it
would take to develop vocabulary lists for thesetwo courses, the CELESE program designers felt
this would be ideal testing ground for AntLister.
The full set of materials created for the program
can be downloaded at the following website:
http://www.celese.sci.waseda.ac.jp.
To date there has been no controlled study tomeasure the effectiveness of the automaticallycreated vocabulary lists over traditional textbook
or handmade lists. However, several indirectindicators of its success in the CELESE program
are available. First, since introducing vocabulary
through these automated lists, students in bothcourses have shown increasing gains on
standardized tests, including the TOEIC, as shownin Figure. 3. These results suggest not only thatstudents are using the lists successfully, but alsothat they are incidentally learning other important
but lower-frequency specialist-subject words thatare contained in the exemplar sentences.

problematic. One of the early problems wascreating an acceptable IPA conversion. A
one-to-one conversation of Arpabet phonetictranscription codes to IPA symbols was
unacceptable. Following the advice of anexperienced phonetician, the solution was to map
codes and in some cases code-sequences directly to
IPA symbols.
Another problem was the selection of suitably
meaningful exemplar sentences. If the required
coverage percentage was set at 98% or above,following the research findings of Nation (2001),
the high-pass filter tended to remove sentences thatprovided contextual information and details. The
result was sentences that were easy to read but
rather abstract. On the other hand, if the coverage
percentage was set lower, e.g., below 95%, theremaining sentences would become meaningful but
overly complex. The impact of the coverage
parameter is shown in Table 2. Based on
preliminary experiments, a filter setting of 95%
was set as the default.
Table 2. Impact of coverage percentage setting on
the AntLister high-pass filter sentence selector
Coverage
Resulting sentences from AntListersetting
Additionally, the offline manager must
respond to any mail sent regarding
90%
failures or unavailability of media
items or units.
If the information is preserved, it will
be in an effort to guarantee its
95%
availability in case of legal dispute.
Attendance at lectures does not appear
to be affected by the availability of98%
outline notes.
Firstly, it is not clear that data are99%
unavailable for some of the countries.
To test AntLister in a real-world environment,
the tool was used to create vocabulary lists for two
required first-year courses in the Waseda
University CELESE English program for scientists
and engineers. Details of the CELESE English
program can be found elsewhere [21], but it isimportant to note that the program is a verylarge-scale English program serving 10,000
undergraduate and graduate students in Japan. All
first-year students (approx. 2000 in total) are

Figure 3. Performance Gains on TOEIC forFirst-Year Students at CELESE (2007 to 2011).
Second, while teachers tend to dismiss
automated approaches to materials development,

12teacher comments and student performance gains
suggest that the approach is effective. However,there are still many improvements that can be madeto the software, and a controlled study to measure
thetool'seffectivenessover
traditional
hand-created lists is needed. This is the aim of
future work.

no complaints have been voiced by either the
full-time or part-time faculty at CELESE regarding
the vocabulary lists automatically created by
AntLister. On the contrary, the success of the lists
created for use in first-year courses has led to thecreation of similar lists for 3rd- and 4th-year
elective courses in technical writing andpresentation.
Despite the positive results of the AntLister
vocabulary lists, there are many areas for
improvement. First, the lists do not always include
the most meaningful examples. Clearly, there is astrong relationship between lexical richness andsentence detail, as shown by the sentences in Table
2. Therefore, the challenge is to identify sentenceswith a low level of vocabulary complexity but ahigh level of concreteness. Another problem is thateach auto-generated vocabulary list entry can takeseveral minutes to generate depending on thenumber of source materials that need to be
searched. In the case of the CELESE materials, acomplete list required several hours of processing
time. Although this is still many times faster thanan equivalent human process, a faster algorithm iscertainly desirable. Finally, the current system runson the command line of a Windows operating
system. For wider applicability, the software needs
to be compiled to run as a multiplatform,
single-file executable with an intuitively-designed
user interface.

References[1] Nation, I.S.P., Learning vocabulary in anotherlanguage, Cambridge: Cambridge University Press,
2001.
[2] Nation, I.S.P., Vocabulary size, growth and use, in R.
Schreuder and B. Weltens (Eds.), The Bilingual
Lexicon. Amsterdam/Philadelphia: John Benjamins,
115-134, 1993.
[3] Beck, I.L. and McKeown, M.G., Social studies texts
are hard to understand: Mediating some of the
difficulties. Language Arts, 68, 482-490, 1991.
[4] Waring, R. and Nation, I.S.P., Vocabulary size, text
coverage, and word lists, in N. Schmitt and M.
McCarthy(Eds.),
Vocabulary:
Description,
Acquisition and Pedagogy, Cambridge: Cambridge
University Press, pp. 6-19, 1997.
[5] Chujo, K., Measuring vocabulary levels of English
textbooks and tests using a BNC lemmatised highfrequency word list, in J. Nakamura, N. Inoue, T.
Tabata (Eds), English Corpora Under Japanese Eyes,
Language and Computers. Amsterdam: Rodopi, pp.
231-249, 2004.
[6] Browne, C. and Culligan, C., Combining technology
and IRT testing to build student knowledge of highfrequency vocabulary, The JALT CALL Journal, 4 (2),pp. 3–16, 2008.
[7] Francis, W.N. & H. Kucera, Frequency Analysis ofEnglish Usage: Lexicon and Grammar. Boston:
Houghton Mifflin. 1982.
[8] Laufer, B., What percentage of lexis is essential forcomprehension, in C. Lauren & M. Nordman (Eds.),
From humans thinking to thinking machines,
Clevedon, UK: Multilingual Matters, pp. 316-323,
1989.
[9] Qian, D.D., Investigating the relationship between
vocabulary knowledge and academic reading
performance: an assessment perspective. Language
Learning, 52, 513-536, 2002.
[10] Laufer, B. and Nation, I.S.P., Vocabulary size anduse: lexical richness in L2 written production.
Applied Linguistics, 16, 307-322.productive ability.
Language Testing, 16, 33-51, 1995.
[11] Ruhl, C., On monosemy: A study in linguistic
semantics. Albany: State University of New York
Press, 1989.
[12] Parent, K., Polysemy A second language pedagogicalconcern, Unpublished doctoral dissertation, Victoria
University of Wellington, 2009.
[13] Nation, I.S.P., and Webb, S., Researching andanalyzing vocabulary. Boston, MA: Heinle, 2011.
[14] Jean-Pierre J., R., Does size matter? The relationshipbetween vocabulary breadth and depth, Sophia

5. Conclusion
This paper has presented a novel approach tothe automatic creation of vocabulary lists for use
in English as a Foreign Language (EFL) classroom
contexts. The approach used here relies on a
combination of online sources, including an
Englishpronunciation
dictionary,an
English-Japanese translation dictionary, and asource of sentences that demonstrate target word
usage in a variety of contexts. It also relies on a
programming script that implements a set of filters
to carefully select exemplar sentences that fulfill aset of 'good exemplar' criteria suggested byresearch. The approach has been applied in thecreation of vocabulary lists for required andelective courses in the Waseda University CELESE
English program for scientists and engineers, and13

International Review, 33, 107-120, 2011.
[15] Nation, I.S.P., and Webb, S., Researching andanalyzing vocabulary, Boston, MA: Heinle, 2011.
[16] Sokmen, A. J. Current trends in teaching second
language vocabulzary, in N. Schmitt and M.
McCarthy(Eds.),
Vocabulary:
Description,
Acquisition and Pedagogy, Cambridge: Cambridge
University Press, pp. 237-257, 1997.
[17] Availablefor
downloadat:
http://www.speech.cs.cmu.edu/cgi-bin/cmudict.
Accessed on 2012/03/13.
[18] For
more
informationsee:
http://en.wikipedia.org/wiki/Arpabet. Accessed on
2012/03/13.
[19] Availablefor
download
at:http://www.csse.monash.edu.au/~jwb/edict.html.
Accessed on 2012/03/13.
[20] Available on request.
[21] Detailed information on the CELESE Program can befoundathttp://www.celese.sci.waseda.ac.jp/.
Accessed on 2012/03/13.

14