Organizing English Reading Materials for Vocabulary Learning

Masao Utiyama, Midori Tanimura and Hitoshi Isahara
National Institute of Information and Communications Technology
3-5 Hikari-dai, Seika-cho, Souraku-gun, Kyoto 619-0289 Japan{mutiyama,mtanimura,isahara}@nict.go.jp

Abstract
We propose a method of organizing reading materials for vocabulary learning. It
enables us to select a concise set ofreading texts (from a target corpus) thatcontains all the target vocabulary to belearned. We used a specialized vocabulary for an English certification test as
the target vocabulary and used English
Wikipedia, a free-content encyclopedia, as
the target corpus. The organized readingmaterials would enable learners not only
to study the target vocabulary efficiently
but also to gain a variety of knowledge
through reading. The reading materials
are available on our web site.

1Introduction

EFL (English as a foreign language) learners andteachers can easily access a wide range of English
reading materials on the Internet. For example, current news stories can be read on web sites such as
those for CNN,1 TIME,2 or the BBC.3 Specialized
reading materials for EFL learners are also provided
on web sites like EFL Reading.4
This situation, however, does not mean that EFL
learners and teachers can easily select proper texts
suited to their specific purposes, for example, learning vocabulary through reading. On the contrary,1

http://www.cnn.com/
http://www.time.com/time/3
http://www.bbc.co.uk/
4
http://www.gradedreading.pwp.blueyonder.co.uk/
2

EFL teachers have to carefully select texts, if theywant their students to learn a specialized vocabulary
through reading in a particular discipline such as
medicine, engineering, or economics. However, it isproblematic for teachers to select materials for learning a target vocabulary with short authentic texts.
It is possible to automate this selection process
given the target vocabulary to be learned and the target corpus from which texts are gathered (Utiyamaet al., 2004). In this research (Utiyama et al., 2004),
we used a specialized vocabulary for an English
certification test as the target vocabulary and usednewspaper articles from The Daily Yomiuri as thetarget corpus. We then organized a set of readingmaterials, which we called courseware5 , using thealgorithm in Section 2. The courseware consistedof 116 articles and contained all the target vocabulary. We used the courseware in university English
classes from May 2004 to January 2005. We foundthat the courseware was effective in learning vocabulary (Tanimura and Utiyama, in preparation).
Based on the promising results, our next goal is
to distribute courseware (produced with our algorithm) to EFL teachers and learners so that we canreceive wider feedback. To this end, the courseware we constructed (Utiyama et al., 2004) is inadequate because it was prepared from The Daily Yomiuri, which is copyrighted. We therefore replaced
The Daily Yomiuri with English Wikipedia,6 a freecontent encyclopedia, and developed new course5
Courseware usually includes software in addition to othermaterials. However, in this paper, the term courseware is used
to refer to the reading materials only.
6
http://en.wikipedia.org/wiki/Main Pageware. It is available on our web site.7
In the following, will we first summarize our algorithm and then describe details on the courseware
we constructed from English Wikipedia.

2g(·) to be effective in retrieving documents relevantto the target vocabulary.
In Eq. (1), α is used to combine the scores ofdocument d, which are obtained by using Vtodo andVdone . It is defined as

Algorithm

We want to prepare efficient courseware for learninga target vocabulary. We defined efficiency in termsof the amount of reading materials that must be readto learn a required vocabulary. That is, efficient
courseware is as short as possible, while containing
the required vocabulary. We used a greedy method
to develop the efficient courseware (Utiyama et al.,2004).
Let C be the courseware under development andV be the target vocabulary to be learned. We iteratively select a document (from the target corpus)
that has the largest number of new types8 (types contained in V but not in C) and put it into C until C
covering all of V . “C covers all of V ” means thateach word in V occurs at least once in a documentin C.
More concretely, let Vtodo be the part of V notcovered by C, and let Vdone be V − Vtodo . We iteratively put document d into C that maximizes G(·),
G(d|α, Vtodo , Vdone )= αg(d|Vtodo ) + (1 − α)g(d|Vdone ),k1 + 1|W (d) ∩ Vx |,(2)

where W (d) is the set of types in d, E(|W (·)|) is
the average for |W (·)| over the whole corpus, andk1 and b are parameters that depend on the corpus.
We set k1 as 1.5 and b as 0.75. g(d|Vx ) takes a large
value when there is a large number of common types
between W (d) and Vx and d is short. These effects|W (d)|are due to |W (d)∩Vx | and E(|W(·)|) respectively. Asg(·) is based on the Okapi BM25 function (Robertson and Walker, 2000), which has been shown to bequite efficient in information retrieval,9 we expected7

http://www.kotonoba.net/˜mutiyama/vocabridge/
A type refers to a unique word, while a token refers to eachoccurrence of a type.
9BM25 and its variants have been proven to be quite efficient in information retrieval. Readers are referred to papers bythe Text REtrieval Conference (TREC, http://trec.nist.gov/), forexample.
8(3)This implies that even if |W (d) ∩ Vtodo | is 1, it is
as important as |W (d) ∩ Vdone | = |Vdone |. Consequently, G(·) uses documents that have new typesof the given vocabulary in preference to documentsthat have covered types.
To summarize, efficient courseware is constructedby putting document d with maximum G(·) into C
until C covers all of V . This allows us to construct
efficient courseware because G(·) takes a large value
when a document has a large number of new types
and is short.

3 ExperimentThis section describes how the courseware was constructed by applying the method described in theprevious section. We will first describe the vocabulary and corpus used to construct the courseware
and then present the statistics for the courseware.
3.1g(d|Vx )|W (d)|)+1
k1 ((1 − b) + b E(|W(·)|)|Vdone |1 + |Vdone |(1)

until C covers all of V . We then define g(·) as=α=Vocabulary

We used the specialized vocabulary used in theTest of English for International Communication(TOEIC) because it is one of the most popular English certification tests in Japan. The vocabulary wascompiled by Chujo (2003) and Chujo et al. (2004),
who confirmed that the vocabulary was useful inpreparing for the TOEIC test. The vocabulary had
640 entries and we used 638 words from it that occurred at least once in the corpus as the target vocabulary.
3.2

CorpusWe used articles from English Wikipedia as the target corpus, which is a free-content encyclopedia thatanyone can edit. The version we used in this study
had 478,611 articles. From these, we first discarded
stub and other non-normal articles. We also discarded short articles of less than 150 words. We thenselected 60,498 articles that were referred to (linked)
by more than 15 articles. This 15-link threshold was3.3 Example article
Figure 1 has an example of the articles in the courseware. It was the first article obtained with the algorithm. It shares 27 types and 49 tokens with thetarget vocabulary. These words are printed in bold.
Corporate financeCorporate finance is the specific area of finance dealing with the financial decisions corporations make, and the tools and analysis used
to make the decisions. The discipline as a whole may be divided between
long-term and short-term decisions and techniques. Both share the same
goal of enhancing firm value by ensuring that return on capital exceeds
cost of capital. Capital investment decisions comprise the long-term
choices about which projects receive investment, whether to finance thatinvestment with equity or debt, and when or whether to pay dividends toshareholders. Short-term corporate finance decisions are called working
capital management and deal with balance of current assets and current liabilities by managing cash, inventories, and short-term borrowing
and lending (e.g., the credit terms extended to customers). Corporate finance is closely related to managerial finance, which is slightly broader inscope, describing the financial techniques available to all forms of business ... (more)

Figure 1: Example article

courseware was referred to by 70.7 articles on average as can be seen from the bottom row. Table
1 indicates that articles in the courseware included
many target words and were heavily referred to byother articles.
3.4.2 Distribution of covered types
Figure 2 plots the increase in the number of covered types against the order (ranking) of articles thatwere put into the courseware. The horizontal axis
represents the ranking of articles. The vertical axis
indicates the number of covered types. The increase
was sharpest when the ranking value was lowest (leftof figure). The dotted horizontal lines indicate 50%
and 90% of the target vocabulary. These lines crossthe curved solid line at the 22nd and 83rd articles,i.e., 16.8% and 63.4% of the courseware, respectively. This means that learners can learn most of thetarget vocabulary from the beginning of the courseware. This is desirable because learners sometimes
do not have enough time to read all the courseware.
700600500num. of types

set empirically to screen out noisy articles. Finally,
we extracted a 150-word excerpt from the lead partof each of these 60,498 articles to prepare the targetcorpus. We set 150-word limit on an empirical basis
to reduce the burden imposed on learners. In short,the target corpus consisted of 60,498 excerpts fromthe English Wikipedia. In the rest of the paper, wewill use the term an article to refer to an excerpt thatwas extracted according to this procedure.

400300200

3.4 Courseware statistics
3.4.1 Basic courseware statistics
Table 1 lists basic statistics for the coursewareconstructed from the target vocabulary and corpus.10
The courseware consisted of 131 articles. Eacharticle was 150 words long because only excerpts
were used. The average number of tokens per article shared with the vocabulary (“num. of common tokens” in the Table) was 18.4 and that oftypes (“num. of common types”) was 12.4. About
12.3%(= 18.4150 × 100) of the tokens in each article
were covered by the vocabulary. Each article in the10On our web site, we prepared 10 sets of article sets called
course-1 to course-10. These 10 courses were obtained by repeatedly applying our algorithm to the English Wikipedia removing articles included in earlier courses. The statistics presented in this paper were calculated from the first courseware,course-1.

100
90%
50%
0020406080
article ranking

100120140Figure 2: Increase in the number of covered types
3.4.3 Document frequency distribution
Figure 3 has target words that occurred in eight articles or more. The numbers in parentheses indicate
the document frequencies (DFs) of the words, where
the DF of a word is the number of articles in which
the word occurred. These words were the most basic words in the target vocabulary with respect to thecourseware.
Table 2 lists the distribution of DFs. The firstcolumn lists the different DFs of the target words.
The values in the “#DF” column are the numbers ofTable 1: Basic courseware statistics (number of articles: 131, length of each article: 150 words)
Num. of common tokens
Num. of common types
Num. of incoming links

Average
18.4
12.4
70.7

words that occurred in the corresponding DF articles. The “CUM” and “CUM%” columns show thecumulative numbers and percentages of words calculated from the values in the second column. As we
can see from Table 2, more than 50% of the targetwords occurred in multiple articles. Consequently,
learners were likely to be sufficiently exposed to efficiently learn the target vocabulary.
service (19), form (17), information (12), feature (12), operation (11), cost (11), individual (10), department (10),consumer (9), company (9), product (9), complete (9),range (9), law (9), associate (9), cause (9), consider (9),offer (9), provide (9), present (8), activity (8), due (8),area (8), bill (8), require (8), order (8)SD Min MedianMax
10.8
11655
5.5
1
1227
145.3
1632 1056
SD means standard deviation.

reading materials prepared from The Daily Yomiuri
were effective in vocabulary learning (Tanimura andUtiyama, in preparation).
Our next goal is to distribute courseware (produced with our algorithm) to EFL teachers andlearners so that we can receive wider feedback. To
this end, we replaced The Daily Yomiuri, whichis copyrighted, with the English Wikipedia, which
is a free-content encyclopedia, and developed newcourseware whose statistics were presented and discussed in this paper. This courseware, which isavailable on our web site, can be used to supplementclassroom learning activities as well as self-study.
We hope it will help EFL learners to learn and teachers to teach a broader range of vocabulary.

Figure 3: Target words and their DFs.

References
Table 2: Document frequency distributionDF
191712
11
10
98
7
6
5
4
3
2
14

#DF
1
12
22
11
7
20
25
35
36
71
118
307

CUM
1
2
4
6
8
19
26
46
71
106
142
213
331
638

CUM%
0.2
0.3
0.6
0.9
1.3
3.0
4.1
7.2
11.1
16.6
22.3
33.4
51.9
100.0

Conclusion

While many teachers agree that vocabulary learning can be fostered by presenting words in context
rather than isolating them from this, it is very difficult to prepare reading materials that contain thespecialized vocabulary to be learned. We have proposed a method of automating this preparation process (Utiyama et al., 2004). We have found that ourK. Chujo, T. Ushida, A. Yamazaki, M. Genung, A. Uchibori, and C. Nishigaki. 2004. Bijuaru beishikku
niyoru TOEIC-yoo goiryoku yoosei sofutowuea noshisaku (3) [The development of English CD-ROM
material to teach vocabulary for the TOEIC test (utilizing Visual Basic): Part 3]. Journal of the College ofIndustrial Technology, Nihon University, 37, 29-43.
K. Chujo. 2003. Eigo shokyuushamuke TOEIC Goi 1 &2 no sentei to sono kouka [Selecting TOEIC vocabulary 1 & 2 for beginning-level students and measuring
its effect on a sample TOEIC test]. Journal of the College of Industrial Technology Nihon University, 36:
27-42.
S. E. Robertson and S. Walker. 2000. Okapi/Keenbow at
TREC-8. In Proc. of TREC 8, pages 151–162.
Midori Tanimura and Masao Utiyama. in preparation. Reading materials for learning TOEIC vocabulary based on corpus data.
Masao Utiyama, Midori Tanimura, and Hitoshi Isahara.
2004. Constructing English reading courseware. In
PACLIC-18, pages 173–179.