A Wikipedia-based Corpus Reference Tool
Jason Ginsburg
University of Aizu
Tsuruga, Ikki-machi, Aizu-Wakamatsu City
Fukushima, Japan, 965-8580

jginsbur@gmail.com

ABSTRACTThis paper describes a dictionary-like reference tool thatis designed to help users find information that is similar
to what one would find in a dictionary when looking up aword, except that this information is extracted automatically from large corpora. For a particular vocabulary item,
a user can view frequency information, part-of-speech distribution, word-forms, definitions, example paragraphs andcollocations. All of this information is extracted automatically from corpora and most of this information is extracted
from Wikipedia. Since Wikipedia is a massive corpus covering a diverse range of general topics, this information isprobably very representative of how target words are used ingeneral. This project has applications for English language
teachers and learners, as well as for language researchers.

Categories and Subject Descriptors
K.3.1 [Computer Uses in Education]: Computer-assisted
instruction (CAI)

General Terms
Human Factors, Design, Theory

Keywords
Corpora, Wikipedia, Language teaching

1.

INTRODUCTION

It is easy to use Natural Language Processing (NLP) methods to determine frequent vocabulary terms in various genresof English (e.g., see [1] for methods of extracting frequency
lists of ngrams from corpora). One can use vocabulary frequency lists of this sort for language teaching. In order tobecome proficient in English, English language learners needto know words that occur frequently in English in general,

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.
HCCE ’12, March 8–13, 2012, Aizu-Wakamatsu, Fukushima, Japan.
Copyright 2012 ACM 978-1-4503-1191-5 ...$10.00.

and also possibly words that occur in particular target genres of interest to the learner. However, simply giving students lists of words, a few examples, and telling them tolearn the words probably is not sufficient. One does not become proficient in a language simply by memorizing a list of
vocabulary terms.
Language learners (e.g., beginning to intermediate English
language learners) frequently come across vocabulary terms
that they do not know. In this case, a dictionary can comein handy. However, especially for a language learner, the information in a dictionary, while useful, may not be sufficient
to truly acquire a new vocabulary term. A dictionary definition may be difficult to understand. Examples containedin a dictionary may not provide sufficient context. It may
be difficult to learn a vocabulary item simply from reading
about it in a dictionary.
In this paper, we describe a Corpus Reference Tool which
provides information of the sort that one can find in a dictionary, except that some of this information is more detailed,
and this information is extracted from large copora. An
important feature of this application is that most of the information it provides is extracted from Wikipedia. Since
Wikipedia is a massive corpus that covers a diverse range of
topics, this information is likely representative of how target
vocabulary terms are used in general, and thus, this information can be of use for helping language learners. Thisapplication provides a user with lists of vocabulary terms
in target genres. For a target term, selected either froma vocabulary list or input directly by the user, a user canview part-of-speech (POS) information, morphological information, definitions, collocations, and example paragraphs.
Example paragraphs, extracted from Wikipedia, are particularly useful since they provide more context than sentence
length examples that are generally provided by dictionaries
or concordancers.
This project has several primary goals: a) to get a computer to automatically create useful reference materials forlanguage teaching, learning, and research, b) to create reference materials of higher quality than can be created without
the aid of large corpora, c) to create software that is easy to
use and that provides useful information to a user, and d) tocreate a product that is language and genre independent andthus can be utilized for various languages and genres. With
regard to d), if you have a suitable corpus (e.g., Wikipedia)
and NLP tools for a particular language, then you should beable to automatically reproduce this reference tool for that
language. Thus, although this application is designed forEnglish, a similar application could be produced for anotherlanguage if there are large corpora (such as Wikipedia) and
NLP tools for that language.

Wikipedia, and we used the Princeton WordNet Gloss Corpus4 to provide definitions for target vocabulary terms.5
Table 2: Resources
Resource
Porter Stemmer (from Natural Language Toolkit)
WordNet Lemmatizer (from Natural Language Toolkit)
Stanford POS Tagger
Princeton WordNet Gloss Corpus

In this paper, we describe the various features of this Corpus
Reference Tool, we describe how these features work, and we
discuss some applications for this tool. The organization ofthis paper is as follows. In section 2, we describe the primaryresources that this application uses. In section 3, we describe
the various features of this tool and some of their possible
applications. Section 4 is the conclusion.

2.

RESOURCESIn this section, we describe the primary resources that areused by this Corpus Reference Tool. This software was created in Python1 and incorporates corpora and some NLP
tools.
This application makes use of the corpora shown in Table
1. The Wikipedia corpus2 consists of all of the English
Table 1: Corpora
Size (# of tokens)
1,003,961,037(after cleaning)
English Gigaword
2,698,666,444
PERC Corpus
27,211,335
PERC Corpus: Computer 3,915,319
Science and Engineering
Sections
CorpusWikipedia

We next describe how we incorporated these various components into this Corpus Reference Tool.

3.

FUNCTIONSIn this section, we describe the functions of the Corpus Reference Tool. We explain what the functions are, how theywork, and what some of their potential uses are.

3.1

Search Functions

This Corpus Reference Tool has a GUI (Figure 1) that enables a user to find vocabulary terms and search for information about vocabulary terms.

Wikipedia text at the time that the file was downloaded.
We stripped away most tags via a Python script. Wikipedia,
a huge corpus of natural language that covers a variety ofsubjects, serves as the primary source of information for thisproject.
We also used two other corpora for the purposes of creating lists of vocabulary terms. English Gigaword [3] is anenormous corpus that contains text from international English news services. The Professional English Research Consortium (PERC) corpus3 consists of 22 different corpora invarious science and technology fields. We utilized two versions of this corpus; 1) the complete corpus, and 2) a portionof the PERC corpus consisting only of the Computer Scienceand Engineering sections.
In addition, we used the language resources for this projectgiven in Table 2. Versions of the Porter Stemmer and a
WordNet Lemmatizer, obtained from the Natural Language
Toolkit [1], were used to extract morphological forms of tokens. We used the Stanford POS Tagger [6] to POS tag

Figure 1: Corpus Reference Tool GUI

1

http://www.python.org/
This was downloaded fromhttp://en.wikipedia.org/wiki/Wikipedia:Database_download#English-language_Wikipedia on September
6, 2010 and consists of all “articles, templates, image
descriptions and primary meta-pages” of Wikipedia.
3
http://scn.jkn21.com/~percinfo/2

A user can choose to view a word list which shows the mostfrequent words in different genres, extracted from corpora4

http://wordnet.princeton.edu/glosstag.shtml
The WordNet Lemmatizer and WordNet Gloss Corpus aretools that were created from WordNet [5], [2].
5(see Table 1 for details about these corpora). The lists canbe sorted in terms of frequency or alphabetically and stop
words can be filtered out or presented (default is without
stop words). In addition, a user can type into the text box
a term of his/her choosing.
These wordlists can be used to find the most common vocabulary terms in particular genres of English, and thus they
can be used for teaching English. For example, the mostcommon terms in Wikipedia, due to its size, are, with some
exceptions due to the nature of Wikipedia, most likely to becommon words that are used in English in general.6

words are used may also find this information to be useful.

Figure 3: show: Frequency, Word Forms, Part of
Speech distribution

Once a user selects a word from a word list or inputs aword in the search box, a display appears (Figure 2 showsa window for the word ‘talk’). By clicking on the appropriate button on the left side of the window, the display
enables a user to view the following types of information fora target term: Frequency, Word Forms, POS distribution,
Definitions, Examples, and Collocations. We discuss thesefunctions in the following sections.

Figure 4: walk: Frequency, Word Forms, Part of
Speech distribution

3.3

Figure 2: Information Window for Vocabulary Term

3.2

Frequency

A user can view frequency information about a target word.
The screenshots of GUIs in Figures 3 and 4 include frequency
information for ‘show’ and ‘walk’. For example, of all unigrams that occur in Wikipedia, ‘show’ is the 118th mostfrequent, if you exclude stop words, and it is the 247th mostfrequent, if you include stop words. The tool also shows thetotal number of times that the word ‘show’ occurs and thepercentage of the corpus (the entire Wikipedia text) that istaken up by ‘show’. Figure 4 shows that ‘walk’ is much lessfrequent; it is the 2606th most frequent word, if you exclude
stop words.
The frequency of a word in Wikipedia, in most cases, islikely representative of the general frequency of that word
in English, considering that Wikipedia is so large. Language
teachers can use this information to help determine what vocabulary to teach and language researchers (linguists, lexicographers, etc.) who are interested in how often particular6
Note that some words may occur especially frequently
in Wikipedia, but not as frequently in general English.
For example, the second most common word (unigram) in
Wikipedia is ‘talk’, which occurs frequently in Wikipedia inthe expression ‘talk pages’.

Word Forms

The Corpus Reference Tool also allows a user to view wordform (morphological) information. Figures 3 and 4 include
word forms for ‘show’ and ‘walk’ that are automatically extracted from Wikipedia utilizing the Porter Stemmer andthe WordNet lemmatizer. All tokens in Wikipedia werestemmed with the Porter Stemmer and lemmatized with the
WordNet lemmatizer. Then for each token, lists of morphological forms were formed by a) finding the stem and lemmaof the token, and b) finding all tokens that have an identicalstem or lemma with the token. For example, the following
tokens are grouped together since they all are stemmed orlemmatized as ‘show’ at least once: ‘show’, ‘showed’, ‘showing’, ‘shows’, ‘showings’, ‘showe’, ‘showes’, and ‘showss’.
Similary, the following tokens are all stemmed or lemmatized as ‘walk’ at some point: ‘walking’, ‘walked’, ‘walk’,‘walks’, ‘walke’, ‘walkes’, ‘walkings’, ‘walkely’.
Note that the word form lists for target words includes some
odd forms such as ‘showe’ and ‘walke’. The word ‘showe’ isstemmed/lemmatized as ‘show’, and thus it is considered aform of ‘show’. In (1a), ‘show’ is a misspelling of ‘showed’.
In (1b), ‘showe’ is a foreign word.
(1)(a) Researched showe7 that Piagetian intelligence iscorrelated but it is not identical withpsychometric intelligence and IQ.
(b) “Eshowe”, is the oldest town of European
settlement in Zululand. Eshowe’s name is said tobe inspired by the sound of wind blowing
through the more than 4 of the indigenous
Dhlinza Forest, the most important and striking

7“Research showed . . . ”

feature of the town. Although the name is most
likely to be derived from the Zulu word for thexysmalobium shrubs, “showe” or “shongwe”.

is reflected in the POS distributions, with ‘show’ being used
about 78% of the time as a noun and 22% of the time as averb, and ‘walk’ being used about 75% of the time as a verb
and 25% of the time as a noun.

“Walke” also appears in Wikipedia. It is stemmed and/or
lemmatized as “walk”, thus resulting in ‘walke’ appearing in
the word forms list for ‘walk’. Examples with “walke” areshown in (2a-b). (2a) contains the name ‘Walke’ and (2b)contains ‘Walke’ as part of the title of an old publication;
this appears to be an old spelling of ‘walk’.

This POS distribution most likely gives a good picture of
the general POS distribution of target words, since the distribution is calculated from an enormous corpus.9 This POS
information can be useful for language researchers and educators, in giving a general idea of the POS distribution oftarget terms in English. In addition, as will be discussed in
the following sections, this tool enables a user to view definitions and examples for particular POS versions of target
terms.

(2)(a) Three ships in the United States Navy have been
named for Rear Admiral Henry A. Walke.
(b) * “A Weekes Worke, And A Worke For Every
Weeke”, 1616, 1628, 1650? (“A Weekes WorkeContaining Rules And Directions How To
Walke In The Wayes Of Godliness Both To
God And to Men”)

Odd word forms, such as ‘showe’ for ‘show’ and ‘walke’ for‘walk’ could be filtered, based on the fact that they tend tooccur very infrequently in Wikipedia. For example, ‘walk’
appears 37,920 times and ‘walke’ appears only 109 times.
The word ‘show’ appears 403,053 times and ‘showe’ only
appears 12 times. However, these ‘odd’ word forms may beof use for identifying certain misspellings (1a), old spellings(2b), etc. This word form function could be improved so thatit generally excludes incorrect word forms using algorithms
to exclude names and misspellings. We leave this for future
work.
Word forms are extracted automatically from Wikipedia andthus they likely give a good representation of possible morphological forms of target vocabulary terms. This information may be of interest to language teachers, learners, andresearchers.

3.4

Part of SpeechThis application enables a user to view the POS distribution for target words. To obtain the POS distribution, wetagged the entire Wikipedia corpus with the Stanford POS
tagger. The large tagset (the Penn Treebank tags [4]) used
for the Stanford POS tagger was simplified for ease of use.
For each word type, we calculated the percentage of occurrences of each POS tag for that word; this distribution canbe viewed by a user. POS tags that occur less than 3% of thetime are not displayed.8 See Figures 3 and 4 for the POS
distributions for the words ‘show’ and ‘walk’ respectively.
Both of these words are used as nouns and verbs, and this8
The POS distribution is calculated for a unigram as is.
A more accurate POS distribution could be calculated byconsidering the POS distribution of all inflected forms ofa unigram. Currently, this tool extracts forms of a target
word, without consideration of whether or not the form isinflected or derived. For example, the word form finder extracts ‘talkativeness’ as a form of ‘talk’. This is accurate,in that ‘talkativeness’ is derived from ‘talk’. However, since‘talkativeness’ is not an inflected form of ‘talk’ it is not clear
to us if it would be useful to consider the POS distributionfor ‘talkativeness’ when considering ‘talk’. Of course, it may
be useful to develop our tool so that it considers inflected
forms of target words. We leave this for future work.

3.5

Definitions

For each vocabulary term, a user can view definitions, whenavailable, for particular POS tags of the term. The user
chooses a particular POS tag for a word, and then definitions
for that POS tag for that word are displayed. See Figures 5
and 6 (see the next page) for examples of definitions for thenoun and verb forms of ‘show’ and of ‘walk’, respectively.10
Definitions are extracted from the Princeton WordNet Gloss
Corpus. This corpus is especially useful since it provides definitions that are categorized into adjectives, adverbs, nouns,
and verbs. Thus, for words that have POS tags of these basic categories, definitions can be extracted. When the user
chooses a particular definition of a target word, the program
searches through the Princeton WordNet Gloss Corpus forthe same word with the same POS tag, and prints the relevant definitions to the GUI.

3.6

ExamplesOne of the most useful components of this Corpus Reference
Tool is its ability to show example paragraphs for particular
POS categories for target words. Examples containing nounand verb versions of ‘show’ and ‘walk’ are presented in Figures 7 and 8. This function takes advantage of the structureof Wikipedia, in which text is stored in paragraphs (there
9
The Stanford POS tagger attains 96.97% accuracy on
WSJ22-24 test set (http://nlp.stanford.edu/software/
pos-tagger-faq.shtml). However, it is not completely accurate and numerous problems arise for POS tagging. The
tagger tags “red” in (i) as a noun. Here, “red” could be considered an adjective that modifies “storm”, in which case thenoun tag would be incorrect, or it could be considered to bepart of a compound noun, in which case the tag would becorrect. In (ii), walk’ is incorrectly tagged as a verb.
(i) “Red Storm” is a supercomputer architecture designed
for the US Department of Energy’s National Nuclear Security Administration Advanced Simulation and Computing
Program.
(ii) The 2009 Capital Challenge Walk MS raised half a million dollars for the National MS Society’s programs and initiatives. .
10Note that the definitions are presented in the order thatthey are found in the Princeton WordNet Gloss Corpus, andthey are not necessarily in the order of the most common
uses. For example, the first definitions that appear for thenoun and verb forms of ‘walk’ are definitions related to baseball. The definitions related to moving feet do not appear
until definition 3 for the verb form and definition 2 for thenoun form. It may be desireable to order the definitions ina better manner.

Figure 7: Example Paragraphs with ‘show’

Figure 5: Definitions of ‘show’

Figure 8: Example Paragraphs with ‘walk’

Figure 6: Definitions of ‘walk’

are line breaks between paragraphs); thus paragraphs are
simple to extract.
Examples are extracted in the following way. When theuser chooses a particular POS tag for a particular word,
the program searches through the Wikipedia corpus for an
instance of the target word with the appropriate tag. If
found, the paragraph containing the target word is printed
to the GUI. A user can click on the same POS tag, or another
POS tag, to view further example paragraphs. If a target
word is frequent, then the user can potentially view manydifferent example paragraphs.
This component of the tool can be especially useful for lan-

guage teachers, learners, and researchers. Dictionaries andconcordancers generally present, at most, sentence length
examples. This Corpus Reference Tool presents paragraphlength examples. Examples of this length contain more context than simple sentence length examples. In addition,these are examples of real language on various general topics, and thus, these examples can give a user a good idea of
how a target word is used. Teachers can use these example
paragraphs to develop lesson materials, students of English
can use these examples to see how target words are used,
etc.

3.7

Collocations

Lastly, this tool provides basic data on collocations for targetwords. A user can choose to view two or three word collocations with or without stop words. For a particular collocation, the user can view example paragraphs. Collocations,as defined for this tool, are simply frequent bigrams andtrigrams containing a target vocabulary word. Collocations
with stop words may be useful for some users and not forothers, so we’ve included the option of letting a user choose
whether to view collocations with or without stop words.
When a user chooses to view collocations, 10 collocations(the most frequent of the appropriate type) are displayed
and the user can view 10 more collocations, etc., by clicking
on the collocation type again. The user can also view example paragraphs containing a target collocation. Examples of
2 and 3 word collocations containing ‘show’, both without
stop words, as well as example paragraphs containing thecollocation “television show”, are shown in Figure 9.

of this tool. Since Wikipedia is so large, this information is
likely quite representative of how target vocabulary terms
are used in general English.
One immediate way that this tool can be used is for development of language teaching materials. An instructor canuse the tool to obtain word lists in a target genre and thenfind example paragraphs containing target words. These example paragraphs can be given to students to study, alongwith exercises.11 Exercises could include questions about
the paragraphs that test for reading comprehension. In the
future, we plan to add a function that automatically creates editable lesson materials. For example, example paragraphs for target terms could be automatically extracted
and printed to a file, and vocabulary terms within a paragraph that may be useful to English language learners canbe automatically identified, via the use of word frequency
lists.
Lastly, we note that this research has applications for a variety of fields. It has applications for applied linguistics,
since one purpose is to aid in the development of materials
that can be used for teaching and learning languages. It
has applications for corpus linguistics and possibly theoretical linguistics, since it can be used for investigating partsof speech, morphology, semantics, discourse analysis, etc. It
has applications for computer science (e.g., natural language
processing) since it requires development of algorithms to
extract natural language information from corpora. It also
has applications for artificial intelligence, since a goal is toget a computer to automatically create useful reference andresearch resources - automatic creation of reference materials could in some sense be likened to ‘intelligence’. Further
research and development remains to improve this tool andapply it for various uses.

5.

Figure 9: Collocations and Example Paragraphswith ‘show’
The collocation function enables a user to see words that
tend to occur together and to see example paragraphs withtarget collocations. This function can be useful for findingout what words tend to occur with a target word.

4.

REFERENCES[1] S. Bird, E. Klein, and E. Loper. Natural language
processing with Python. O’Reilly, Sebastopol, CA, 2009.
[2] C. Fellbaum, editor. WordNet: An electrionic lexical
database. MIT Press, Cambridge, MA, 1998.
[3] D. Graff, J. Kong, K. Chen, and K. Maeda. English
Gigaword Third Edition. Linguistics Data Consortium,
Philadelphia, 2007.
[4] M. P. Marcus, B. Santorini, and M. A. Marcinkiewicz.
Building a large annotated corpus of English: The Penn
Treebank. Computational Linguistics, 19:313–330, 1993.
[5] G. A. Miller. WordNet: A lexical database for English.
Communcations of the ACM, 38:39–41, 1995.
[6] K. Toutanova, D. Klein, C. Manning, and Y. Singer.
Feature-rich part-of-speech tagging with a cyclic
dependency network. In Proceedings of the
HLT-NAACL 2003, pages 252–259. 2003.

CONCLUSION

This Corpus Reference Tool provides a variety of information (word lists, frequency information, POS information,
morphological forms, example paragraphs, collocations, anddefinitions) that is automatically extracted from corpora.
We have presented a few examples of the output of this tool(see the Appendix for more examples). Wikipedia, fromwhich most of this information is extracted, lies at the heart

11

Paragraphs are currently printed out to GUI but they canbe easily printed to a file or a user can copy directly fromthe GUI.

Appendix: Further Examples of Data12
Word
DetailsEx.(V)

Ex.(N)

Word
DetailsEx.(N)

Ex.(V)

Word
Details:Ex.(Adj)Ex.(Adv)

Ex.(N)

12fly
Freq.: 2793; POS: V (81%), V (18%); Forms: fly flies flying flis fli flyings flied flys
An Air Force analysis (Project Red Baron II) showed that a pilot’s chances ofsurvival in combat dramatically increased after he had completed 10 combat missions.
Red Flag was created in 1975 to offer US pilots the opportunity to fly 10 realisticallysimulated combat missions in a safe training environment with measurable results.
Many aircrews had also fallen victim to SAMs and Red Flag exercises provided pilots
experience in this regime as well.
“Tenkara lines” are special lines used for the fixed-line fishing method of tenkara.
Traditionally these are furled lines the same length as the tenkara rod. Although original
to Japan, these lines are similar to the British tradition of furled leaders. They consist of
several strands being twisted together in decreasing numbers toward the tip of the line,
thus creating a taper that allows the line to cast the fly. It serves the same purpose as the
fly-line, to propel a fly forward. They may be tied of various materials, but mostcommonly are made of monofilament.
study
Freq.: 655; POS: N(81%), V(18%); Forms: studis studying study studied studiing studys
studyings studi studyed studies studiess
After the revolution, Ramon Castro was forced to abandon the family farm as it became
legal property of the state. He continued to dedicate himself to the study of agriculture
however, and is chiefly responsible for many of Cuba’s agricultural initiatives since therevolution.
“Lydie Salvayre” (b. 1948) is a French writer. Born in the south of France to Republican
refugees from the Spanish Civil War, she went on to study medicine in Toulouse andcontinues to work as a practicing psychiatrist. She has been awarded both the Prix
Hermes and the Prix Novembre for her work.
fast
Freq.: 2177; POS: Adj (45%), Adv(37%), N(18%); Forms: fast fastnesses fasting fasts
fasted fastness fastings fastes faste
The “Breviarium” was completed by Eutropius within 369: Paeanius’ translation
appeared before 380. The “Breviarium” was a compendium of ancient Roman history,
used both as a textbook in schools and as a fast course on Roman history for the higher
social classes (it was dedicated to Emperor Valens): Paeanius’ translation allowed
Greek-speaking people to have a graceful version of this compendium.
Aside from her powers, Gypsy is an expert in hand-to-hand combat stealth. She’s also anaccomplished acrobat, able to leap high, run fast, swim, and execute unexpectedly quick
martial arts tactics with relative ease. Gypsy also has a strong aptitude in electronics andcomputers, and has become skilled in the use of firearms. She has been trained by Bronze
Tiger, and in issues of “Birds of Prey”, Lady Shiva has offered to train her as well.
Benjamin of Tudela mentions an Ahimaaz ben Paltiel in Amalfi in southern Italy, in theyear 1162 (see his “Travels,” ed. Asher, i. 13, 14). This may well have been a descendantof his earlier namesake; for it is known that two brothers of the grandfather of Ahimaaz
ben Paltiel were sent with presents to Paltiel by the prince of Amalfi. In a list of twentytwo selihah (elegiac) poets (Italy, fifteenth century?), Ahimaaz ben Paltiel is mentioned
as the author of two poems; and a Mahzor of the Roman rite attributes to him a selihah
for the Fast of Esther.

Freq. = frequency rank, not including stop words; POS =POS distribution; Forms = word forms, Ex. = example