Please leave your questions
and/or comments here if any.

Text Covering Efficiency and Word Tier Analysis for
the Proposal of Vocabulary Learning Order and theAnalysis of Text Genres

Or simply send an email to:tatsu.matsushita@ninjal.ac.jp

QUALICO 2023, at the University of Lausanne, 28-30 June 2023

Tatsuhiko Matsushita (National Institute for Japanese Language and Linguistics, NINJAL)⚫L2 vocabulary learning is a significant burden for learners.
⇒In what order should learners learn vocabulary to increase text coverage most efficiently?
←The higher the text coverage, the better the comprehension will be. (e.g., Laufer & Ravenhorst-Kalovski, 2010)⇒How can we evaluate the efficiency of a word list (e.g., Academic Word List, Coxhead, 2000) for learners?
*Simply checking the frequencies and text coverage of the grouped words (e.g., Hyland & Tse, 2007) is not enoughto understand the efficiency (i.e., usefulness) of the words, especially when the total number of words aredifferent between word lists (groups) (Matsushita, 2012). Solution ⇒TCE

⚫ How can the characteristics of text genres be discerned based on the proportion of groupedwords with different domains and levels (Basic/Intermediate/Advanced and so on) ?
⚫ To demonstrate how useful an index ‘Text Covering Efficiency (TCE)’ （Matsushita, 2012） is fordetermining the vocabulary learning order for a particular text genre.
⚫ To demonstrate the ‘Word Tier Analysis (WTA)’ （Matsushita, 2012） applied to Japanese medical texts
as an example to evaluate word groups (lists) and show the lexical characteristics of the target genre.
＊The term “word tier” in Matsushita (2012) means a somewhat similar (especially in purpose) concept to the one used in
Burch and Egbert (2022), but totally different in the method for identifying the tiers.

[# of lemmas, in one million lemmas*]〇Word Tier Analysis (WTA): An analysis to see the difference of〇Extraction of Domain-specific Words for Grouping Words

genres based on TCE figures of various grouped words (e.g., Academic(4D-3D) /Technical (2D-1D) /Literary words) by frequency level (Rank
1-1,291:Basic; 1,292-5,000:Inter.; 5001-10000: H-Adv.; 10001-15000:H-Adv.; 15001-20000: S-Adv.) Rankings are by Matsushita (2012).

➢ Target corpora: Academic texts in the 4
domains of Arts & Humanities (Arts)[0.47], SocialSciences[1.76] (Soc), Science & Technology(Tec)[0.41], Biology & Medicine (Bio)[0.26], (aswell as Literary Works (Lit.)[8.25])➢ Reference corpus:
Non-academic texts from BCCWJ 2009
monitor version [29.9] *BCCWJ: Balanced
Corpus of Contemporary Written Japanese➢ Index for keyness: Log-likelihood Ratio
Words extracted from x domains are called AcaxD words,e.g., Aca4D words are the words shared in the domainspecific word lists extracted from the 4 academic
domains respectively.

〇Text Covering Efficiency (TCE)
・TCE （E） (Matsushita, 2012)

E=𝐹𝑡
𝐿𝑡𝑤

Mean coverage by a word
In the tested word group1,000,000
×𝑁𝑡

𝑭𝒕 ×𝟏,𝟎𝟎𝟎,𝟎𝟎𝟎=𝑳𝒕𝒘 ×𝑵𝒕

Standardization

E: Text Covering Efficiency (TCE) = Expected

Corpus Genre
Corpus Size

WordTierCum. ## ofofLemma
Lemmass

ExampleWords(Japanese)

Example Words(Translation)

TechnicalFor
For
Students Doctors

Whole Essays,Bio
Tec News- BCCWJNovels
Articles Articles paper
2009
etc.

0.59875 1073
612 628394 322384 415278 246
263 341200 178
182 189
164 1211088991948586443639693834315283

Basic_Aca1D_Bio

9921,027

1,119

H_Adv_Aca1D_Bio

541,173

Basic_Aca3D
Adv_Aca1D_Bio
Inter_Aca4D
Inter_Aca3D
Basic_Aca2D
Inter_Aca3D
Inter_Aca2D
Adv_Aca2D

39

1,212

68

1,280

559

1,839

53

1,892

451,937

場合 行う
症状 治療
子 鏡
する ある動脈 投与
必要 試験
肝 血管-性/性 -的腎 リンパ原因 -やすい

542

2,479

障害 パーセント

impairment; percent

391

2,870

cell; method of medical treatment

429

3,299

53,3041423,446

細胞 療法
肺 来す-用 - 室血 壁
鉄 ‐波iron; -wave

20

不良 小-/-小mal-; small/little

12

Basic_Aca1D_Tec

Basic_Lit
Inter_Aca1D_Tec

46 4,452

Inter_Aca1D_Soc

111774element; mirrordo; beartery; administration(of drug)

need; test
liver; blood vessel[suffix] -ty (quality);-ive (adjectival)

kidney; lymph

cause; easilylung; bring about
for the use of; -roomblood; wall

0.72

2.71

→ Informal

5.68 32.82

2.10

Order

31

symptom; medical treatment

1.27← Formal ←TCE values in Medical Books

Basic_Aca4D
Inter_Aca1D_Bio
Basiccase; act

Medical Books

1099 1069

667

525 3824225
103
78
640 67352354 356
11682
56
3
196
59
48
2747
25
9592
61
149 248

89
78
496
22
425
20
241
18
113
95
59
2292
44

9
53
481
3
384
2271
2
119
84
48
26
97
46

50
71
531
6475
12
156
3274
121
113
20
97
55

21

39

77

40

43

23

1730

15 1685726➢ Medical books are extremely biased toward technical words in biology and medicine.
text coverage (=number of tokens) of a word ➢ WTA allows you to say things like, “Learning the intermediate words in biology andin the tested word group in a one-millionmedicine (Inter_Aca1D_Bio) is 12+ times more efficient in covering medical books
for students than in covering newspaper texts, and 3.1 times more efficient thantoken text in the target domain
learning advanced words in biology and medicine (Adv_Aca1D_Bio) .

𝐹𝑡: Number of tokens (= text coverage) of thetested word group in the target text
𝐿𝑡𝑤: Number of lemmas of the tested word
group
𝑁𝑡: Number of tokens in the target text (textlength)◆ WTA using TCE can clarify 1) differences in the relative importance ofdifferent groups of words according to the purpose of learning, and 2)
lexical differences among various text genres.
◆ The higher the TCE figure is, the more useful the words will be. The mostefficient learning order can be determined by the TCE figures.
TCE is relatively easy to calculate, and the results
can be easily applied with relatively little distortion ◆ TCE is a simple and robust index for comparing the usefulness of word
groups (lists) as well as for discerning the lexical characteristics ofdue to different corpus sizes. If word lists such as
different text genres.
basic/academic/technical vocabulary for various
genres are analyzed by a vocabulary frequency
Limitations & Future Research
profiler such as AntWordProfiler (Anthony, 2022),◆ The quality of the words in the text is not considered.
for example, the word tiers can be shown by◆ by incorporating these methods into a word frequency profiler such as Jcalculating the TCE easily.
LEX (Suganaga and Matsushita, 2013), such analysis can be facilitated.

References (excerpt) *For the full reference list, please check it out here.

This work was supported by JSPS KAKENHI
Grant Number 18H00679, 20KK0005 and23H00072

Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213–238.
Hyland, K., & Tse, P. (2007). Is there an “Academic Vocabulary”? TESOL Quarterly, 41(2), 235–253.
Matsushita, T. (2012). In what order should learners learn Japanese vocabulary? A corpus-based approach (PhD thesis). Victoria University of Wellington.
Downloaded from https://researcharchive.vuw.ac.nz/xmlui/handle/10063/4476
Suganaga, Y. and Matsushita, T. (2013). J-LEX: An Online Lexical Analyzer of Japanese Texts. Available from http://www17408ui.sakura.ne.jp/index.html