名古屋外国語大学論集

第10号

2022年2月

ArticleLearning Vocabulary with SkELL: Developing a
Methodology with University Students in Japan Using
Action ResearchHenry TROY

Abstract
Corpora are becoming more prevalent in the language classroom, especially inthe development of dictionaries and course materials. Nevertheless, corpora are
still perceived by many educators as difficult to use directly in the classroom,
which is also known as “data-driven learning” (DDL). This study investigatesthe student and teacher reactions to the use of SkELL, a free online corpus
designed to be user-friendly, for vocabulary learning at a university in Japan.
Action research is utilized to refine the teaching methodology, with changes tothe method based on student and teacher feedback after each of the four implementations of DDL. The results show that the students enjoyed using SkELL
and felt it was effective for vocabulary learning, while the teaching methodology
grew in efficiency throughout the course.

Introduction
The use of corpora in the language classroom, in one form or another, isincreasing. The most common ways in which they may be utilized are via
dictionaries and textbooks which now draw on corpus-acquired data. With163

dictionaries, it has been described as a “revolution”, and corpora are now used
as an integral part of their development (Breyer, 2011, p. 12; Tribble, 2015,
p. 38), and while they are still less utilized in course books, their influence is
growing (McCarten, 2010). However, despite a rise in investigative studies ofmore direct applications of corpora in EFL and ESL classrooms (Breyer, 2011;
Mizumoto & Chujo, 2015), overall they have not become prevalent in the way
some in the early 1990s predicted (Breyer, 2011). Thus, the idea of data-driven
learning (DDL) developed by Johns (1986; 1988; 1991), where a corpus is useddirectly in the language classroom by students, remains on the “periphery” oflanguage teaching (Karras, 2015, p. 169).
The main reasons that DDL has failed to break through in all but specifichigher education settings are threefold. First, teachers themselves either lackthe training or motivation to employ such methods. Although teachers may beaware of corpora, they lack the skills needed to incorporate them into lessons(Tribble, 2015). Second, there are logistical requirements that many educational
institutions simply cannot meet, such as a need for computers or tablets (LenkoSzymanska, 2017). Finally, students can be resistant to DDL, as they also require
training and user-friendly resources to engage with it effectively, both of which
may be lacking (Hirata & Hirata, 2019; Kilgariff et al., 2015; Lenko-Szymanska,2017).
In terms of DDL research, although the number of relevant studies has risen
significantly since 2000, these investigations have tended to have a relatively
narrow focus. Almost all DDL studies are done at universities, and so they are noteasily applied in other educational fields (Lenko-Szymanska, 2017). Moreover,the majority of DDL studies are more greatly concerned with quantitative
results by assessing gains in test scores with a particular learning methodology.
Although this is of value, qualitative studies of DDL are also extremely important(Chambers, 2007). In order to create effective and efficient methods with whichto utilize DDL in the language classroom, the reactions of teachers and students
164

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■to those methods need to be investigated. In a rare study of teachers’ views on
DDL conducted by Breyer (2011), most said they do not use it in their classes,
but would be keen to do so if there were ready-made materials. Similarly, when
discussing his own approach to DDL, McNair (2018) explained “as a teacher, I
would be more likely to try something if a fellow teacher has had success with it
than if I only read about it in an academic article” (p. 232). As for student reactions, it is of “crucial importance” that more studies of student-based evaluationsof DDL are conducted, according to Hirata and Hirata (2019, p. 236).
EFL education in Japan is especially lacking in DDL and the use of corpora
in general. The country is making some progress with English education, as from2020 it became a compulsory subject even for elementary school students, andall university students do at least one year of English even if it is not their major(Jones, 2019). However, EFL lessons in primary and secondary education in
Japan are still overwhelmingly focused on passing exams rather than achieving
communicative competence (Hirata, 2018), and those that do want to be able
to speak English effectively usually rely on private language schools, private
lessons or cram schools to achieve this (Jones, 2019). Japan ranked 55th out of
100 nations in the English Proficiency Index (2020), and one way to improve
EFL education in Japan is through modernising the way it is taught. There havebeen very few studies into the effectiveness of DDL in the EFL classroom inJapan, and those that have done them strongly urge further research (Hirata &Hirata, 2019; Mizumoto & Chujo, 2015).
This study, therefore, investigates the effectiveness of an action researchbased approach to DDL for vocabulary acquisition in an EFL class at a university
in Japan, using SkELL as the corpus. The study aims to assess student reactions
to the methodology in order to improve it, and also to document the teacher’s(myself) experience of its utilization. This will somewhat cover the shortfall in
qualitative research on teacher and students’ reactions to DDL, and do so in alocation where such research has been scarce. The study follows on from my165investigation conducted via an indirect implementation of DDL, with concordance lines on paper used in a mixed-methods approach to helping students learnphrasal verbs at a private language school in Japan (Troy & Millar, 2019). In
contrast to that study, this investigation has the students using a corpus directly inthe classroom, is more focused on the qualitative rather than quantitative results
and is conducted at a university rather than a private language school.

Literature Review
SkELL: What Is It?
SkELL (Sketch Engine for Language Learning) is a free, online corpus
which allows users to search for words or phrases without requiring registration(SketchEngine, 2020). The English Corpus for SkELL contains texts obtained
from a variety of fields, with the most prevalent Wikipedia articles (about
38.88%), English Web (30.91%), Time stamped web corpus (14.07%) and theBritish National Corpus (8.71%). It is updated regularly and is currently on
its 3.10 version (SketchEngine, 2020). SkELL is itself a component of Sketch
Engine, which allows for the “building, managing and exploring of large text
collections in dozens of languages” (SketchEngine, 2020). In essence, Sketch
Engine allows users to develop their own corpora, while SkELL is a ready-madecorpus which is designed to be user friendly.
SkELL was developed with language learners and teachers in mind, so its
interface and features are made to be as user-friendly as possible. “Corpora
for all” is its strapline, according to the developers (Kilgariff et al., 2014). It is
widely used in lexicography and to some extent in English language teaching(Barrs, 2016; Kilgariff et al., 2014). As is the case with most corpora, the user
searches for a word or phrase, and sentences including the expression from thetexts are listed. The user can then use other features such as Word Sketch, whichis “used to reveal how the words behave collocationally and grammatically”(Barrs, 2016), and Similar Words, which functions as a thesaurus. The frequency
166

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■of the word in a “hits per million” measurement is also given.
Given its functionality and accessibility, there are several potential benefits tousing SkELL for language researchers, teachers and learners. As with all corpora,
analysis of the frequency, collocations and other lexico-grammatical featuresof certain words can be undertaken by linguists. For teachers and students,
there is the possibility of SkELL assisting with language learning, especiallyas the interface is easier to use than other corpora (Kilgariff et al., 2015). As
Barrs (2016) puts it, “for language education in particular, the advancements in
simplifying the access to large-scale corpora means that corpus-based data canbecome a regular part of language learning” (p. 25).
Previous DDL Studies with SkELL
There have been several studies of Sketch Engine or SkELL applications in
the language classroom, although very few of these have taken place in Japan,
and most have focused on languages other than English. Chinese seems to be themost-commonly analysed (Kilgariff, Keng & Smith, 2015; Smith et al., 2008).
There have also been attempts to outline particular methodologies with which
SkELL could be used in the classroom (Thomas, 2016).
One of the few pieces of research on SkELL used in the English language
classroom in Japan was conducted by Hirata and Hirata (2019). In their study,
26 female university students were trained in using SkELL before being asked
to create dialogues and text utilizing the corpus, after which they presented
and discussed them in the lesson. Finally, they would identify difficult words,
answer questions about them and attempt to make their own texts using the target
vocabulary. After the process was complete, the students were surveyed about
their reactions to SkELL and their impression of English education in Japan. On
the positive side, the results showed students generally thought SkELL was more
useful for study than dictionaries, most rated SkELL as user-friendly, and the
majority enjoyed the opportunity to use authentic materials. However, there weremixed responses in terms of whether SkELL or dictionaries assists with learning167

more, and just 46% of respondents said SkELL increased their motivation to
study. The study concludes that it would be best for teachers to carefully selectwhich sentences from SkELL the students use for study, and that it is useful forthe students to use SkELL on their phones or tablets (Hirata & Hirata, 2019).
Previous DDL Studies Without SkELL
There has been a wealth of investigations into DDL in the language classroom
using corpora aside from Sketch Engine and SkELL. Most of these have been
concerned with whether using DDL can result in improvements in student learning, specifically by using pre and post-tests to measure vocabulary acquisition,and often by comparing with the performance of a control group.
A meta-analysis of 29 studies, all of which compared DDL with a control
group, conducted by Lee et al. (2019) found that positive-to-medium effect sizes
were recorded for both investigations which took place over short-term andlong-term periods. Their results also suggested that advanced students improved
more significantly. Similarly, a large meta-analysis by Boulton and Cobb (2017)
looked at the results of 64 studies and also found that the vast majority resulted
in significant gains in test scores from DDL treatment. A smaller meta-analysis
by Mizumoto and Chujo (2015) looked exclusively at DDL applications in Japan,
and of the only 14 they found, 12 were carried out by one of the two authors ofthe meta-analysis, suggesting a lack of research into DDL in Japan in general.
This also meant almost all the studies had the same types of students, which
tended to be low-level, and the authors themselves urged other researchers to
conduct more research on DDL in Japan (Mizumoto & Chujo, 2015).
The settings, participants, and methods by which DDL has been implemented
in studies have varied considerably. Studies using DDL often have a control
group learning with a dictionary for comparison, such as the one by Karras(2015) with Vietnamese students, which found an advantage for DDL overdictionaries, but there are also those where a traditional method of teaching iscontrasted with DDL. In the investigation by Troy and Millar (2019), a scaled168

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■down version of the British National Corpus called BNCweb was utilized withpaper-based concordance lines used in the classroom, with another class learning the same vocabulary via a more traditional approach for comparison. The
findings showed test score rises for both groups, although with this study there
was a small but not statistically significant advantage for the non-DDL class.
There has also been comparison of how effective DDL can be if used directlyor deductively (e.g., using the corpus with a computer, tablet or phone; moreautonomous), versus indirect or inductively (e.g., with paper-based materials;
more teacher-controlled). Lee and Lin (2019) found both approaches effectivein their study of EFL students in Taiwan, although they argue that the inductive
approach “prevents learners from fully benefiting from its (DDL’s) potential
advantages” (p. 24).
Student and teacher reactions to DDL are usually included as supplementary
findings in studies rather than the primary focus. Common positive responses
to DDL from students include that it uses authentic language (Chambers, 2005;
Yoon & Hirvela, 2004), it allows for more autonomous learning (Friginal,2018), and that students tend to respond positively to the opportunity to learn bydiscovery (Chambers, 2007; Chambers, 2010; Yoon & Hirvela, 2004). However,
DDL is often criticised for being too time-consuming, technology-dependentand requiring significant training for teachers and students to use effectively(Gilquin & Granger, 2010).
Research Questions
Given the stated aim of garnering student and teacher reactions in order todevelop an efficient method of using DDL in the EFL classroom, the following
research questions were formulated for this study:
1.  ‌To what extent do the students enjoy using DDL via SkELL for vocabulary learning?
2.  To what extent do the students feel learning is achieved using thismethod?
169

3.  How effective is an action research-based approach to refining a method
for DDL implementation?

MethodSetting and Participants
The research was conducted at a university in Japan with one class of EFL
students. In total 16 second-year students took part, including 14 females andtwo males, all of whom were over a lower intermediate to upper intermediate
proficiency level. Although a relatively small study, the size is typical of DDL
studies in general (Boulton & Cobb, 2017). All participants signed a consent
form before the course, agreeing that their participation would be used forresearch purposes, and their gender and age may be identified, but not any
personal information such as their name. None of the students had ever used or
encountered SkELL before.
SkELL Implementation
The classes took place in 2019 and were part of a course called ‘Core English
3’, which is a second year EFL class for students who have chosen English as
their major. In this class, six new advanced vocabulary connected with the topic
are studied each unit, with there being four units during the semester for a totalof 24 words. The lessons focusing on these new vocabulary occurred once every
three or four weeks.
The class members were told not to do the textbook work on the vocabulary,or to check the meaning in a dictionary, and instead were given a training session
on how to use SkELL. In the training session, the students were shown how tosearch for a word on SkELL, as well as the functions of the corpus. They werethen asked to search for a word, assess its frequency, read four teacher-chosensentences with the word and discuss its form and meaning with a partner, before
checking Word Sketch for more information and writing an example sentencewith the word. This methodology would then be updated after each lesson based170

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■Figure 11.   Go to skell.sketchengine.co.uk and search for incredulous.
2.   Look at the hits per million. Is this a common word?
3.  ‌Now read sentences 3, 6, 13 and 30. What type of word is incredulous?
What can you tell about the word’s meaning from the sentences?
4.   Now click on ‘word sketch’. Which words are often used with incredulous?
5.   Try writing an example sentence for incredulous.

on student survey responses and teacher observations. This process was repeatedfour times (one for each unit). The students were allowed to use a mix of English
and Japanese when discussing with their partner. This method is similar to the
one recommended for SkELL use in the language classroom by Kilgariff et al.
(2015). Figure 1 illustrates the steps to the initial methodology.
Data Collection
Data were collected mostly via student surveys. The students were surveyed
after each of the four lessons when SkELL was used, with a longer questionnaireasking for more detailed reactions given at the end of the course. These surveys
all asked about enjoyment and degree of learning, but also what they thought
could be improved about the method. Based on this feedback and the teacher’s
own observations on the method efficacy, the method was then adapted forthe next application. This is an action research approach, where a particularmethod is attempted, then adapted, then tried again, and this process is repeateduntil (it is hoped) an optimal methodology is developed (Loewen & Plonsky,2016). It is argued by McNair (in Friginal, 2018) that using action research inorder to achieve a more effective method for DDL is an area in need of greater
exploration.
At the end of the course, the students were also asked if they would be willingto participate in a follow-up survey on whether they still used SkELL six months
later in early 2020. Nine students indicated they would be willing to do this, but
in the end only two completed it.

171

Findings
Action Research in the DDL Class
After each application in the DDL class, the procedure was edited to reflect
student feedback and teacher observations. A summary of the most common
feedback from students, as well as the changes to methodology based on the
feedback received can be seen in Table 1.
Table 1Class

Areas identified as inneed of improvement(no. of students)

Areas identified as inneed of improvement(by teacher)

Changes for subsequent
application

1- More time needed to
do the tasks (2)- Want feedback to
see if their example
sentences wereappropriate (4)- Some pairs included
two low-level students
unable to cope
- Word Sketch
confused somestudents- Give more time
- Only recommend Word
Sketch for some words- Do feedback at the end
highlighting good example
sentences- Groups of three or four
instead of pairs- Allow students to checksimilar words on some words

2- More time needed (2)- Sometimes still do
not know the meaningat the end of theprocess (2)- Students spend too
long on each word

- Do one word beforehand as
homework to increase classtime- Use “good” example
sentences from a variety ofstudents at the end to increase
motivation- All students to check similarwords on any they do notunderstand after partner
discussion

3- More time needed (3)- Some words are
difficult to learn
through SkELL (2)- Students still
spending too long oneach word- Sense of achievement
is somewhat lost for
students if they cannot
complete the process- Encourage students to moveon after a maximum of fiveminutes per word
- Partners instead of groups to
speed up the process- Spend more time on feedback
at the end to ensure students
feel they understand the words

4- Similar Words andWord Sketch were notalways useful (1)- Difficult tounderstand the nuance
(1)

172

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■The final method proved to be the most efficient in terms of time management, and it was notable that no students wrote that more time was needed in the
final survey, even though this had been a frequent complaint in all the previous
questionnaires. By the end of the course, I could conceive of no obvious changes
that I felt would significantly improve the methodology.
Student Enjoyment
The first question on each survey the DDL class took asked them to score theirlevel of enjoyment of the methodology for vocabulary learning on a scale of 1(“I loved it”) to 5 (“I hated it”). The descriptive statistics showing the averages,
range and mode in the DDL class are displayed in Table 2.
The results show that in general the students enjoyed the SkELL method
throughout the course, with no student ever saying they disliked this technique,and the average hovering around 2 (“I liked it”) in all four lessons. However,there was no clear increase in enjoyment of SkELL and DDL as the course
progressed, even as the method was adapted.
In terms of student comments, in all surveys they identified that being able toguess the meaning of words was enjoyable. This was mentioned by at least four
students in every survey, and in the final survey seven students wrote this wasa reason for enjoying the approach. It was notable that after the first application
several students said they enjoyed the novelty of trying a new way to learn new
Table 2
DDL ClassLessonAverage

Range: 1 (I loved it) – 5 (I hated it)

Mode

1

2.15

1 (I loved it) – 3 (It was okay)

2 (I enjoyed it)

2

1.92

1 (I loved it) – 3 (It was okay)

2 (I enjoyed it)

32

1 (I loved it) – 3 (It was okay)

2 (I enjoyed it)

4

2.08

1 (I loved it) – 3 (It was okay)

2 (I enjoyed it)173words, but this was not mentioned on any follow-up surveys. On the other hand,
after students were allowed to use the Similar Words tool, this started being
mentioned as a positive feature of the learning method in the final two surveys.
Student Assessment of Learning
The second question asked in the surveys was for the students to assess if themethod was helping with their learning of vocabulary. As with the question on
student enjoyment, the scale was 1–5, with this time 1 representing “Yes, completely”, and 5 indicating “No, not at all”. The results are illustrated in Table 3.
The assessment of vocabulary learning in the DDL class was overwhelmingly
positive throughout the course. The average response score remained between
1 and 2 in every survey, and there was a slight improvement in the students’views on their learning from the beginning of the course to the end, although
this mostly occurred after the second application and then remained stable forthe final two classes. There was no significant change in the range or mode of
the answers given.
The most frequent positive feedback from students about vocabulary learning
through SkELL was that it helped them to remember the meaning. This was often
explained with reasons like “imagining helps me to remember” and “I can think
more deeply about the meaning this way”. One student even compared SkELL
with learning via a dictionary by asserting that “if I search with dictionary, it’s
Table 3
DDL ClassLessonAverage

Range: 1 (Yes, completely) –5 (No, not at all)

Mode

1

1.92

1 (Yes, completely) – 3 (Sort of)

2 (Yes, somewhat)

2

1.75

1 (Yes, completely) – 3 (Sort of)

2 (Yes, somewhat)

3

1.77

1 (Yes, completely) – 3 (Sort of)

2 (Yes, somewhat)

4

1.77

1 (Yes, completely) – 3 (Sort of)

2 (Yes, somewhat)174

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■easy to search but easy to forget. Using SkELL, I can remember for long time”.
On the other hand, some students struggled to understand the meaning even after
the process was complete, and two students mentioned explicitly that they would
rather learn via another method, such as a translation device or illustrations.
End of Course Surveys
The students were given a short questionnaire at the conclusion of the course.
They were asked to compare SkELL with learning via a dictionary, whether theywould use SkELL in the future and to list any other comments. The results ofthis survey are illustrated in Table 4.
The students that preferred dictionaries pointed to their greater speed andconvenience, while those who chose SkELL repeated the reasons given in previous surveys, such as the deeper understanding of meaning and that the meaning
is easier to remember that way. The majority of respondents said they would use
SkELL in the future, and they liked the various features such as Word Sketch
and Similar Words, as well as wanting to use the example sentences so they feelthey are using a word naturally. The students who do not plan to use SkELL in
the future mostly said it was too difficult for them to use, or that it takes too long.
There was also a delayed survey sent out to the DDL class students who
volunteered six months after the course conclusion to check if they were stillusing SkELL. Only two students answered the survey; one said they still used
Table 4
Question

Answers

1 - Comparing using
SkELL to learn words andjust checking the meaning
in a dictionary, which do
you prefer?

Dictionary: 46.2%

SkELL: 53.8%

2 - Do you think you will
use SkELL to learn words
in the future?

Yes: 61.5%

No: 30.8%

175

Unsure: 7.7%

SkELL, while the other said they did not. Both students still felt very positive
about having practised vocabulary learning with SkELL, and expressed a desire
to learn via a corpus more in the future.
Teacher Observations
Most of the observations I made regarding the strengths and weaknesses of
the teaching method matched those of the students. On the positive side, there
was much more student engagement in the DDL class in attempting to learn the
vocabulary than with previous classes who had just done the textbook activities.
Most of the students seemed to enjoy the process of deduction, and they also
benefited from being able to use Japanese with their partner when necessary.
The method also became more efficient as the classes went on, as the students
got more used to using SkELL and the process itself was streamlined.
On the other hand, time management remained an issue throughout thecourse. Although the methodology grew in efficiency, it still took up longer thanseems justifiable given vocabulary practice was only a supplementary rather than
primary goal of the course. I tried to act as facilitator throughout the activities,
giving hints to students that were struggling and pushing along those that weretaking too long, but this meant that every time the process seemed rushed.
One important observation I made during the course was that positive feedback at the end of the activity is vital. The students appreciated the checking of
example sentences at the end as it not only showed them the “correct” meaningof the expression, but also boosted their motivation when it was their sentencethat was selected to be shown. By the end of the course, I attempted to ensureeach student had at least one of their sentences displayed as a “good” example,
as I could see the positive impact it had.

Discussion
Assessment of Enjoyment and Learning
It is clear from the student responses that in general they enjoyed the DDL
176

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■method of learning vocabulary. The least positive answer received about themethodology overall was that “it was okay”, and even that answer was theleast common of those given. This enjoyment of using a corpus in the language
classroom matches the findings of previous studies (Chambers, 2005; Chambers,
2007; Hirata & Hirata, 2019; Yoon & Hirvela, 2004). Students often pointed tothe satisfaction of discovering word meaning via a deductive strategy, and thishas been noted as one of the benefits of DDL (Chambers, 2007; O’Keeffe &McCarthy, 2010; Yoon & Hirvela, 2004). Similarly, the fact that the process isperceived to be too time consuming by students and teachers has been observedin past investigations (Gilquin & Granger, 2010).
One significant aspect of the student feedback in terms of enjoyment wasthat the number of specific negative comments from students in regard to themethodology decreased significantly as the course progressed. In the survey after
the first application of DDL, 12 “areas to improve” comments were received
from students. This fell to eight after the second time, rose to ten after the third,
before finally falling to just two following the final class. In the post-course
survey, five negative comments about using SkELL were made. This reduction
in issues students had with SkELL may suggest that the process of actively using
a corpus and improving on how it is used by taking into account student feedback
reduced any issues they had with the method. This seems to endorse McNair’s(in Friginal, 2018) argument that action research is an appropriate way to findeffective methods of DDL delivery. Moreover, with some researchers suggesting
greater teacher training in DDL would improve and increase its application in
the language classroom (McNair in Friginal, 2018, Tribble, 2015), it seems fromthis feedback that greater student exposure to DDL would also have similarly
positive results.
Although assessing any learning that took place in the classroom requires
more quantitative results, this investigation did allow for the students’ selfassessment of their learning. The results showed that the students overwhelm177

ingly felt the DDL method had been effective at assisting them with vocabulary
learning. As was mentioned, there is strong evidence that DDL can assist withgains in test scores related to learning vocabulary (Boulton & Cobb, 2017; Lee,
Warschauer & Lee, 2019), but there has been less research on how much students
feel they are improving. The results of this study, therefore, which showed thestudents felt even more positive about their vocabulary learning than they did
about their levels of enjoyment of the methodology, suggest that they do feel
DDL improves their ability to learn. The slight preference of those who took the
DDL class for using SkELL to learn vocabulary over dictionaries was further
evidence of the faith students developed in the method, and this was broadly inline with the results of Hirata and Hirata’s (2019) study.
Teaching Methodology
Having been through four iterations, by the end of the course a final teaching method utilizing SkELL for vocabulary learning had been developed. The
instructions for the students when working on the word “accomplishment” areoutlined in Figure 2.
The students used iPads to search for the words while working in pairs. Since
each pair only received one iPad, this forced the students to work together since
they had to browse with another person. The steps varied somewhat dependingon the word, as I would assess each word beforehand to see whether using the
Figure 2
Go to https://skell.sketchengine.co.uk/
1.   Search for the word accomplishment.
2.  Read sentences 22, 27 and 36. What type (noun, verb, adjective, etc.)
of word do you think it is? What might the meaning be?
3.  If you wish, check the ‘modifiers of accomplishment’ section of word
sketch.
4.   If you want to, check the similar words section.
5.   Try writing an example sentence using accomplishment.

178

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■Figure 3
1.   Search for the word flourish.
2.  Read sentences 7, 11, and 29. What type (noun, verb, adjective, etc.) of word
do you think it is? What might the meaning be?
3.   Try writing an example sentence using flourish.

Word Sketch or Similar Words functions was useful for that particular expression
or not. As an example, Figure 3 shows a word with fewer steps.
With the word “flourish”, the information provided through Word Sketch
and Similar Words was not deemed to be useful for the students, so they were
not included in the process. As mentioned, the students would do one word bythemselves for homework before class, and the other four or five in the lesson.
The process differed from the original shown in Figure 1 mostly in that there isless for the students to do. Instead of reading four sentences, this was reduced
to three. Moreover, using Similar Words or Word Sketch was made optional forsome expressions, and removed as a step entirely for others, as was the case with“flourish”. Once all the pairs had finished discussing the words and writing their
example sentences, “good” sample sentences would be displayed on a projector.
Deciphering which sentences were deemed to be “good” examples was done by
the teacher while monitoring the class.
This structure for using SkELL for vocabulary learning bears some similarities with that of Hirata and Hirata (2019), as both include looking words up in
the corpus and writing text using the target language. However, the process wasless controlled in their study, as the students themselves identified words to lookup with SkELL rather than having words chosen for them to research. It wasalso noted in Hirata and Hirata’s (2019) study that some students found being
presented with the 50 sentences per page that SkELL provides “overwhelming”(p. 246), so it is perhaps preferable to have just a few sentences identified as
useful, as was done with this final version of the methodology.
This approach to having students learn vocabulary with SkELL has three179main advantages. First, the process is relatively fast for DDL, as the students
need only read three sentences on SkELL, and if they soon feel they understandthe word, they only need to write an example sentence to demonstrate this. This
is in contrast to my previous study (Troy & Millar, 2019), where there was anaverage of eight sentences per word, as well as comprehension questions for the
students to answer. The second major advantage is that the students have several
chances to discover meaning. While the main objective is to gain an understanding of the word by reading the target sentences, the students also have Similar
Words and Word Sketch to assist them if necessary, along with the assistance of
their partner and the teacher. Even when the students fail to grasp the meaning
after all of those steps, there is feedback at the end when students can see theirclassmates’ example sentences. Finally, the student feedback shows that theygenerally enjoyed the process more than their usual method of using a dictionary, and a small majority thought it was better for their learning too. Although
the process was still time consuming even after four attempts, its efficiency
increased as the course went on.

Conclusion
In answer to the first of the research questions this study asked, it was foundthat the participants generally enjoyed the process of using SkELL. Moreover,they overwhelmingly felt the method assisted them with vocabulary learning.
The argument can also be made that using action research did assist with making
the process more efficient and effective, although this is based mostly on theteacher’s assessment, as student reactions got only slightly more positive as thecourse went on.
Despite the attempt to make this study longitudinal by surveying students six
months after the course, the lack of responses meant that whether the students
felt the method effective enough to continue on their own or not could not be
determined. Another limitation of this study was that it was small, with just 16
180

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■students taking the DDL class. However, this is not unusual for DDL studies, as
in Boulton and Cobb’s (2017) meta-analysis 19 of the 64 studies had 16 participants or fewer. The focus of this study was also narrow, with only qualitative
data acquired rather than an investigation of more quantitative findings, such as
potential gains in test scores. Nonetheless, this was done because the majorityof DDL studies concentrate mostly on quantitative data, and it is not possible todevelop an effective method for implementing corpora more widely in the language classroom without a greater focus on what students want from the process.
This paper echoes Hirata and Hirata’s (2019) call for larger scale studiesof this type to get the views of a wider range of students. Particularly useful
would be studies done with SkELL for different levels of students, to assess
the system’s effectiveness at each level, or even with a different corpus madespecifically for a certain level of student. There also needs to be greater qualitative research done on a longitudinal scale. This was an aim of this study, but
a far higher number of student responses are needed to better ascertain how
effective SkELL can be in the long term. Most of all, more applications of DDL
in the language classroom need to focus on efficiency and convenience, as thetime-consuming nature of the process as well as its perceived difficulty remain
the biggest roadblocks to its wider usage. As was the case with this study, action
research is perhaps the most effective way to achieve this, and I am in agreementwith McNair (in Friginal, 2018) that “action research is how teachers spread
corpus-based lessons to other teachers” (p. 232).
It is my belief that SkELL can be useful in language classrooms for vocabulary learning. Although some training is required for both teachers and students
to use it effectively, it is user-friendly enough that this training need not last long,and it can for the most part be done reactively via teacher moderation, facilitationand feedback. I would also argue that corpora are still underused in the language
classroom in general, and those that believe DDL should be used more widely
need to endeavour to make it as easy, effective and efficient as possible so as to181persuade those less eager to include it in their classes and curriculums.

References
Barrs, K. (2016). Using the Sketch Engine Corpus Query Tool for Language Teaching. The Language
Teacher, 40(2), 23–26.
Boulton, A., & Cobb, T. (2017). Corpus Use in Language Learning: A Meta-Analysis. Language
Learning, 67(1), 1–46. http://dx.doi.org/10.1111/lang.12224
Breyer, Y. A. (2011). Corpora in Language Teaching and Learning: Potential, Evolution, Challenges.
Peter Lang.
Chambers, A. (2005). Integrating Corpus Consultation in Language Studies. Language Learning andTechnology, 9(2), 111–125.
Chambers, A. (2007). Popularising corpus consultation by language learners and teachers. In Hidalgo,
E., Quereda, L. and Santana, J. (Eds.), Corpora in the Foreign Language Classroom (1–16).
Rodopi.
Chambers, A. (2010). How can data-driven learning be used in language teaching? In O’Keeffe, A.
and McCarthy, M. (Eds.), The Routledge Handbook of Corpus Linguistics (345–358). Routledge.
EF English Proficiency Index. (2020). Retrieved from https://www.ef.com/wwen/epi/
Friginal, E. (2018). Corpus Linguistics for English Teachers: New Tools, Online Resources, andClassroom Activities. Taylor & Francis.
Hirata, Y. (2018). E-learning courseware for language education in Japan: its application and student
perceptions. Open Learning, 33(2), 83–98.
http://dx.doi.org/10.1080/02680513.2018.1454833
Hirata, Y. & Hirata, Y. (2019). Applying ‘Sketch Engine for Language Learning’ in the JapaneseEnglish classroom. Journal of Computing in Higher Education, 31, 233–248.
http://dx.doi.org/10.1007/s12528-019-09208-z
Johns, T. (1986). Micro-Concord: A Language Learner’s Research Tool. System, 14(2), 151–162.
http://dx.doi.org/10.1016/0346-251X(86)90004-7
Johns, T. (1988). Whence and Whither Classroom Concordancing? In Bongaerts, P. de Haan, P. et al.
(Eds.), Computer Applications in Language Learning (9–27). Dordrecth: Foris.
Johns, T. (1991). Should you be persuaded?: Two examples of data-driven learning materials. English
language research journal, 4, 1–16.
Jones, B. A. (2019). The Role of English Education in Japan. Memoirs of Learning Utility Center for
Konan University Students, 4, 21–31.
Karras, J. N. (2015). The effects of data-driven learning upon vocabulary acquisition for secondary
school students in Vietnam. ReCALL, 28(2), 166–186.
http://dx.doi.org/10.1017/S0958344015000154
Kilgariff, A., Baisa, V., Busta, J., Jakubicek, M., Kovar, V., Michelfeit, J., Rychly, P., & Suchomel,
V. (2014). The Sketch Engine: ten years on. Lexicography, 1, 7–36.
http://dx.doi.org/10.1007/s40607-014-0009-9

182

Learning Vocabulary with SkELL: Developing a Methodology with University Students in Japan Using Action Research ■Kilgariff, A., Keng, N., & Smith, S. (2015). Learning Chinese with the Sketch Engine. In Zou, B.,
Smith, S., and Hoey, M. (Eds.), Corpus Linguistics in Chinese Contexts (63–73). Palgrave
Macmillan.
Kilgariff, A., Marcowitz, F., Smith, S., & Thomas, J. (2015). Corpora and Language Learning with the
Sketch Engine and SkELL. Revue Francaise de Linguistique Appliquee, 20(1), 61–80.
http://dx.doi.org/10.3917/rfla.201.0061
Lee, H., Warschauer, M., & Lee, J. H. (2019). The Effects of Corpuse Use on Second Language
Vocabulary Learning: A Multilevel Meta-analysis. Applied Linguistics, 40(5), 721–753.
Lee, P. & Lin, H. (2019). The effect of the inductive and deductive data-driven learning (DDL) on
vocabulary acquisition and retention. System, 81, 14–25.
http://dx.doi.org/10.1016/j.system.2018.12.011
Lenko-Szymanska, A. (2017). Training Teachers in data-driven learning: Tackling the challenge.
Language Learning & Technology, 21(3), 217–241.
Loewen, S. & Plonsky, L. (2016). An A-Z of Applied Linguistics Research Methods. Palgrave.
McCarten, J. (2010). Corpus-informed course book design. In O’Keeffe, A. & McCarthy, M. (Eds.),
The Routledge Handbook of Corpus Linguistics (413–427). Routledge.
McNair, J. (2018). Using a Concordancer for Vocabulary Learning with Pre-Intermediate EFL
Students. In Friginal, E. (Ed.), Corpus Linguistics for English Teachers (224–232). Taylor &Francis.
Mizumoto, A. & Chujo, K. (2015). A meta-analysis of data-driven learning approach in the Japanese
EFL classroom. English Corpus Studies, 22, 1–18.
O’Keeffe, A. & McCarthy, M. (eds.) (2010). The Routledge Handbook of Corpus Linguistics. London:
Routledge.
Sketch Engine. (2020). Retrieved from https://www.sketchengine.eu/skell/
Smith, S., Chen, A., & Kilgariff, A. (2008). A corpus query tool for SLA: learning Mandarin with thehelp of Sketch Engine. In Lewandowska-Tomaszcyk, B. (Ed.), Corpus Linguistics, Computer
Tools, and Applications: State of the Art (673–686). Peter Lang.
Thomas, J. (2016). Discovering English with Sketch Engine: A Corpus-based Approach to Language
Exploration. 2nd ed. Versatile.
Tribble, C. (2015). Teaching and language corpora: Perspectives from a personal journey. In LenkoSzymanska, A. & Boulton, A. (Eds.), Multiple Affordances of Language Corpora for Data-driven
Learning (37–64). John Benjamins.
Troy, H., & Millar, N. (2019). Evaluating the use of data-driven learning to teach phrasal verbs at a
private language school in Japan. Bulletin of Nagoya University of Foreign Studies, 5, 321–347.
Yoon, H. & Hirvela, A. (2004). ESL student attitudes toward corpus use in L2 writing. Journal of
Second Language Writing, 13, 257–283. http://dx.doi.org/10.1016/j.jslw.2004.06.002

183