名古屋外国語大学論集第5号

2019年7月

ArticleEvaluating the Use of Data-driven Learning to Teach
Phrasal Verbs in a Private Language School in Japan
Henry Robert TROY, Neil MILLAR

Abstract
This paper assesses the use of data-driven learning to teach phrasal verbs at a
private English language school in Japan. Concordance lines from the British
National Corpus were utilised with the aim of allowing students to discover themeaning of a set of phrasal verbs over the period of one month. This kind of
application of corpora in the language classroom has for the most part been thedomain of university courses. By gathering both quantitative and qualitative
data from the classes, the study aimed to ascertain the usefulness of this type oflearning for teachers in smaller language schools. Overall, the results suggestthat data-driven learning did assist the students in acquiring the target language,
with motivation levels also showing an increase on the whole. The results imply
that a corpus can be used effectively by language teachers outside of universities,
but that the time-consuming nature of this endeavour may make it impracticalfor some instructors.

1. Introduction
Corpora have had a tangible impact in the field of language teaching. Their321

influence is perhaps most visible in commercial materials where publishers areoften keen to foreground the corpus credentials of their products. For example,
learner dictionaries typically draw on corpus evidence, both from target language
corpora (for description of the language) and learner corpora (for descriptionsof errors that learners typically make) (Longman Essential Activator, 1997).
Similarly, many textbooks draw, to varying degrees, on corpus data (McCarthy,
McCarten & Sandiford, 2014; Handford, Lisboa, Koester & Pitt, 2011). For
learners and teachers, these examples represent the indirect use of corpora – i.e.
the data has been distilled into easily accessible descriptions of language use.
An alternative is the direct use of corpus data in the classroom – an approach
which has come to be known as Data Driven Learning (DDL) (Johns 1986;
1988; 1991). Here students take on the role of ‘researchers’ in the classroom
using authentic corpus data, usually in the form of concordance lines, to identify
language patterns, while the teacher’s role becomes that of a facilitator andguide. Materials are typically presented in paper format, with the target word orphrase centred in the concordance (key word in context – KWIC), making lexicogrammatical regularities easier to identify. Students are then provided with tasks
to guide them through the process of independently identifying form-meaning
relationships from multiple examples.
As such, DDL represents a “cognitive” strategy for language acquisition.
Cognitive theories of SLA state that “mental operations” (i.e. working to discover meaning) are necessary for learning new words “in order to understand,
categorize and store them in the mental lexicon” (Hedge, 2000, p. 117). The
process of Cognitive SLA starts with exposure to the target language in context,
followed by organizing the words via the patterns noticed, before finally “going
beyond the data” by inferring meaning and attempting to produce the words
independently (Brown, 2014, p. 126).
Proponents suggest that DDL “can significantly enrich the pedagogic environment” (Aston, 1995, p. 261). Kennedy and Miceli (2001) and Yoon and Hirvela
322

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■(2004) argue that since the process involves the students discovering meaning
mostly of their own fruition, this can result in increased student confidence
and motivation. Similarly, Harris and Moreno Jaen (2011) suggest that as thisapproach is more learner-centred, it might improve student motivation (see also
Grigaliuniene, 2013), while Allan (2006) suggests DDL can lead to greater
student participation and language acquisition. Moreover, research has also
indicated that students appreciate being given access to authentic language(Cheng et al., 2003; Chambers, 2005; Boulton & Cobb, 2017).
Data-driven learning has been shown, on the whole, to have a positive impact on
student learning, in particular vocabulary learning (Stevens 1991; Cobb 1997;
Allan 2006; Koosha & Jafarpour 2006; Boulton 2011; Yilmaz 2017). Yilmaz(2017), for her part, found that in the context of a Turkish university DDL
resulted in significantly more gains than studying with an online dictionary,and concludes that “DDL can have a strong effect on learners’ improvementin the use of lexico-grammatical patterns” (p. 85). However, not every study
has found DDL results in significant gains in test scores, with Rapti (2001) an
example of this.
A drawback of DDL appears to be its time-consuming nature both for theteacher, in terms of material and classroom preparation, and for the students,
in learning to utilise a method they are likely to be unfamiliar with (Kennedy& Miceli, 2001; Cheng et al., 2003; Chambers, 2005; Boulton & Cobb, 2017).
The approach also places the focus overwhelmingly on individual words. As
McCarthy (1990) explains “over-concentration on learning single words mayhinder the development of the L2 phrasal lexicon and deny the opportunities
this gives for rapid retrieval and fluent, connected speech” (p. 45). Furthermore,some students may simply dislike the approach and prefer a more traditional
teacher-centred classroom or a more deductive approach (Whistle, 1999, citedin O’Keeffe & McCarthy, 2011, p. 365).
Research on DDL is a flourishing field – Boulton and Cobb (2017) cite 205323

publications reporting empirical evaluation of DDL. However, as an approach tolanguage teaching DDL remains relatively marginalized. A survey of publishedstudies suggest that it remains almost exclusively the domain of university(writing) classes. The application of DDL in private language schools is uncommon – of the 205 cited by Boulton and Cobb (2017), only one takes place in a
private language institution. Possible reasons for this might include less teacher
autonomy in language schools, a lack of awareness of the approach, and practicalissues, such as access to classroom technology. Furthermore, language school
teachers, particularly in Japan, might also face challenges implementing anapproach that requires paying customers, who may be more familiar with ateacher-centred teaching/learning environment (Lee, 2007), to adopt a moreautonomous role.
To address this gap, the present study explores the applicability of DDL in a
private English language school in Japan. Specifically, the study focuses on thelearning of phrasal verbs. A phrasal verb is a combination of lexical verb andverbal particle (usually an adverb or preposition), the meaning of which can be(1) literal – it matches that of the individual words (e.g. take out the trash); (2)aspectual – it is not completely transparent but nor is it completely figurative(e.g. use up all the supplies); or (3) idiomatic (e.g. chill out) – it is difficult todecipher from its individual components (Thim, 2012). The grammatical formof phrasal verbs can also pose difficulties for learners. Phrasal verbs are either(1) transitive type A – where the verb can take an object, and its two parts can be
broken up (e.g. turn on the radio/turn the radio on), (2) transitive type B – wherethe verb can take an object, but its two parts cannot be separated (e.g. count on afriend), (3) intransitive – where the phrasal verb cannot take an object and cannot
be separated (e.g. go back), and (4) three-word phrasal verbs (e.g. look forwardto), which have the same grammatical properties as intransitive phrasal verbs.
Knowing whether the phrasal verb can or cannot take an object, and whether it
is able to be separated or not, is another skill learners need to acquire in order324

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■to use them effectively.
Phrasal verbs were selected as the target structure for this study as they are often
avoided by language learners (Laufer & Eliasson, 1993; Gardner & Davies,
2007) and are considered one of the more difficult elements of English toteach and learn (Moon in Schmitt & McCarthy, 1997; Celce-Murcia & LarsenFreeman, 1999; Gardner & Davies, 2007; Gonzalez in Campoy-Cubillo et al.,2010; Ryoo, 2013). Sinclair (1991) suggests that concordance lines might beparticularly effective in highlighting non-transparent meanings of phrasal verbs.
Only a few published studies have assessed the use of DDL to teach phrasal
verbs. Azarro (2012) tested the acquisition of ten phrasal verbs for some 112
university students. Five of the verbs were taught via concordance lines, while
for the other five dictionaries were used. The post-test showed significant gains
in test scores for words studied by DDL, with less significant improvements
for dictionary-learnt words. Phrasal verbs were also taught via DDL by Spring(2018). Once more, there were significant gains in test scores for the Japanese
university students who participated, although the study focused more on how
the phrasal verbs themselves should be presented than the effectiveness of the
DDL treatment. Just two phrasal verbs were studied by Boulton’s (2008) 113
students as part of his study on using DDL to assist with phrasal verb acquisition, and results again showed significant rises in test scores. In this study, thegreatest improvements were observed in higher level learners (Boulton, 2008).
All of these studies focused on university students, again highlighting the lackof DDL application outside higher education.

2. Methods
2.1 Evaluation purpose
To assess the efficacy of DDL in the context of a private language school, we
employed a micro-evaluation (Ellis, 2003). This comprised a ‘learning-based
evaluation’ assessing the extent which DDL resulted in student learning of the325phrasal verbs. To this end, a pre-test, a treatment (i.e. DDL) and a post-test wereadministered to a group of students. For comparison, a pre-test, treatment andpost-test using a non-DDL approach was administered to a separate group of
students in the same school. We also conducted a ‘response-based evaluation’,the purpose of which was to assess whether the use of DDL resulted in the
expected classroom processes, and a ‘student-based evaluation’ assessing how
students react to DDL. Finally, we sought to evaluate the ‘overall feasibility’of using DDL is this context.
2.2 Setting and participants
The study was conducted in a private English conversation school in Nagoya,Japan. A total of 16 adult students, split equally into two groups, took part. Boththe DDL group and non-DDL group comprised of seven females and one male.
Students’ ages ranged from 16 to 70, with most between 26 and 50 years old,while their levels, as assessed by the teacher and classes they were taking, werepredominantly intermediate or advanced. This range, although not representative of all private language schools, generally reflects the diversity found atsuch schools in Japan. Students took part in the study voluntarily and provided
informed written consent. Although they were aware that the class would focus
on phrasal verbs, they were unaware of the specific teaching approach. Bothclasses were taught by the first author, who was at the time a CELTA qualified
teacher of seven years’ experience undertaking a Masters in Applied Linguistics.
2.3 Treatments
Two sets of three tasks were prepared – one set based on the principles of DDL
and the other on a more traditional deductive approach (non-DDL). Each setof tasks targeted the same 22 phrasal verbs – see Table 1. Target verbs wereselected based on (1) their form – six transitive verbs (Task 1), eight intransitive verbs (Task 2) and eight three-word verbs (Task 3); (2) non-transparency
326

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■Table 1: Phrasal verbs for teaching
Task 1transitive verbs

Task 2
intransitive verbs

Task 3three-word verbs

figure out

drop ingo out with

get overget byget round tocount onback down

measure up to

turn down

nod offlook down onrule out

grow apartclamp down on

call off

watch out

make fun ofcatch onput up withfall throughfill in forof meaning – i.e. verbs that would likely pose a challenge for students; and (3)frequency in the British National Corpus – i.e. they are in common usage andthus worth teaching.
DDL tasks followed an inductive format, first presenting students with exemplars
and asking them to reach their own metalinguistic generalisations. Concordancelines which illustrated salient aspects of meaning and form were selected fromthe BNC, and, on occasion, simplified. Students were asked to read through theconcordance lines in pairs and discuss any patterns that they noticed. A numberof targeted questions then helped them notice any aspects of meaning and form,
without expressly informing them of it. Finally students were encouraged to use
the target verbs in a semi-controlled question-answer activity. Figure 1 is an
example, and full materials can be found in Appendix 1.
The non-DDL tasks followed a more traditional deductive format, where students
were first presented with a grammatical rule governing phrasal verbs. They werethen given an example sentence for each target verb and encouraged to write
their own, before the lesson concluded with the same speaking practice activity
used in the DDL tasks. Figure 2 is an example of such a task; the full materials
can be found in Appendix 2.
327

Figure 1

328

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■Figure 2

All treatments took place over one month and each of the DDL and non-DDL
classes lasted one hour.

329

2.4 Evaluation data
For the learning-based evaluation, separate, but comparable, pre- and post-tests
were administered to both groups of students. The first part comprised a selfassessment of how well they knew the target verbs – ‘know it and can use it’,‘understand it but can’t use it’ or ‘do not know it’. The second part comprised acloze test with the target verbs deleted from sentences and provided as options.
Both of these sections were objectively scored. Finally, students were asked
to write a sentence for each target verb demonstrating understanding of themeaning. This section was subjectively scored with points given correct use of
the phrasal verb. Tests scores were analyzed in SPSS (Larson-Hall, 2010) usinga Repeated Measures ANOVA, given that there was a within-groups variable,
which was time in the form of the pre-test and post-test scores of the participants,
and a between groups variable, in the type of treatment the students received,
The pre-test was administered a week prior to start of the treatment, and thepost-test a week after the completion of the treatment. Each lasted approximately
30 minutes.
For the response-based evaluation, classroom recordings were collected withthe students’ permission. For the student-based evaluation, one-on-one interviews were conducted after the course. These were undertaken to gain qualitative
insights into students’ thoughts on the class and perceptions of improvement.
The interviews were ‘semi-structured’ (Dornyei, 2007, p. 136), in that althoughquestions were decided upon beforehand, the format was flexible, allowing forparticular issues to be explored in more depth if necessary. The interviews took
place after the post-test had been completed, as it allowed for the students toassess their level of improvement both before and after their test results wererevealed.
To evaluate the overall feasibility, the instructor (also the first author) engaged
in qualitative self-reflective enquiry – i.e. a form of action research (Stringer,2014). This involved taking into account the findings of the micro-evaluation
330

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■as well as drawing on his own personal knowledge of the context. The aim wasto holistically assess whether a DDL approach would be attractive to teachers
at private language schools. In doing so, he took into account factors such asteacher preparation time, student speaking time, and overall teacher workload.

3. Results and Discussion
3.1 Learning-based evaluation
Average pre and post-test scores are shown in Table 2. The distribution of thesescores is shown in the boxplot in Figure 3. Both classes showed gains in theirmean test scores. In the DDL class, the average score in the pre-test was 8.31,
and this increased to 14.38 in the post-test. The rise was even greater for those
who undertook the non-DDL class, with the mean pre-test score for that group6.50, rising to 15.69 in the post-test. There was greater variation between scoresin DDL class; this was due to one student scoring 20 and 22 in the two pre- andpost-test, considerably higher than other students – shown as the outlier markedas ‘13’ in the boxplot (Figure 1). A Wilcoxon signed ranks test demonstrated asignificant gain in test scores in the DDL treatment (Z = –2.52, p = .012).
A repeated measures ANOVA was performed to assess whether the two treatments together had a significant effect on improvements in test scores. There
was a significant rise in test scores for both treatments, F(1, 14) = 69.39, p = <.001, and also a large effect size, ηp2 = .83. Thus, treatment in both the DDL andnon-DDL resulted in significant gains in test scores. As for the relation between
class type and test score improvements, F(1, 14) = 2.92, p = .11, with a medium
effect size of ηp2 = .17. This indicates that although there was some effect forthe class taken, the difference in test scores between the two classes was notstatistically significant.
In short, the statistical analysis indicates that students achieved significantly
better post-test scores than pre-test scores regardless of which class they took.
However, there is no suggestion that either method of learning resulted in a331

Table 2: Average pre and post-test scoresClass

Pre-test average

Post-test average

Average gain

DDL
Non-DDL(standard deviation)
8.31 (5.26)
6.50 (2.87)(standard deviation)
14.38 (3.74)
15.69 (2.90)

+6.60
+8.28

Figure 3: Boxplot of Means

statistically greater gain in test scores than the other.
Consistent with other studies, test scores in the DDL class showed a statisticallysignificant increase. For example, Spring (2018), who used a corpus to teachphrasal verbs, reports similar levels of significance. Indeed, most of the studies
reported in Boulton and Cobb’s meta-analysis (2017) report statistically significant results. Although the effect size detected in the present study, albeit for both
treatments, was ‘large’ (0.83), it is actually lower than the mean of 1.5 reported
in meta-analysis mean. Boutlon and Cobb (2017) note that lowered effect sizes
can be tied to smaller sample sizes. Although 16 participants represents a small
sample size, and, thus, a limitation of the study, it is not unusual for a study ofthis kind. Of the 64 studies in Boulton and Cobb’s (2017) meta-analysis, 17 had
fewer than 16 participants, and two had exactly 16.
332

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■As there was no significant difference between the treatment types, it can beposited that both treatments were effective in improving test scores. In otherwords, the DDL class was no more effective in helping the students to acquire
the target phrasal verbs than the non-DDL approach. As students exposure to
the target verbs was limited to the classroom (i.e. they did not review at home),
we infer that test score gains were a result of the treatment. The recordings,
interviews and student surveys provide further insights as to how this student
learning was achieved.
3.2 Response-based evaluation
The lesson recordings revealed a generally more active classroom in the DDLclass, with considerably more student speaking time. A summary of percentage
estimates of student speaking time in the two classes can be seen in Table 3.
Except for in the first lesson of the course, when a considerable amount of time
was required for procedure explanation, the teacher was mostly able to take abackseat role as monitor in the DDL class. However, the recordings also showedthat timing was a major issue for the teacher in the DDL class. On several occasions the class took up to ten minutes to analyze the concordance lines of a single
phrasal verb. Overall, the efficiency in the DDL class seemed to be low, with thesix-to-eight phrasal verbs per class barely covered in the time. This also meant

Table 3
DDL ClassStudent
TalkingTime

Non-DDL Class

Teacher
TalkingTime

Other
Student(including Talking
silence)Time

Teacher
TalkingTime

Other(including
silence)

Lesson 1

70%

20%

10%

36%

34%

30%

Lesson 2

79%

10%

11%

42%

35%

23%

Lesson 3

77%

11%

12%

41%

36%

23%

Average

75%

14%

11%

40%

35%

25%

333

there was little time for the students to practise using the phrasal verbs in thefreer speaking activity at the end of the class – between five and fifteen minutes
per pair, depending on how quickly they got through the concordance lines.
Student talk in the DDL class focused predominantly discovering the meaning
and form of the target language. Some pairs were more effective at this thanothers. For example, one pair discussed each phrasal verb in detail, and collaboratively managed to work on the target language with almost no teacher
assistance. With another pair, however, there was little collaboration with thehigher-level student dominating, while one pair required a considerable amountof teacher feedback.
Allowing for ample student talking time (STT) in language learning class isgenerally perceived to have benefits for learning (Ellis, 2008) and motivation(Dornyei, 2001). Clearly, what constitutes an appropriate balance of STT isdependent on context (e.g. aims of the class, level of the students). While Nunan(1991) recommends that students should be talking between 70 and 80 percentof the time, Davies (2011) warns that SST can be an unreliable indicator ofclassroom effectiveness. In this context, the greater SST in the DDL class canbe seen as a positive aspect of the approach. The fact that much of this STT was
spent on the discovery of meaning is also believed to be beneficial for vocabularyacquisition (Ellis, 2008), although the lack of freer practice was not ideal.
The class recordings showed that DDL is time-consuming, and this was alsomentioned by a few students in the interviews. This represents a drawback of
the DDL approach and was not unexpected. Cheng et al. (2003) and Chambers(2005) report similar issues, as do many of the studies in Boulton and Cobb’s(2017) meta-analysis. The teacher often had to push the students to work throughthe sentences faster, despite which the concordance line analysis still took up
the majority of each lesson. This gave the students a feeling of being rushed. A
possible solution to these problems would be, as one of the students suggested
in the interviews, to reduce the number of concordance lines analyzed. Another334

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■alternative would be to have fewer grammar and meaning questions.
It was hoped that as they progressed through the course, students would become
more efficient users of DDL. In terms of speed, the recordings indicate that
efficiency increased only slightly. In the first class, the students spent an average of five minutes discussing the concordance lines of each phrasal verb, but
this fell to just under five minutes in the final lesson. Moreover, as the lessons
progressed students were more likely to ‘skip’ concordance lines that they did
not understand.
In the non-DDL class, lesson timing appeared to be more consistent. The students
were able to have 15 to 20 minutes of freer speaking time at the end, and there
was even enough time left for group feedback. However, despite the longer
freer speaking segment, overall student speaking time was much less prevalentthan in the DDL class overall, as more time was spent with teacher explanation,individual student speaking time (e.g. with only one student answering a question in front of the class) and silent classwork (e.g. writing example sentences).
The recordings also showed the role of teacher in the students’ learning process.
The teacher aimed to be “facilitator and guide” rather than instructor, as recommended by Aston (1995, p. 261). For most of the DDL lessons the teacher
monitored and gave occasional feedback. This feedback was either advisory
or evaluative. Advisory feedback included intended to push the students in theright direction, such as ‘How are these two sentences similar?’ or ‘What tends
to come after the phrasal verb in these sentences?’. There were also statements
which served the same purpose, including “this sentence is a clear example” or“focus on the difference between these two”. Finally, the evaluative feedback
was utilized for time management or encouragement purposes. This feedback
included advice such as “good, that’s right, move on” or “you’re doing well,
keep going”. Overall, the teacher’s role was generally successful as it met therequirements of “facilitator and guide” instead of lecturer that had been a goal
beforehand.
335

3.3 Student-based evaluation
The majority of students reacted positively to DDL. In the interviews, four DDL
students said they preferred this method to their usual lessons, two liked bothways, while two preferred a more inductive approach. From those who preferred
DDL, comments tended to focus on enjoyment, motivation, learning method
and amount of speaking time. In terms of enjoyment, one student said that theapproach “was like a fun game; not passive study”, while another described theclasses as “fun, interesting and beneficial”. Two students specifically mentioned
motivation in their feedback, with one calling the approach a “very motivatingstyle” and another labelling it “more motivating than the usual way”. The learning method was discussed by several students, with one saying it was “good toimagine meaning” and another saying “it takes longer but is eventually more
effective”. Finally, one student spoke about the amount of STT, praising theclasses as they “include a lot of discussion with other students”. It is also worth
noting that the most positive feedback tended to come from more advanced
students.
The two students who enjoyed the course less both stated that they preferred to
be told the meaning of words rather than uncovering it themselves. The first said
simply “I prefer to be told the meaning” while the other similarly stated “thisway of learning is not for me; I prefer to be told (the meaning)”. There were also
drawbacks mentioned by the interviewees, even those who enjoyed the course
overall. The main criticism was that the process was too time-consuming, while
another was that there were too many concordance lines to get through.
The motivational benefits of DDL mentioned above seem to match those foundby Kennedy and Miceli (2001) and Yoon and Hirvela (2004). Moreover, thepositive feedback from several students on the DDL method seems to match theresults of other studies (Cheng et al., 2003; Chambers, 2005; Boulton & Cobb,
2017) which have suggested that students enjoy self-learning via authentic language. On the other hand, the two students who stated clearly that the approach
336

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■did not suit them as they would prefer a more instructive approach correspondswith Whistle’s (1999, cited in O’Keeffe & McCarthy, 2011, p. 365–367) findingsthat DDL is not liked by some students, as they find it ‘frustrating’ to have todecipher meaning when it is faster simply to be told.
The interviews demonstrated that the students did generally respond positively to
the DDL approach, which matches the expectations of the study given previous
research. Regardless of whether the students enjoyed the new approach because
it was novel or for its value in learning, their enjoyment is beneficial as motivation is intrinsically linked to learning (Dornyei, 1998).
The responses from those who took the non-DDL class were also positive overall, but less enthusiastic. The most common piece of positive feedback was thatthe course was “useful”, while another who enjoyed the course labelled it “fun”
and one said “I had more chance to speak than in my university class”. Negative
responses tended to focus on the lack of time spent speaking with others. Thiswas represented in statements such as “there was not enough time talking withothers” and “it needs more writing and speaking time”. Two students labelled
the course “difficult”, while another felt that they “didn’t get meaning fromothers, only the teacher”.
3.4 Overall feasibility
Implementing DDL in this setting was found for the most part to be feasible,
but with considerable limitations. The teacher was able to construct materials
with relative ease using BNCweb, and despite no prior experience, successfully
use them the concordance lines in the classroom. Moreover, the teacher himself
enjoyed the process. As the students grew accustomed to the new approachit actually became less work for the teacher overall, due to the fact that onlya monitoring role was required. However, timing remained an obstacle; bothpreparation time and in-class activity timing. Creating the lessons took longer
than a typical teacher at a private language schools might be expected to be337

able to spend on preparation. For example, for a regular lesson at the school inquestion, less than an hour of preparation time is considered reasonable. In total,
about 12 hours were required to prepare for all three lessons. Also, as mentioned
in 3.3, the DDL class lacked efficiency, with much more time being taken forstudents to understand each phrasal verb via the concordance lines than in the
non-DDL class.
There are potential solutions for teachers who might wish to incorporate DDL
into their lessons. Firstly, repeated use of the materials with different would gosome way to offset the time spent on materials preparation. Creating new teaching materials can be time-consuming irrespective of whether they are DDL based
or not. Furthermore, repeated use might also improve in-class efficiency as boththe instructor and students get used to the new lesson structure. With repeated
use the teacher hone the materials based on classroom experience (i.e. action
research) – for example, by identifying which concordance lines and questions
are effective and adapting the materials appropriately. Finally, the students could
even be trained to utilize a corpus at home to revise the words learned in class.

4. Conclusion
This study presented a micro-evaluation of DDL in a Japanese private language
school. Although comprehensive meta-analyses such as Boulton and Cobb’s(2017) indicate there a corpus-based approach to language teaching can resultin learning, it is still unclear whether for vocabulary acquisition it holds any
advantages over other approaches. The present study has also been unable toshow that DDL resulted in more learning than a non-DDL approach.
However, that is not to say that the findings are not positive. The learning-based
evaluation suggested that learning was achieved via this method – test scores forthe DDL class increased significantly. The response-based evaluation indicatedthat the DDL tasks seemed to result in patterns of interaction and cognitive
processes thought to be conducive to language learning – e.g. increased student338

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■talk time, a student-centred classroom, a focus on meaning and form in student
talk. In addition, the student-based evaluation showed that students reacted
positively to the approach, supporting previous claims that DDL often has apositive effect on student motivation.
One limitation of this study was that it was relatively small, although having
just 16 participants is not unusual for a study of this kind. Of the 64 studies in
Boulton and Cobb’s (2017) meta-analysis of similar research, 17 had fewer than
16 participants, and two had exactly 16. Nevertheless, a study with a greaternumber of participants would have allowed for greater quantitative analysis,
with post-hoc tests perhaps revealing more as to which type of student benefited
most from the approach.
The study has allowed the class teacher (also first author) to conclude that a DDL
approach could assist with the teaching of phrasal verbs and that DDL can beincorporated into lessons at private language schools, although the creation ofmaterials does take time. In the classroom, the teacher’s role was less prevalentthan in other classes, suggesting the approach may reduce in-class teacher
workload. For other teachers at private language schools implementing a DDL
course may therefore have benefits, but only if they or their organization havetime to create materials that would do the methodology justice.

5. References
Allan, R. (2006). Data-Driven Learning and Vocabulary: Investigating the Use of Concordances with
Advanced Learners of English. Dublin: Trinity College Dublin.
Aston, G. (1995). Corpora in language pedagogy: Matching theory and practice. In Cook, G &Seidlhofer, B. (Eds.), Principle and practice in applied linguistics: Studies in honour of H.G.
Widdowson (257–270). Oxford: Oxford University Press.
Azarro, G. (2012). Phrasal Verbs through DDL. Journal of Theories and Research in Education,
7 (1), 1–24.
Boulton, A. (2008). Looking for empirical evidence for DDL at lower levels. In LewandowskaTomaszczyk (Ed.), Corpus Linguistics, Computer Tools, and Applications. Frankfurt: Peter Lang.
Boulton, A. (2010). Data-driven learning: Taking the computer out of the equation. Language
Learning, 60 (3), 534–572.

339

Boulton, A. (2011). Data-Driven Learning: On Paper, in Practice. In Harris, T. & Moreno Jaen,
M. (Eds.), Corpus Linguistics in Language Teaching (17–52). Internationaler Verlag derWissechaften: Peter Lang AG.
Boulton, A. and Cobb, T. (2017). Corpus Use in Language Learning: A Meta-Analysis. Language
Learning, 67 (1), 1–46.
British National Corpus. (2017). BNCWeb. Retrieved from http://bncweb.lancs.ac.uk/
Brown, H. D. (2014). Principles of Language Learning and Teaching. New York: Pearson Education.
Campoy-Cubillo, M. C., Belles-Fortuno, B., & Gea-Valor, M. L. (2010). Corpus-Based Approaches
to English Language Teaching. London: Continuum.
Celce-Murcia, M. & Larsen-Freeman, D. (1999). The Grammar Book: An ESL/EFL Teacher’s Course.
USA: Heinle & Heinle.
Chambers, A. (2005). Integrating Corpus Consultation in Language Studies. Language Learning andTechnology, 9 (2), 111–125.
Chambers, A. (2010). How can data-driven learning be used in language teaching? In O’Keeffe, A.
and McCarthy, M. (Eds.), The Routledge Handbook of Corpus Linguistics (345–358). London:
Routledge.
Cheng, W., Warren, M. & Xun-fend, X. (2003). The Language Learner as Language Researcher:
Putting Corpus Linguistics on the Timetable. System, 31 (2), 173–186.
Cheng, W. (2010). What can a corpus tell us about language teaching? In O’Keeffe, A. and McCarthy,
M. (Eds.), The Routledge Handbook of Corpus Linguistics (319–332). London: Routledge.
Davies, M. J. (2011). Increasing students’ L2 usage: An analysis of teacher talk time and student
talk time. Retrieved from University of Birmingham website: https://www.birmingham.ac.uk/
Documents/college-artslaw/cels/essays/languageteaching/Daviesessay1TTTessaybank.pdf
Dörnyei, Z. (1998). Motivation in second and foreign language learning. Language teaching, 31 (3),
117–135.
Dörnyei, Z. (2001). Motivational Strategies in the Language Classroom. Cambridge: Cambridge
University Press.
Dörnyei, Z. (2007). Research Methods in Applied Linguistics. Oxford: Oxford University Press.
Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.
Ellis, R. (2008). The Study of Second Language Acquisition. Oxford: Oxford University Press.
Gardner, D. & Davies, M. (2007). Pointing out frequent phrasal verbs: A corpus-based analysis.
TESOL Quarterly, 34 (5), 1–24.
Gonzalez, R. A. (2010). L2 Spanish Acquisition of English Phrasal Verbs: A Cognitive Linguisitic
Analysis of L1 Influence”. In Campoy-Cubillo et al. (Eds.), Corpus-Based Approaches to English
Language Teaching (149–166). London: Continuum.
Grigaliuniene, J. (2013). Corpora in the classroom. Vilnius: Vilnius University.
Handford, M., Lisboa, M., Koester, A., & Pitt, A. (2011). Business advantage. Cambridge: Cambridge
University Press.
Harris, T. & Moreno Jaen, M. (Eds.). (2011). Corpus Linguistics in Language Teaching. Internationaler
Verlag der Wissenschaften: Peter Lang AG.

340

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■Hedge, T. (2000). Teaching and Learning in the Language Classroom. Oxford: Oxford University
Press.
Johns, T. (1986). Micro-Concord: A Language Learner’s Research Tool. System, 14 (2), 151–162.
Johns, T. (1988). Whence and Whither Classroom Concordancing? In Bongaerts, P. de Haan, P. et al.
(Eds.), Computer Applications in Language Learning (9–27). Dordrecth: Foris.
Johns, T. (1991). Should you be persuaded?: Two examples of data-driven learning materials. English
language research journal, 4, 1–16.
Kennedy, C. & Miceli, T. (2010). Corpus-assisted creative writing: Introducing intermediate Italianlearners to a corpus as a reference resource. Language Learning & Technology, 14 (2), 28–44.
Koosha, M. & Jafarpour, A. (2006). Data-Driven Learning and Teaching Collocation of Prespositions:
The Case of Iranian EFL Adult Learners. Asian EFL Journal Quarterly, 8 (2), 192–209.
Larson-Hall, J. (2010). A Guide to Doing Statistics in Second Language Research Using SPSS.
London: Routledge.
Laufer, B. and Eliasson, S. (1993). What Causes Avoidance in L2 Learning: L1-L2 Difference, L1-L2
Similarity, or L2 Complexity? Studies in Second Language Acquisition, 15 (1), 35–48.
Lee, E. L. (2007). Linguistic and Cultural Factors in East Asian Students’ Oral Participation in US
University Classrooms. International Education, 36 (2), 29–47.
Longman Essential Activator. (1997). Harlow, UK: Longman.
McCarthy, M. (1990). Vocabulary. Oxford: Oxford University Press.
McCarthy, M., McCarten, J., & Sandiford, H. (2014). Touchstone Level 4 Student’s Book (Vol. 4).
Cambridge: Cambridge University Press.
Moon, R. (2012). Vocabulary connections: multi-word items in English. In Schmitt, N. and McCarthy,
M. (Eds.), Vocabulary: Description, Acquisition and Pedagogy (40–63). Cambridge: Cambridge
University Press.
Nunan, D. (1991). Language Teaching Methodology. Hertfordshire: Prentice Hall International.
O’Keeffe, A. & McCarthy, M. (eds.) (2010). The Routledge Handbook of Corpus Linguistics. London:
Routledge.
Rapti, N. (2010). A study of classroom concordancing in the Greek context: Data-driven grammar
teaching and adolescent EFL learners (Doctoral dissertation). Available from: http://eprints.
nottingham.ac.uk/11472/1/Thesis_-_Nikoletta_Rapti.pdf
Ryoo, M. L. (2013). A Corpus-based Study of the Use of Phrasal Verbs in Korean EFL Students’Writing. The Journal of Asian TEFL, 10 (2), 63–89.
Schmitt, N. & McCarthy, M. (Eds.). (1997). Vocabulary: Description, Acquisition and Pedagogy.
Cambridge: Cambridge University Press.
Sinclair, J. (1991). Corpus, concordance, collocation. Oxford: Oxford University Press.
Spring, R. (2018). Teaching Phrasal Verbs More Efficiently: Using Corpus Studies and Cognitive
Linguistics to Create a Particle List. Advances in Language and Literary Studies, 9 (5), 121–135.
Stevens, V. (1991). Concordance-Based Vocabulary Exercises: A Viable Alternative to Gap-filling.
In Johns, T. and King, P. (Eds.), English Language Research Journal, 4, 47–61.
Stringer, E. T. (2014). Action Research. Los Angeles: Sage.

341

Thim, S. (2012). Phrasal Verbs: The English Verb-Particle Construction and its History. Berlin:
Druyter.
Whistle, J. (1999). Concordancing with Students using an ‘Off-the-Web’ Corpus. ReCALL, 11 (2).
74–80.
Yilmaz, M. (2017). The Effect of Data-driven Learning on EFL Students’ Acquisition of Lexicogrammatical Patterns in EFL Writing. Eurasian Journal of Applied Linguistics, 3 (2), 75–88.
Yoon, H. and Hirvela, A. (2004). ESL student attitudes toward corpus use in L2 writing. Journal of
Second Language Writing, 13, 257–283.

342

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■6.
Appendices 




 





         

      






  


      
 



    

    


 
         
 
 

               


      









343





 






          
                    
                               









             







344

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■







           

              


            
 



  



 

 


  




  

  
  

  

 



345











Type

Phrasal verb

(matching line)

MeaningA

Call offA

Figure out

To cancel.

A

Rule out

To recover from something, usually aATurn down

To rely on and trust someone orsomething.

negative experience.
To understand something or solve a
problem.
B

Get over

B

Count onTo eliminate something; to make
impossible.
To say no/reject someone/something.







 







   







346

Evaluating the Use of Data-driven Learning to Teach Phrasal Verbs in a Private Language School in Japan ■  





 

  






             













347