<figure><img src="image_1.png" /></figure>

HAL
open science

# ‘Opening up’ Corpus Linguistics

Elen Le Foll

► To cite this version:

Elen Le Foll. ‘Opening up’ Corpus Linguistics. Second Language Teacher Education, 2024, 2 (2), pp.161-186. 10.1558/slte.25371. hal-04827194

HAL Id: hal-04827194

https://hal.science/hal-04827194v1

Submitted on 9 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
This is the author accepted manuscript (AAM) of a contribution to a Special Issue of the *Journal of Second Language Teacher Education* on the integration of corpus linguistics in second language teacher education (Eds. of the SI: Agnieszka Leńko-Szymańska & Fiona Farr): Le Foll, Elen. 2023. 'Opening up' Corpus Linguistics: An Open Education Approach to Developing Corpus Literacy among Pre-Service Language Teachers. *Journal of Second Language Teacher Education* 2(2). 161–186. https://doi.org/10.1558/slte.25371.

Please cite the version of record.

# 'Opening up' Corpus Linguistics: An Open Education Approach to Developing Corpus Literacy among Pre-Service Language Teachers

Elen Le Foll

University of Cologne

elefoll@uni-koeln.de

**Keywords:** Data-Driven Learning (DDL), corpus-based language pedagogy, materials design, Open Educational Resources (OER), OER-enabled pedagogy, teacher education, English Language Teaching (ELT)

## Abstract

Despite a multitude of empirical studies pointing to the benefits of integrating corpora in second language (L2) teaching and learning, corpus use in the pre-tertiary L2 classroom remains the exception rather than the norm. This much-discussed research-practice gap can be attributed to the limited physical, intellectual and social accessibility of both corpus research and corpus resources. To address this issue, I explore the potential of Open Education in a course aimed at imparting corpus literacy to pre-service EFL teachers. I present the design of a course focused on the collaborative creation of a new Open Educational Resource (OER): a guide to creating corpus-informed teaching materials. The present study evaluates the effectiveness of three iterations of this course in enhancing the physical, intellectual, and social accessibility of corpus linguistics for L2 education. Analyses of pre- and post-course surveys, students' OER chapter submissions, and reflection statements show that the semester-long course successfully developed participants' technical and pedagogical corpus literacy. The findings also suggest that the adoption of OER-enabled pedagogy in an initial teacher training course can make a positive contribution to bridging the corpus research-teaching gap.
# 1 Introduction

At the turn of the millennium, Conrad (2000) published a programmatic article in which corpus linguistics was prophesised to revolutionise grammar teaching in the 21st century. In the second decade of the 21st century, the long-awaited "corpus revolution" is still a long time coming – especially in pre-tertiary educational settings (Boulton & Vyatkina, 2020; Chambers, 2019). Indeed, whilst a plethora of intervention studies has shown that integrating corpora can be effective in a wide range of instructed language learning contexts (Boulton & Cobb, 2017; Crosthwaite, 2020; Lee et al., 2019), the gap between applied corpus research and language teaching practice persists, most notably at pre-tertiary level (e.g., Boulton & Vyatkina, 2020; Chambers, 2019). To bridge this gap, a number of researchers have pointed to the centrality of teacher education (e.g., Conrad, 2000; Leńko-Szymańska, 2017, 2022; Mukherjee, 2004; Römer, 2006, 2009).

In this article, I present the rationale, design and evaluation of a teacher education course designed to bridge this research-practice gap within the framework of Open Education. The course aimed at developing pre-service language teachers' corpus literacy and, in doing so, at breaking down barriers to increase the physical, intellectual and social accessibility of corpus linguistics for L2 teaching and learning by centring the course design on the co-creation of an Open Education Resource (OER) by pre-service teachers.

# 2 Literature review and theoretical framework

In this section, I provide a brief survey of previous studies that have reported on the design and effectiveness of courses aimed at imparting corpus literacy skills to pre- and in-service English as a Foreign Language (EFL) teachers. I then outline the theoretical framework and define the key concepts that underpin the present empirical study.

## 2.1 Previous teacher education courses

Various reports on courses aimed at teaching pre- and/or in-service teachers how to use corpora in the second and foreign language (hereafter: L2) classroom have highlighted an array of challenges and barriers to equipping L2 teachers with the necessary skills and knowledge to make meaningful use of corpora in their own teaching (e.g., Breyer, 2009; Farr, 2008; Farr and Riordan, this SI; Leńko-Szymańska, 2017; Ma et al., 2023; Schaeffer-Lacroix, 2019; Zareva, 2017). The most obvious barriers are technical. These arise at various levels: from physical access to a computer and/or the internet (e.g., Breyer, 2009; Ebrahimi & Faghih, 2017; Farr, 2008; Ma et al., 2023) and basic computer literacy (e.g., Farr, 2008; Ma et al., 2023; Zareva, 2017) to the ability to formulate corpus queries and choose meaningful parameters in specialised corpus tools (e.g., Karlsen & Monsen, 2020; Ma et al., 2023; Schaeffer-Lacroix, 2019). Corpus results may also be overwhelming and their 'messiness' can discourage (pre-service) teachers who may struggle to identify relevant language patterns (e.g., Chen, 2011; Geluso & Yamaguchi, 2014).

Some studies have examined course participants' perceived confidence in conducting corpus queries and interpreting the results. Here, barriers to the effective implementation of corpora in L2 education seem to stem from the limited autonomy of (pre-service) teachers in querying corpora and interpreting corpus results, coupled with insufficient pedagogical skills required for harnessing the full potential of corpora in the L2 classroom (e.g., Leńko-Szymańska, 2014, 2015, 2022; O'Keeffe & Farr, 2003). In some cases, (pre-service) teachers reported being less motivated to make use of corpora in their teaching on completion of such a course (e.g., Ebrahimi & Faghih, 2017; Latif, 2021; Zareva, 2017). Although they often
recognised the value of corpora for their own L2 learning, course participants considered
many of the corpora and corpus tools available to be unsuitable for pre-tertiary educational
contexts (e.g., Karlsen & Monsen, 2020; Ma et al., 2023; Schaeffer-Lacroix, 2019). Moreover,
they frequently reported that using corpora is too complicated and time-consuming to be
feasibly integrated in their teaching practices (e.g., Leńko-Szymańska, 2014, 2015, 2022;
Schaeffer-Lacroix, 2019; Tribble, 2015).

The usually relatively short duration of the courses evaluated in these studies is frequently listed among their most pressing limitations (e.g., Ebrahimi & Faghih, 2017; Leńko-Szymańska, 2015, 2022). Hence, in designing such a course, the first question that arises is: Which skills and knowledge should be prioritised? To clarify these priorities, I focused on the concept of corpus literacy for L2 learning and teaching.

## 2.2 Defining corpus literacy

The ability to use corpora for L2 teaching and learning requires a complex combination of technical, corpus-linguistic and pedagogical skills. Mukherjee (2002, pp. 179–180, 2006, p. 14) was the first to subsume these under the term: *corpus literacy*. In the present study, I will refer to the four subcomponents of corpus literacy as summarised by Callies (2016, p. 395):

1) understanding basic concepts in corpus linguistics

2) searching corpora and analysing corpus data by means of corpus software tools

3) interpreting corpus data

4) using corpus output to generate teaching material and activities.

In the context of the present study, the fourth, pedagogical subcomponent is interpreted as encompassing both indirect (e.g., the creation of corpus-based teaching materials) and direct (e.g., data-driven learning, DDL; see e.g., Boulton & Vyatkina, 2020) applications of corpora to the L2 classroom. It corresponds to what Ma et al. (2022) refer to as ‘corpus-based language pedagogy’.

## 2.3 Accessibility as a barrier to the integration of corpora in the L2 classroom

In an overview of the factors that contribute to the notorious corpus research-practice gap, Chambers (2019, p. 464) highlights the fact that most empirical studies on applications of corpora to L2 learning are conducted by academics "who are prepared to devote a lot of time and effort to an activity which will both produce what they believe are worthwhile learning experiences for their learners and at the same time constitute a valid research activity for them, resulting in a research publication." Most of these publications are behind paywalls and, even when articles or pre-/post-prints of the articles are published open-access, very few include the teaching materials used in the classroom interventions (Chambers, 2019, p. 464; Vyatkina, 2020, p. 363). In addition, they are rarely targeted at a teacher readership (Chambers, 2019, p. 464). Those that have been primarily written with teaching practitioners in mind (e.g., Friginal, 2018; Viana, 2023) often remain physically inaccessible because they, too, are commercial academic publications. Pre-service teachers in the global North may have access to these via institutional library licences whilst they are enrolled as students; however, this only represents a small proportion of the global teaching community.

Moreover, many of the pedagogical interventions reported on academic publications rely on corpus tools that are often physically inaccessible to most schoolteachers. Although there is now a plethora of freely available corpora and corpus tools, teachers are, if at all familiar with corpora, frequently only aware of large corpora accessible via commercial
online platforms that require registration and/or paid-for accounts (see, e.g., Callies, 2019; Karlsen & Monsen, 2020; Latif, 2021). In a small-scale study of in-service EFL teachers who had been exposed to corpus linguistics as part of their studies, teachers stressed the lack of accessibility of corpus resources due to paywalls and registration requirements as a primary reason for avoiding corpora in their own teaching practice (Karlsen & Monsen, 2020).

Borrowing a term from library and information studies, I therefore argue that many of the factors that impede the integration of corpora in language education can be subsumed under the term: *accessibility* (Burnett et al., 2008). The most immediate dimension of accessibility is *physical access*: it involves the cost of a resource, its format, and the technical skills required to access it. Cost has already been mentioned. Format is also highly relevant in the context of corpus linguistics for L2 education. Indeed, Braun (2007, p. 308) and others have underlined the fact that most corpora and corpus interfaces have been designed for linguistic research purposes rather than pedagogical ones. It can also be argued that traditional academic publication formats (books, book chapters and journal articles) are only partially suitable for hands-on introductions to electronic corpora and software tools. Whilst peer review can provide valuable quality assurance, the process often leads to notoriously long publication cycles so that the corpus tools described in the publications may have changed significantly by the time an article or book is published. Even when made available as (usually equally expensive) e-resources, these tend to only ever be in the form of static PDF or EPUB files that are only rarely, if ever, updated to reflect updates in corpora and corpus tools. In the context of this study, physical accessibility encompasses the existence of freely accessible corpora and corpus tools, the hardware to access them, and the basic computer literacy necessary to access them.

Whilst physical access is a prerequisite to developing corpus literacy, it does not suffice. Less obvious but equally important aspects of accessibility are *intellectual* and *social accessibility* (Burnett et al., 2008). Intellectual accessibility refers to the technical and pedagogical skills necessary to make meaningful use of corpora in L2 education, which are essential aspects of corpus literacy as defined in 2.2. The social dimension of accessibility is modelled on the theory of normative behaviour, which posits that information is not universally valued in the same way by everyone but that, instead, its significance is shaped by the norms and attitudes of the particular social community they belong to (Burnett et al., 2008, p. 58). In other words, the intellectual and social dimensions of accessibility refer to users' ability on the one hand, and willingness, on the other, to access the information contained in a resource.

Applied to the concept of developing corpus literacy, teachers' ability and willingness to make use of corpus resources for L2 learning and teaching depend on a wide range of diverse factors including their language skills, level of general education, (computer/data) literacy, cognitive ability, information-seeking behaviours, emotional involvement and subjective views. These are all factors that can readily be mapped onto those that have been identified in the literature as frequently impeding (pre-service) teachers' uptake of corpus linguistics in their teaching practice (see 2.1). With this in mind, an Open Education approach to teaching pre-service teachers corpus literacy was chosen in the present study with the explicit aim of increasing the physical, intellectual, and social accessibility of corpus linguistics for L2 learning and teaching.
## 2.4 Open Education and OER-Enabled Pedagogy

Over the past two decades, Open Education has emerged as a global movement. However, it has, so far, only been "marginally embraced" (Blyth & Thoms, 2021, p. 7) by the L2 education community. Initially, the movement focused on the development, sharing and use of Open Educational Resources (OERs). In stark contrast with traditional (academic) publications, OERs are pedagogical materials made freely accessible to educators and learners, usually under Creative Commons licences. This enables users to engage in the "5R activities" to retain, reuse, revise, remix, and redistribute pedagogical content while circumventing traditional copyright restrictions (Wiley, 2013). Examples of OERs include textbooks, instructional videos, infographics, podcasts, lesson plans and interactive online courses. OERs are usually distributed as online resources which means that they can be rendered more physically accessible, e.g., by providing options to enlarge text and images and ensuring that the latter include written descriptions ('alt-texts') for visually impaired users. Furthermore, interactive and multimedia publication formats (e.g., with embedded videos and quizzes in the form of HTML e-books.) can contribute to the improved intellectual and social accessibility of the contents.

More recently, the Open Education movement has expanded to encompass ‘Open Pedagogy’, defined as an “access-oriented commitment to learner-driven education AND as a process of designing architectures and using tools for learning that enable students to shape the public knowledge commons of which they are a part” (DeRosa & Jhangiani, 2017, p. 14, emphasis in the original). As part of this framework, ‘OER-Enabled Pedagogy’ is defined as “the set of teaching and learning practices that are only possible or practical in the context of the 5R permissions that are characteristic of OER” (Wiley & Hilton, 2018, p. 134). Central to OER-enabled pedagogy is the concept of ‘renewable assignments’ (Wiley, 2013). Whilst many higher education courses require learners to produce so-called ‘disposable assignments’ (Wiley, 2013) that primarily serve to assess whether learners have met the course objectives and are discarded once they have been marked because they have no value beyond assessment, ‘renewable assignments’ not only support the learning of the individuals that produce them, but also result in new or improved OERs that “may provide a lasting benefit to the broader community of learners” (Wiley & Hilton, 2018, p. 137). Examples of OER-enabled pedagogy projects include the creation and revision of anthologies, textbooks and free online courses, and the creation and editing of Wikipedia articles, blogs and instructional videos.

## 3 An OER-enabled ELT course in initial teacher education

Having identified accessibility as a major barrier to overcoming the corpus research-practice gap and inspired by previous efforts to teach corpus literacy in initial L2 teacher education (see 2.1), I set out to design and evaluate an OER-enabled L2 teacher education course with accessibility as its core tenet. The aim of the course was to equip pre-service EFL teachers with the necessary corpus literacy to allow them to create their own corpus-informed<fnref n="1" /> teaching materials and teach others how they achieved this. In line with OER-enabled pedagogy, the course's final assessment was conceptualised as a renewable assignment (see 2.4) in the form of a chapter contribution to an open-access textbook. In the following, I present the research questions that guide the empirical research presented

<footnote n="1">¹ In this article, the term ‘corpus-informed’ is used as an umbrella term that covers both corpus-based and corpus-/data-driven learning (DDL) activities and teaching materials.</footnote>
in this article (3.1). This is followed by a brief introduction to the institutional context in
which the course was embedded (3.2), the design of the OER-enabled course (3.3), and the
resulting OER (3.4).

**3.1 Research aims of the present study**

The semester-long initial teacher education course outlined in 3.3 was designed to increase the physical, intellectual, and social accessibility of corpus linguistics for pre-service EFL teachers. The primary aim of the present study is to determine to what extent this three-fold aim was successfully met. In doing so, the specific institutional constraints under which the three iterations of the course were taught (see 3.2) are taken into consideration. A secondary research aim consists in identifying the strengths and limitations of taking an OER-enabled pedagogy approach to developing pre-service ELF teachers' corpus literacy. In seeking answers to these research questions, I examine a range of data encompassing perspectives from both the course participants and the instructor.

**3.2 Institutional context and participants**

The course was taught by the present researcher at a higher education institution in
Germany as part of pre-service teacher education for future primary, secondary, and
vocational school English teachers. Pre-service teacher education in Germany generally
comprises a Bachelor's degree and Master of Education (M.Ed.). For both degrees, students
are required to take courses in two school subjects and follow a core 'general pedagogy'
curriculum. At B.A.-level, students attend compulsory introduction courses to English
linguistics, English Language Teaching (hereafter: ELT) and general teaching methodology. To
be admitted to the Master's programme, students must have an excellent command of the
English language (equivalent to CEFR C1 level). Over the course of their studies, they are also
expected to complete school internships and to spend at least one semester in an English-
speaking country.

As part of their M.Ed. at this institution, future EFL teachers must complete a module entitled: "English Language Teaching and Learning: Theory and Practice". Depending on whether students are qualifying to teach in primary or lower secondary schools, on the one hand, or upper secondary schools or vocational schools, on the other, this module is either worth 7 or 10 ECTS, split over two seminars worth 3, 4 or 5 ECTS. In practice, all seminars are open to all students and can be completed with an assignment worth 3, 4 or 5 ETCS. Each seminar comprises a taught period of 12 to 14 weeks with weekly 90-minute sessions after which students generally have two to three months to complete a final seminar assignment.

The project-based course evaluated in this study was offered as one such seminar over three consecutive semesters to three different student cohorts. The first iteration of the course was attended and completed by 22 M.Ed. students, the second by 31 and the third by 38, all of whom were aiming to become EFL teachers.

**3.3 Course design**

Following a Flipped Classroom approach, the necessary corpus-linguistic and pedagogical competences to complete the final assessment were taught partly in synchronous course sessions and partly asynchronously in an online learning portfolio (see Figure 1). All course materials were compiled into the portfolio built in the university's in-house Learning Management System (LMS) (see Online Supplement A). During the first iteration of the course, the weekly synchronous sessions were held on-site. As a result of the COVID-19 pandemic, the second and third iterations were taught exclusively online with bi-weekly
synchronous sessions held using the videoconferencing tool integrated in the LMS. Hence, all three iterations of the course were characterised by a mixture of asynchronous and synchronous components (see also Le Foll, 2020, 2021).

The overarching project goal – the co-creation of an OER that would enable (future) teachers to create innovative teaching materials based on freely available corpus data using browser-based tools – and its rationale (see 2.3–2.4) was communicated in the first synchronous session and was thus at the heart of the course. The final course assignment took the form of a chapter contribution to this (at first, hypothetical) new OER (see Online Supplement C for the chapter outline template). Although some students chose to write their chapters individually, participants were strongly encouraged to co-write their chapters in groups of 2–4.

In addition to this graded assignment, students were also required to submit an individual reflection statement (see Online Supplement E). The first two questions of this statement covered the nature and perceived success of the collaboration within their groups (students who chose to write their OER chapters alone did not have to answer these two questions). The third question asked students to describe and reflect on the problems that they encountered when developing their chapters, whilst the fourth required them to reflect on their personal learning journey.

<figure><img src="image_3.png" /><figcaption>Figure 1: Visualisation of the course structure</figcaption></figure>

As shown in Figure 1, the course was divided into three phases. First, students acquired foundational knowledge of both corpus linguistics and ELT materials design. To complete each section of the learning portfolio, students were required to answer questions that tested their understanding and/or encouraged them to (critically) reflect on what they had learned through the set readings and videos. Some sections featured short transfer tasks with which they could practice their newly acquired corpus literacy. Some of the tasks were to be completed collaboratively, others individually. None of these questions or tasks were graded.

The afore-mentioned foundational phase was followed by a work-in-progress phase during which participants developed and refined their corpus-based lesson ideas to eventually be able to submit a (co-authored) OER chapter. This phase included two rounds of peer review in which students tested each other's "chapter prototypes" and gave each
other feedback (see Online Supplement D for peer review template). As the instructor, I also
provided feedback and individual support during this work-in-progress phase.

Students were not asked whether they wanted to publish their OER chapter
contributions until after they had received both their grade and my feedback on the extent
of the revisions necessary for publication. It was made clear from the very beginning that
publication in the OER was optional and that only original, high-quality chapters would
eventually be published. Publication required the signed informed consent of all group
members.

3.4 The Open Educational Resource (OER)

Over the three iterations of the course, students submitted 39 chapters, either as single authors or in co-authoring groups. Initially, 29 chapters were selected for potential publication post-revision. The selection criteria revolved around the quality of the submissions and the feasibility and originality of the proposed corpus-informed activities. Four qualitatively acceptable chapters were not selected because their contents overlapped too much with chapters from a previous iteration of the course. Five chapters originally selected for publication were not published because at least one group member did not consent to publication or could no longer be contacted via their university e-mail address and a further ten because one or more group member(s) lacked the time and/or motivation to make the necessary revisions, or the students did not submit their revisions in time for publication. Thus, 15 revised chapters were ultimately published. They feature a broad range of both paper- and computer-based corpus-informed activities with full lesson plans, downloadable PDF worksheets and suggestions for learners of different proficiency levels. The web-version of the full OER can be found on: https://pressbooks.pub/elenlefoll/.

4 Methodology

To address the risk of bias inherent in an empirical study where the researcher also serves as
the course instructor, I adopted a triangulation method. This approach involves analysing
and cross-referencing data from various sources (see 4.1) using different analysis methods
(see 4.2). Participants were informed of my intention to carry out research to evaluate the
course. Students were informed that participation in this study was voluntary and would not
impact their grades in any way.

4.1 Data

Students were invited to take part in two anonymous surveys (see Online Supplement F for questionnaire items): the first before the course began and the second after submitting their final course assignment. At each data collection stage, participants were required to consent to their data being collected and analysed for research purposes. Across the three iterations of the course, 53 students (58% of course participants) completed both surveys and entered a valid anonymisation code at the start of the second survey that allowed for the matching of their pre- and post-course responses. Of these, 22 were studying to teach EFL in primary schools, five in lower secondary schools (*Haupt-* and *Realschulen*), 19 in upper secondary schools (*Gymnasien*) and seven in vocational schools. With just four males and one non-binary participant, there was a strong female bias in the group. The median age was 24 years (MAD = 1.48). In addition, 87 students (95%) gave consent for their submitted chapters and reflection statements to be analysed for research purposes after having obtained their grades.
## 4.2 Methods

The surveys were administered on https://www.soscisurvey.de/. The data were processed and analysed in R. The use of anonymised personal codes facilitated a participant-level analysis of pre- vs. post-course development, considering additional metadata information for each respondent. This approach enabled a finer-grained analysis to test specific hypotheses, such as whether the course contributed to a greater boost in the intellectual and social accessibility of corpora in L2 education for participants with prior as opposed to no corpus experience, or younger vs. older participants. Given the small scope of the study, only descriptive statistics are reported.

Students' reflection statements were analysed using quantitative and qualitative corpus-linguistic methods. To this end, a corpus of anonymised versions of the statements was tokenised, lemmatised and POS-tagged using Sketch Engine's default parameters for English. Key lemmas were then extracted using the EnTenTen19 as the reference corpus. Key lemmas of particular interest to the research questions (e.g., PROBLEM, DIFFICULT, UNDERSTAND) were then further explored using collocation analysis and by examining the corresponding concordance lines.

The chapter submissions were analysed qualitatively. Each section was manually coded to reflect the extent to which it illustrated the authors' mastery of the relevant subcomponent(s) of corpus literacy (as defined in 2.2). Successful and problematic strategies were identified and annotated. In describing some of the successful strategies, references are made to published chapters of the OER (Le Foll, 2021) which the reader is invited to consult online (https://pressbooks.pub/elenlefoll). Due to space constraints, only one problematic chapter submission is discussed in detail in 5.3.

# 5 Results and discussion

In the following, I investigate the extent to which course participants successfully developed corpus literacy along the three dimensions of accessibility outlined in 2.3. Given that they frequently converge, the results of the analyses of the surveys, the reflection statements and the submitted chapters are presented thematically and discussed together.

## 5.1 Physical accessibility of corpus linguistics for L2 teaching

Before beginning the course, students had very limited means of accessing corpus linguistics primarily due to a lack of basic corpus-linguistic knowledge. As shown in Figure 2, few students reported having any previous knowledge or experience of corpora.
<figure><img src="image_4.png" /><figcaption>**Figure 2:** Pre-course responses to questionnaire items related to corpus-linguistic knowledge and skills (N = 53)</figcaption></figure>

By the end of the course, however, most participants described themselves as knowledgeable and competent users of corpora (see Figure 3). These results point to a considerable increase in the foundational knowledge and skills necessary to make use of corpora for L2 teaching within one semester. Given that the example activities provided during the course focused mostly on developing L2 learners’ lexical and phraseological competences, the fact that 81% of participants claimed to be able to “use corpora to search information on grammar” suggests an impressive transfer of skills and knowledge in a relatively short period.

Whilst these survey responses only reflect self-assessed competences, they were broadly confirmed in my assessment of the students’ submitted OER chapters. Across the three iterations of the course, students demonstrated a sound understanding of basic corpus-linguistic concepts and proficiency in the use of online corpus tools. Most students explained the process of choosing a corpus and querying it very thoroughly in the teacher instruction sections of their submitted assignments (for examples, see the published chapters of the OER). The few technical issues that students reported can be summarised as software updates leading to queries no longer working, lacking or unclear corpus tool documentation, and divergent query languages and terminology across different tools and resources.
<figure><img src="image_5.png" /><figcaption>Figure 3: Post-course responses to questionnaire items related to corpus-linguistic knowledge and skills (N = 53)</figcaption></figure>

## 5.2 Intellectual accessibility of corpus linguistics for L2 teaching

While some technical issues were reported, the analysis of both the submitted chapters and students' reflections clearly showed that pedagogical issues dominated (see Leńko-Szymańska, 2014, 2015 for similar results). To begin, and echoing Zareva's (2017, p. 76) findings, many students expressed difficulties in finding suitable ELT topics or ideas for their OER chapters:

(1) The first problem we came up with was to decide for one topic that is not too broad but also not too specific for the lesson.<fnref n="2" />

This issue may be partly attributed to the fact that some course participants adopted a 'method-before-content' approach to designing their corpus-informed lesson. In other words, these students took as their starting point a corpus and/or tool function and attempted to formulate learning objectives that could be achieved using this corpus and/or corpus tool. By the end of the course, many had successfully reflected on the limitations of this approach:

(2) If I had to start over, I would not think of a fancy way to implement corpus analysis tools right away without truly engaging with the topic of my planned lesson and giving special consideration towards it learning goals. While a corpus analysis can prove itself as a valuable tool to approach secondary language learning, I would not construct a lesson for students around it again. It is of great importance to focus on your learning objectives before choosing a medium you want your students to engage with.

Although for some groups of students several rounds of peer and lecturer feedback were necessary to reach this stage, most students successfully integrated the results of corpus queries and/or hands-on corpus activities in the ELT materials they designed as part of their final assignment. Across all three course iterations, only seven chapter submissions (all co-authored in groups of 2–3 by a total of 17 students) did not reach this stage and were

<footnote n="2">² All numbered quotations are extracts from student reflections. Apart from the editing necessary to anonymise the statements, they are cited as they were submitted and have not been edited in any other way.</footnote>
therefore not considered for publication for this reason. Most of the chapters submitted, however, testify to the authors' solid grasp of the potential of corpora and their ability to successfully design and integrate corpus-informed activities into a realistic EFL lesson plan (see published OER). This is corroborated by the analysis of the reflection statements in which many students reported a strong interest in harnessing the potential of learner-corpus interactions to boost their future students' autonomy and critical thinking skills:

(3) [This course] made me realize that using a corpus is way more efficient than solely using dictionaries and that students' themselves could and should be taught how to work with corpora themselves. Therefore, I am planning on implementing corpus-based lessons in school, staring early with the most basic features, and progressively building upon that knowledge to enable students to autonomously work with corpora [...].

## 5.3 Social accessibility of corpus linguistics in L2 education

This section begins with an analysis of participants' attitudes to the use of corpora in the EFL classroom before turning to social norms and expectations in relation to the learning outcomes of the course.

### 5.3.1 Attitudes

Figure 4 summarises students' attitudes towards corpus linguistics in ELT on completion of the course. It shows that over 70% of respondents agreed or strongly agreed that (a) "it is important for future EFL teachers to learn how to use corpora" and (b) "corpora are very useful to create ELT materials". Furthermore, 60% claimed to be "very likely to use corpora in the future". Whilst just under half of respondents agreed that creating corpus-based activities is too time-consuming for schoolteachers, 60% disagreed or strongly disagreed that using corpora in the EFL classroom is "very difficult". Only a small minority of respondents admitted that they "will probably never use corpora again" or agreed that "corpus linguistics is not at all useful for future language teachers".

<figure><img src="image_6.png" /><figcaption>Figure 4: Post-course survey responses to attitudinal items (N = 53)</figcaption></figure>
To examine individual participants' shifts in attitudes before and after the course, flow diagrams were analysed, incorporating colour coding to facilitate cross-referencing with a third metadata variable, such as participants' future school form, as shown in Figure 5. These finer-grained analyses revealed no systematic differences in attitudes towards the use of corpora in L2 education as a function of respondents' previous experience of corpus linguistics, their second school subject, their age, gender, or whether they took part in the first (hybrid), or the second or third (online) iteration of the course. However, as shown in Figure 5, across all course iterations, pre-service primary school teachers were found to generally be more critical of the use of corpora in L2 teaching than aspiring upper secondary and, to a lesser extent, vocational school teachers.

<figure><img src="image_7.png" /><figcaption>Figure 5: Individual students' change in response to two attitudinal survey items on the use of corpora in the L2 classroom (N = 53)</figcaption></figure>

This trend was also observed in the analysis of students' reflection statements and their submitted chapters. By the end of the course, some of the pre-service primary school teachers were not convinced, or only partially convinced, of the value of corpora for their own teaching practice. However, some valued their newly acquired corpus literacy for their own language learning, whilst others saw benefits in teacher-corpus interactions, e.g.:

(4) [...] I am not sure whether I would take the course again because I feel like the practice does not specifically work for my future teaching plans with primary school students. However, it was very helpful for my personal approach to the English language and I have used one or two different corpora a few times to find out about how to say certain things like a native English speaker and will do so again for future topics and questions.

(5) While I [a primary school teacher trainee] enrolled for this class in the belief that data-driven learning is more viable for advanced learners, I now believe there are some legit options for corpus interaction in primary school learning contexts. Particularly a corpus-based teacher preparation that allows students an indirect access to the corpus in class. By doing so, I will be able to make my language classroom more
diverse and more realistic along with materials designed for a rich and meaningful exposure, catered for the
individual.

According to the post-course survey results, 81% of aspiring primary, 80% lower secondary, 89% upper secondary and 72% of vocational EFL teachers claimed to disagree or strongly disagree with the statement “After this course I will probably never use corpora again”. Whilst these results are very encouraging, some caveats are in order. As always with such survey-based results, not all course participants took part in the surveys and there is thus a risk of self-selection bias. Second, it is well known that good intentions do not necessarily reflect actual practice (cf. Latif, 2021).

5.3.2 Norms and expectations

Despite students expressing much excitement about searching for language patterns from
“authentic” and/or L1 language data, some admitted to having difficulties dealing with
corpus results that did not match their (normative) expectations:

(6) Then we found words we could use for our chapter [using a corpus] even if we had to adapt some of the [corpus] data, we found in order to make the words fit with our concept ideas. We supposed that every English teacher has, based on their own experiences, the competence to decide whether vocabulary is useful or not and especially if it is used in the context of speaking.

This attitude corresponds to a key aspect of social accessibility whereby “if [information] is
at odds with the community’s norms and worldview or if it comes from a source who is not
trusted—[it may] be dismissed as worthless, inaccurate, or even dangerous and thus ignored
or resisted” (Burnett et al., 2008, p. 59). Although only a minority of students described the
process in terms as explicit as in extract (6), this phenomenon was observed in several
submitted chapters (cf. Chambers, 2019, p. 471, who also reports on this issue). It is
illustrated in the following case-study analysis of a final assignment in which a pair of
students competently described how to find adjectives collocates of the noun *weather* in
spoken American English (see Figure 6 from the students’ initial chapter submission). The
top results of this query were illustrated with a screenshot (Figure 7) and the students’
detailed step-by-step instructions of the process demonstrated a sound understanding of
the technical aspects of the tool’s ‘Collocates’ function.

<figure><img src="image_8.png" /><figcaption>Figure 6: Screenshot of collocate search on english-corpora.org from students' initial submission</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>Figure 7: Screenshot of the results of the collocate search from students' initial submission</figcaption></figure>

However, as shown in Figure 8, these results were subsequently dismissed in the design of the ELT materials supposedly based on this data. Indeed, according to results that the authors of the chapter obtained from their corpus query, neither *sunny*, *rainy* or *cloudy* are particularly frequent adjective collocates of *weather* in spoken English (see Figure 7).

<figure><img src="image_10.png" /><figcaption>Figure 8: Extract of the teaching materials from students' initial submission
(cliparts from publicdomainvectors.org)</figcaption></figure>

Firmly entrenched pedagogical norms and worldviews seemingly posed a fundamental
dilemma for some course participants who reported grappling with tension between their
aspiration to teach corpus-based “authentic English” (which, particularly at lower proficiency
levels, they often equated with conversational English) and their perceived duty to teach
vocabulary and grammar as presented in school textbooks. At the same time, many students
stated that, for them, an important learning outcome of the course had been gaining
insights into the differences between the language of textbooks and English use outside the
L2 classroom, suggesting that the course had helped them gain social access to corpora for
L2 teaching:

(7) Apart from now having a more elaborate idea of what corpus linguistics actually is and how it can contribute to creating innovative teaching methods, I have become more critical when looking at traditional text books which are often very “rule-based” rather than pointing to the diverse ways in which language is used in real life contexts.

(8) In my future teaching career, [...] I want to increase the students' awareness of authentic sentences, words or phrases by offering alternative categories for evaluating their language use, such as looking at the frequency or appropriateness of certain structures in a specific context instead of simply classifying them as "wrong".
Looking beyond the linguistic content of textbooks, some students saw the development of corpus-informed materials as an opportunity to address complex, topical and/or controversial issues that are rarely featured in commercial ELT materials. This is well illustrated in three of the revised student-authored chapters published in the OER on Black Lives Matter (Chapter 10), skin issues (Chapter 14) and organ donations (Chapter 15).

## 5.4 Strengths and limitations of OER-enabled pedagogy for the integration of corpus linguistics in L2 education

Contrary to the conclusions of previous studies (notably Breyer, 2009; Karlsen & Monsen, 2020; Leńko-Szymańska, 2017), the results of the present study suggest that a semester-long course in initial teacher education may be sufficient to equip pre-service teachers with the basic technical, corpus-linguistic and pedagogical skills required to make meaningful use of corpora in their teaching. In the present study, this was achieved through OER-enabled pedagogy in a project-based seminar that aimed at the co-creation of a new OER on the design of corpus-informed ELT materials. Although ultimately only 15 chapters by 22 students were published post-revisions, a total of 74 students (81% of participants across the three iterations of the course) produced 32 chapters which were considered for publication post-revisions (see 3.4).

Whilst it was not a specific question in the reflection statement worksheet (see Online Supplement E) and the term was therefore mentioned during the course, many course participants nonetheless made positive mention of the 'renewable' nature of the final course assignment (see 2.4). For many, producing something directly relevant to their future teaching practice appeared to have had a very motivating effect:

(9) Generally, I missed the connection to my future teaching job in a lot of my other [university] courses, so I really liked that this course's final task was designing something that could actually be used in school one day.

These statements suggest that the centrality of the OER in the design of the course helped to tightly focus its contents on (future) teachers' needs, which Römer (2009) identified as crucial to bridging the corpus research-practice gap. It seemingly motivated students to produce high-quality outputs both for themselves and the wider teaching community. Having a common project aim also encouraged students to actively support each other in their learning. This happened both in the formally defined peer-reviewing sessions of the course, as well as informally. Hence, applying OER-enabled pedagogy appears to have contributed to creating a learning community that bolstered students' acquisition of corpus literacy.

For all its benefits, implementing an OER-enabled course in initial teacher education is not without its limitations, however. For a start, a few students reported that it was too time-consuming for the number of credit points awarded (see 3.2). In 5.3, I also highlighted the fact that some participants were not convinced that corpus linguistics is relevant to teaching L2 English to primary school pupils and lower-ability secondary students. That said, others contributed excellent chapters for these levels (see Chapters 1–3 in the OER).

It is also important to acknowledge that OER-enabled pedagogy is not necessarily applicable to all institutional contexts. Considerable time and energy was invested in the process of revising the chapters prior to publication. This happened during the semester breaks in both my and the student authors' free time. Truth be told, the final OER could have featured considerably more student-contributed chapters had I not run out of time and
resources. However, the fact that many students were keen to carry on working on this OER project well beyond the end of the semester testifies to their intrinsic motivation and genuine interest in integrating corpora in the L2 classroom and contributing to the greater accessibility of corpus linguistics for L2 learning and teaching.

Not only did students collaborate and benefit from each other's feedback as co-authors and peer reviewers, but the 'renewable' nature of the assignment also means that, for the 27 students who chose to revise and publish their chapters, their work now contributes to making corpora and corpus-informed materials physically and intellectually accessible to pre- and in-service teachers with no previous experience of corpus linguistics. Hence, following a semester-long course, corpus novices became proficient users of corpora and multipliers of corpus use in ELT with international reach.

# 6 Conclusion and outlook

While acknowledging the intrinsic limitations of pre- and post-intervention surveys and the potential biases associated with a researcher-lecturer conducting action research (see Section 4), the triangulated results of this study provide converging evidence for the relative success of the project. Together, the results suggest that the majority of students successfully acquired all four subcomponents of corpus literacy (see 2.2) as a result of the intervention. The course's focus on the collaborative development of an OER that aims to teach corpus literacy to other (trainee) teachers appeared to have been a valuable and effective source of motivation.

A follow-up study examining participants' use of corpus resources in their teaching practice several years down the line would allow for the assessment of the long-term sustainability of the course's outcomes. Given that corpus linguistics is not (yet) integrated in all language teacher education programmes (see, e.g., Callies, 2019; Schaeffer-Lacroix, 2019), it would also be worthwhile to explore the extent to which an OER co-created by and for (trainee) teachers can support the autonomous acquisition of corpus literacy among pre- and in-service L2 teachers.

The platform on which the web-based version of the OER is hosted does not keep track of how many people access the resource, which makes it difficult to evaluate its long-term impact. In personal correspondence, however, numerous colleagues from higher education institutions (including many from the Global South) have informed me that they are finding the OER very useful and have started using it in their own teacher education courses. It is therefore hoped that the OER will continue to be reused, revised, remixed, and redistributed (see 2.4). Potential follow-up projects include its integration in in-service teacher trainings and the adaptation and/or translation of individual chapters to different corpora, tools and/or L2 languages.

In sum, the results of this study suggest that pre-service teachers are ready and eager to take part in OER-enabled pedagogies as part of their teacher education and that this approach can support the development of corpus literacy skills over a relatively short period of time. For instructors, the practical hurdles to adopting open educational practices are low. Hence, if we genuinely want to bridge the corpus research-practice gap, it may be time for those of us involved in applied corpus research and teacher education to explore ways to 'open up' our research and teaching to ensure the physical, intellectual, and social accessibility of corpus linguistics for L2 education.
# Appendices

The appendices are available as online supplements on: https://osf.io/5ma3b/.

# References

Blyth, C. S., & Thoms, J. J. (Eds.). (2021). *Open Education and Second Language Learning and Teaching: The Rise of a New Knowledge Ecology*. Multilingual Matters.
https://doi.org/10.21832/9781800411005

Boulton, A., & Cobb, T. (2017). Corpus use in language learning: A meta-analysis. *Language Learning*, 67(2), 348–393. https://doi.org/10.1111/lang.12224

Boulton, A., & Vyatkina, N. (2020). Thirty years of data-driven learning: Taking stock and charting new directions over time. *Language Learning*, 24.

Braun, S. (2007). Integrating corpus work into secondary education: From data-driven learning to needs-driven corpora. *ReCALL, 19*(3), 307–328.
https://doi.org/10.1017/S0958344007000535

Breyer, Y. (2009). Learning and teaching with corpora: Reflections by student teachers. *Computer Assisted Language Learning*, 22(2), 153–172.
https://doi.org/10.1080/09588220902778328

Burnett, G., Jaeger, P. T., & Thompson, K. M. (2008). Normative behavior and information: The social aspects of information access. *Library & Information Science Research, 30*(1), 56–66. https://doi.org/10.1016/j.lisr.2007.07.003

Callies, M. (2016). Towards corpus literacy in foreign language teacher education: Using corpora to examine the variability of reporting verbs in English. In R. Kreyer, S. Schaub, & B. Güldenring (Eds.), *Angewandte Linguistik in Schule und Hochschule* (pp. 391–415). Peter Lang. https://doi.org/10.3726/978-3-653-05953-3

Callies, M. (2019). Integrating corpus literacy into language teacher education: The case of learner corpora. In S. Götz & J. Mukherjee (Eds.), *Learner Corpora and Language Teaching* (pp. 245–263). John Benjamins. https://doi.org/10.1075/slcs.201.12cal

Chambers, A. (2019). Towards the corpus revolution? Bridging the research–practice gap. *Language Teaching*, 52(4), 460–475. https://doi.org/10.1017/S0261444819000089

Chen, H.-J. H. (2011). Developing and evaluating a web-based collocation retrieval tool for EFL students and teachers. *Computer Assisted Language Learning, 24*(1), 59–76.
https://doi.org/10.1080/09588221.2010.526945

Conrad, S. (2000). Will Corpus Linguistics Revolutionize Grammar Teaching in the 21st Century? *TESOL Quarterly*, 34(3), 548–560. https://doi.org/10.2307/3587743

Crosthwaite, P. (Ed.). (2020). *Data-driven learning for the next generation: Corpora and DDL for pre-tertiary learners*. Routledge.

DeRosa, R., & Jhangiani, R. S. (2017). Open Pedagogy. In E. Mays (Ed.), *A Guide to Making Open Textbooks with Students*. The Rebus Community for Open Textbook Creation.
https://press.rebus.community/makingopentextbookswithstudents/chapter/open-pedagogy/

Ebrahimi, A., & Faghih, E. (2017). Integrating corpus linguistics into online language teacher education programs. *ReCALL, 29*(1), 120–135.
https://doi.org/10.1017/S0958344016000070

Farr, F. (2008). Evaluating the use of corpus-based instruction in a language teacher education context: Perspectives from the users. *Language Awareness, 17*(1), 25–43.
Friginal, E. (2018). *Corpus linguistics for English teachers: New tools, online resources, and classroom activities*. Routledge. https://doi.org/10.4324/9781315649054

Geluso, J., & Yamaguchi, A. (2014). Discovering formulaic language through data-driven learning: Student attitudes and efficacy. *ReCALL, 26*(2), 225–242.
https://doi.org/10.1017/S0958344014000044

Karlsen, P. H., & Monsen, M. (2020). Corpus Literacy and Applications in Norwegian Upper Secondary Schools: Teacher and Learner Perspectives. *Nordic Journal of English Studies, 19*(1), 118. https://doi.org/10.35360/njes.500

Latif, M. M. M. A. (2021). Corpus literacy instruction in language teacher education: Investigating Arab EFL student teachers' immediate beliefs and long-term practices. *ReCALL, 33*(1), 34–48. https://doi.org/10.1017/S0958344020000129

Le Foll, E. (2021). *Creating Corpus-Informed Materials for the English as a Foreign Language Classroom: A step-by-step guide for (trainee) teachers using online resources* (3rd ed.). https://pressbooks.pub/elenlefoll

Le Foll, E. (2020, July). *Development and Evaluation of a Corpus Linguistics Seminar in Pre-Service Teacher Training*. Teaching and Language Corpora Conference (TaLC) 2020, Perpignan. https://youtu.be/PtgW5y-xFW8

Lee, H., Warschauer, M., & Lee, J. H. (2019). The Effects of Corpus Use on Second Language Vocabulary Learning: A Multilevel Meta-analysis. *Applied Linguistics, 40*(5), 721–753.
https://doi.org/10.1093/applin/amy012

Leńko-Szymańska, A. (2014). Is this enough? A qualitative evaluation of the effectiveness of a teacher-training course on the use of corpora in language education. *ReCALL, 26*(2), 260–278. https://doi.org/10.1017/S095834401400010X

Leńko-Szymańska, A. (2015). A teacher-training course on the use of corpora in language education: Perspectives of the students. In A. Turula, B. Mikolajewska, & D. Stanulewicz (Eds.), *Insights into Technology Enhanced Language Pedagogy* (pp. 129–144). Peter Lang. https://doi.org/10.3726/978-3-653-04995-4

Leńko-Szymańska, A. (2017). Training teachers in data-driven learning: Tackling the challenge. *Language Learning & Technology, 21*(3), 217–241.

Leńko-Szymańska, A. (2022). Training Teachers and Learners to Use Corpora. In R. R. Jablonkai & E. Csomay (Eds.), *The Routledge Handbook of Corpora and English Language Teaching and Learning* (1st ed., pp. 509–524). Routledge.
https://doi.org/10.4324/9781003002901-39

Ma, Q., Chiu, M. M., Lin, S., & Mendoza, N. B. (2023). Teachers' perceived corpus literacy and their intention to integrate corpora into classroom teaching: A survey study. *ReCALL, 35*(1), 19–39. https://doi.org/10.1017/S0958344022000180

Ma, Q., Tang, J., & Lin, S. (2022). The development of corpus-based language pedagogy for TESOL teachers: A two-step training approach facilitated by online collaboration. *Computer Assisted Language Learning, 35*(9), 2731–2760.
https://doi.org/10.1080/09588221.2021.1895225

Mukherjee, J. (2002). *Korpuslinguistik und Englischunterricht: Eine Einführung*. Peter Lang.

Mukherjee, J. (2004). Bridging the Gap between Applied Corpus Linguistics and the Reality of English Language Teaching in Germany. In U. Connor & T. Upton (Eds.), *Applied Corpus Linguistics: A Multidimensional Perspective* (pp. 239–250). Rodopi.

Mukherjee, J. (2006). Corpus linguistics and language pedagogy: The state of the art—and beyond. In S. Braun, K. Kohn, & J. Mukherjee (Eds.), *Corpus technology and language pedagogy: New resources, new tools, new methods* (pp. 5–24). Peter Lang.
O'Keeffe, A., & Farr, F. (2003). Using Language Corpora in Initial Teacher Education: Pedagogic Issues and Practical Applications. *TESOL Quarterly*, 37(3), 389–418.
https://doi.org/10.2307/3588397

Römer, U. (2006). Pedagogical applications of corpora: Some reflections on the current scope and a wish list for future developments. *Zeitschrift Für Anglistik Und Amerikanistik*, 54(2), 121–134.

Römer, U. (2009). Corpus research and practice: What help do teachers need and what can we offer? In K. Aijmer (Ed.), *Corpora and language teaching* (pp. 83–98). John Benjamins.

Schaeffer-Lacroix, E. (2019). Barriers to trainee teachers’ corpus use. In P. Crosthwaite (Ed.), *Data-Driven Learning for the Next Generation* (pp. 47–64). Routledge.

Tribble, C. (2015). Teaching and language corpora: Perspectives from a personal journey. In A. Leńko-Szymańska & A. Boulton (Eds.), *Multiple affordances of language corpora for data-driven learning* (pp. 37–82). John Benjamins.

Viana, V. (Ed.). (2023). *Teaching English with Corpora: A Resource Book*. Routledge.

Vyatkina, N. (2020). Corpora as Open Educational Resources for Language Teaching. *Foreign Language Annals*, 53(2), 359–370. https://doi.org/10.1111/flan.12464

Wiley, D. (2013). What is Open Pedagogy? [Open content]. *Improving Learning*.
https://opencontent.org/blog/archives/2975

Wiley, D., & Hilton, J. L. (2018). Defining OER-Enabled Pedagogy. *The International Review of Research in Open and Distributed Learning*, 19(4).
https://doi.org/10.19173/irrodl.v19i4.3601

Zareva, A. (2017). Incorporating corpus literacy skills into TESOL teacher training. *ELT Journal*, 71(1), 69–79. https://doi.org/10.1093/elt/ccw045