all of her results into the spoken and written subcorpora for the ENL and the learner English corpora, this register differentiation is not made for the textbook corpus, even though language textbooks typically include registers as diverse as conversation, newspaper writing and fiction (see 3.3.1.4). Here, too, it is difficult to draw any meaningful conclusions from the results of this analysis since the frequencies reported for each part-of-speech use also vary greatly across the different reference subcorpora. Nevertheless, it would appear that the frequencies of the learner corpora are considerably closer to those of the textbook corpus than to those observed in the reference ENL corpora. Such observations lend tentative support to the hypothesis that, given they represent a major source of L2 input, textbooks play a crucial role in EFL learners' language acquisition processes (see 1.6–1.7) but also remind us that mode and register differences need to be accounted for when describing and evaluating the language of EFL textbooks.

### 2.1.2.7 Textbook language as learner target language

As in the previous section, this section presents a methodological approach to the study of Textbook English that also compares data from a learner corpus to that of a textbook corpus. However, as illustrated with a study by Tono (2004), this particular approach not only adds a layer of cross-linguistic comparison using an L1 (Japanese) corpus, it also turns the equation around by assuming textbook language to be the learners' target language.

In stark contrast to the corpus-based comparative approaches reviewed so far, Yukio Tono (2004: 51) claims that “textbook English is a useful target corpus to use in the study of learner language”. He convincingly argues that comparing learner language to texts produced by native speaker professionals makes little sense. Indeed, all of the well-known general English corpora used in most of the Textbook English studies reviewed so far (e.g., BNC1994, Brown, FLOB, etc.) predominantly feature professionally written or spoken texts such as newspaper articles, extracts of novels and political speeches. Whilst Tono recognises that the use of such reference corpora may make sense when it comes to supporting advanced L2 learners or professional translators, he considers that the majority of English learners in Japan have no such aspirations:

> In the present case, it is certainly not the language of the BNC that the Japanese learners of English are aiming at, but, rather, a modified English which represents what they are more exposed to in EFL settings in Japan (Tono 2004: 51).

Although he acknowledges that Textbook English often does not reflect actual language use, he nevertheless argues that, since Textbook English is constructed so as to facilitate learning, it makes sense to apply Textbook English as a benchmark when investigating EFL attainment.
In the Japanese context, Tono (2004) emphasises the fact that textbooks represent the primary source of English language input, noting that even when teachers use English as medium of instruction, they tend to restrict themselves to the structures represented in the textbooks. Hence, Tono (2004: 52) claims that "it is fair to say that the English used in ELT textbooks is the target for most learners of English in Japan". Whilst the present author disagrees with the idea that Textbook English necessarily is (see also Timmis 2003) or should be the learners' target (see 2.1.2.7), Tono undoubtedly raises an important point: the need to reflect on the suitability of using general English corpora such as the BNC as benchmark reference corpus when analysing both textbook and learner language (see Winter & Le Foll forthcoming for an example of a study that justifies the use of only specific subcorpora of the BNC in such a comparative analysis).

In a study on the acquisition of English argument structures by Japanese learners, Tono (2004) compares three different types of corpora: a) an interlanguage corpus of free compositions written by Japanese learners of English, b) a native language corpus consisting of English newspaper articles, and c) an EFL textbook corpus. First, sentence frame patterns with the three most frequent verbs (except *have* and *be*) are extracted from all three corpora. For each high-frequency verb use, Tono collects data on a range of variables, e.g., frequency in the textbook corpus, number of learner errors, learners' year group, Japanese equivalents of the verb constructions, etc. Log-linear analysis is used to tease out the most important factors influencing Japanese learners use of these sentence frame patterns. To this end, all frequencies are converted to categorical data (i.e., to high, mid, or low occurrence); thus, presumably reducing the degree of accurateness and adding a layer of arbitrariness in the statistical analysis. The results of the best fitting models show that the learners' school year exerts the most influence on learners' idiomatic production of sentence frame patterns. Interestingly, the second most influential factor is the frequency of a pattern in the textbook corpus. Strong two-way interaction effects between the factors 'school year' and 'textbook frequency' are also observed. By contrast, 'learner error' only significantly interacts with 'school year in one case (for the verb *get*). This suggests that textbook frequencies mostly impact students' overuse or underuse of a particular verb pattern, rather than their rate of success in producing the pattern idiomatically. In addition, the results also show that whether or not a verb argument structure has a comparable equivalent structure in Japanese has less impact on Japanese L1 learners' production of the target structures than how often the structure is featured in the textbooks they learn from.

Although working from radically different premises, both this section and the one preceding it have revealed the value of integrating learner corpora in Textbook English evaluation. At least since the late 1990s, a number of academics have advocated integrating observations gleaned from learner corpora into the design of
new EFL publications (Granger 1998; e.g., Kaszubski 1998). Indeed, some major
textbook publishers have now latched onto the idea; as mentioned in 1.8, Cambridge
University Press now draws on the error-tagged Cambridge Learner Corpus, which
was compiled on the basis of student responses taken from Cambridge English
Language Assessment examinations.

2.1.3 Evaluating the impact of textbook language

Most of the studies of Textbook English outlined thus far have aimed to describe the
linguistic input of EFL textbooks. By contrast, this section examines studies that also
aim to evaluate the potential pedagogical impact of this textbook-based input. Of
course, language teachers regularly reflect on the quality of the textbook materials
they introduce in class and will thus periodically conduct at least impressionistic
retrospective analyses of the textbooks' content. However, attempts to formalise and
quantify such retrospective evaluations on the effectiveness of foreign language
textbooks have tended to focus on the nature of the tasks and activities featured in
the textbooks (see, e.g., Ellis 1997), rather than home in on the quality and usefulness
of their linguistic input. This section presents two studies that investigate the
linguistic content of EFL publications with regards to their impact on learners in
terms of learning outcomes and efficacy. As an extension of comparative corpus-based
approaches described in 2.1.2.3, the methodology of the first study (Alejo González
et al., 2010) will be familiar to the reader. The methodology of the second study
(Gouverneur 2008a), however, relies on the analysis of a corpus of textbook activities
annotated with a complex pedagogical annotation scheme.

Alejo González et al. (2010) delve into both the implicit and explicit mentions of phrasal verbs in textbooks, focusing on the learning efficacy gains for the textbook users. To this end, they select eight popular EFL textbooks targeted at the Spanish secondary school market. Their research on the likelihood of incidental learning of phrasal verbs in the ELT material is based on frequency counts within the textbooks and on frequency comparisons with the BNC1994. They report that the vast majority of the phrasal verbs featured in the examined textbooks only appear once or twice in any one textbook, and thus do not occur nearly frequently enough to warrant incidental learning. Moreover, comparisons of the frequencies of the 25 most frequently occurring phrasal verbs from the BNC1994 with the data from the textbooks show that while two of those phrasal verbs (*go out* and *look after*) are vastly over-represented in the ELT material, many others are largely under-represented (e.g., *go back*, *point out* and *take over*), if not entirely absent (*carry on*).

The explicit part of the investigation examines the metalanguage used to describe
phrasal verbs and related phenomena in the textbooks, as well as the types of exercises
designed to encourage the acquisition of these lexical items. Referring to pedagogical
approaches inspired by cognitive linguistics (see 1.4), Alejo González et al. (2010)
deplore that none of the textbooks examined organise explicit mentions of phrasal verbs in a way that is likely to facilitate acquisition by encouraging learners to understand the ‘motivated’ nature of the particles in combination with their corresponding lexical verbs (for more on the cognitive linguistics’ view that phrasal verb particles display a certain degree of compositionally, see, e.g., Condon 2008; Spring 2018; Torres-Martínez 2019; Tsaroucha 2018). Alejo González et al. (2010: 72) argue that “[i]f materials create too few opportunities for incidental uptake, then this should be compensated by explicit targeting” and conclude that their sample of eight Spanish secondary school EFL textbooks fail to adequately do so.

In conclusion to a large-scale learner corpus study on collocation, Nesselhauf (2005: 238) also postulates that collocations are not taught in a way that spurs on their idiomatic acquisition since there appears to be no correlation between the number of years of classroom teaching and the idiomaticity of the collocations learners produce (see also Le Foll 2016). Although Gouverneur (2008) does not directly compare Textbook English to learner language, she takes Nesselhauf’s (2005) corpus-based learner error analysis as her starting point and designs a study that aims to tease out whether “learners’ deficiencies in the production of phraseological patterns of simple verbs might be teaching-induced or, more precisely, material-induced” (Gouverneur 2008a: 224). To do so, Gouverneur (2008) draws on a textbook corpus (TeMa; Meunier & Gouverneur 2009) which includes the full pedagogical materials from each textbook series including the reading texts, transcripts, vocabulary exercises and instructions from both student’s coursebooks and workbooks. Uniquely, the TeMa corpus also includes detailed pedagogical annotation of the subcorpora containing the vocabulary exercises with some 80 codes referring to various aspects of task design and content (Meunier & Gouverneur 2009). As part of this study, all the instances of *make* and *take* were automatically retrieved from the vocabulary exercise subcorpora and the results were manually sorted for meaning and collocational patterns. High-frequency verbs are found to feature prominently in the context of restricted collocations in all the textbooks, thus suggesting that material designers had taken due care “to include a significant number of phraseological uses [of *make* and *take*] in the exercises” (Gouverneur 2008: 234).

Next, all instances of restricted collocations identified were categorised according to the degree of focus on the collocation in the corresponding exercises. It transpires that direct, explicit focus on these lexical units was largely found in the intermediate level textbooks. Gouverneur (2008: 235) notes that, in more advanced textbooks, these collocations are no longer dealt with explicitly. This trend was found to be true for all three series of textbooks examined. Gouverneur (2008: 235) suggests that:

[t]his lack of direct focus on restricted collocations at the advanced level might well be one of the reasons why more proficient learners have so many problems dealing with high-frequency verbs.
The vocabulary exercises were also annotated according to eight types of pedagogical activities, which were themselves grouped into four larger categories corresponding to the cognitive processes they are (presumably) designed to activate. According to this annotation scheme, whilst 12% of the intermediate learning activities on collocations of *make* and *take* are designed to activate understanding, such activities are entirely absent from the advanced textbooks (Gouverneur 2008a: 236–237). Another striking finding is that fewer than 20% of all the advanced exercises require learners to produce an answer that requires full retrieval from the mental lexicon. Most exercises merely require students to select the correct solution from a given list of words or expressions (Gouverneur 2008a: 236–237).

## 2.2 Part Two: Key findings of Textbook English studies

Part one of this literature review chapter provided an overview of the wide range of methodologies that have so far been applied to survey the linguistic content of EFL textbooks, reporting on the results of individual studies to illustrate the advantages and potential weaknesses of the various methods. Part two, by contrast, homes in on some of the key results of previous studies examining the language of English textbooks. Whenever possible, emphasis is placed on the results of EFL textbooks designed for secondary school contexts but, where studies are sparse and they are deemed to be relevant, the results of adult EFL, EGP, ESL and EAP textbook studies are also mentioned.

The following section falls into three subsections. First, the results of studies principally exploring the lexis of Textbook English will be summarised. Second, studies investigating more complex lexico-grammatical features denoting verb tense, aspect and argument structures are presented before, third, the results of the few Textbook English studies focusing on pragmatics and discourse are reviewed. Note that this division of the examined linguistic phenomena into these broad categories purely serves an organisational purpose. Indeed, and as will be made evident in the discussion of the studies' results, many of the examined linguistic features straddle any artificial boundaries between lexis, grammar, discourse, semantics, and pragmatics.

### 2.2.1 Lexis

Perhaps the most immediately obvious aspect of Textbook English is its vocabulary – in other words, the range of words and multi-word units presented in English textbooks. In the following, the results of a small selection of studies focusing on the lexis of English textbooks are outlined. Subsection 2.2.1.1 focuses on individual words whilst 2.2.1.2 looks at the treatment of multi-words units such as collocations, phrasal verbs and lexical bundles in Textbook English.
2.2.1.1 Individual words

The tradition of examining the vocabulary of EFL textbooks goes back a long way. The results of Ljung's (1990; 1991) analysis of the vocabulary featured in upper secondary school Swedish EFL publications have already been discussed in 2.1.2.1. As a reminder, the studies pointed to an overrepresentation of concrete words to the detriment of abstract ones and deplored the poor representation of lexical units commonly used in communicative interaction and in the establishment of social relationships.

As part of another early corpus-based Textbook English study, Renouf (1984; cited in Sinclair & Renouf 1988) investigated learners' vocabulary input in nine major EFL coursebooks. Her analysis shows that, in the first coursebook of each series, the number of different word forms introduced ranged from just over 1,000 to nearly 4,000 – thus representing an incredibly wide variation. The average rate of re-occurrence of each word form across the different textbook series was also calculated. Here, too, the patterns of reinforcement also ranged widely: from six to 17 times.

Based on an analysis of the same textbook series, Sinclair & Renouf (1988) explored learners’ exposure to delexical constructions (also frequently referred to as support verb constructions). The authors concluded that such constructions are mostly neglected in Textbook English, in spite of their preponderance in ENL corpora. This study, however, disregards occurrences of delexical constructions occurring within “the rubric of the text” as of secondary importance, rather than as an integral part of the teaching programme (Sinclair & Renouf 1988: 153). Thus, whilst Sinclair & Renouf (1988) deplore that ditransitive uses of the verb *give* are not explicitly highlighted in the coursebooks, they acknowledge that such patterns are featured within the coursebooks’ “text rubrics”.

A few decades later, Reda (2003) conducted a large-scale analysis of (adult) EGP textbooks designed for the global EFL market. The study concludes that the vast majority of textbooks across all proficiency levels are largely based on a “limited number of ‘general interest’ topics”, such as *cooking, food and drink* or *holidays and travel* (Reda 2003: 264). Hence, in spite of the rise of English as an international language in the context of globalisation, the lexical syllabus taught in the EFL/EIL textbooks examined confines itself to “the basic area of the English vocabulary – the ‘visitors’ wing’” (Reda 2003: 268). Even the more advanced coursebooks in each series do not depart from these “basic topics” of “general interest”.

Whilst Reda's (2003) analysis of English textbooks targeted at adults appears to point to a common understanding by textbook publishers as to the “topics of general interest” to be covered in EFL textbooks, Catalán & Francisco (2008) conclude that the textbook authors of EFL textbooks used at two levels in Spain (6th grade of
primary education and 4th grade of secondary education) disagree on the core
vocabulary learners ought to acquire at these stages. The authors measure the number
of tokens and types for each textbook and compare type-token ratios. Moreover, they
compute lists of the 50 most frequent content words from each textbook. Comparisons
of these frequency lists show that Spanish learners of English are exposed to very
different words and with varying frequencies depending on which textbook they have
been assigned. Catalán & Francisco (2008: 161–162) point out that the Spanish
authorities do not specify how many or which words students ought to have acquired
by any particular stage and conclude from their study that the textbook authors also
appear to lack a systematic approach to vocabulary selection and presentation.

Whilst the studies reviewed so far have focused on the breadth of vocabulary covered
by English textbooks, the following studies examine three specific functional
categories of words: linking adverbials (Conrad 2004), the definite article (Yoo 2009),
and adjectives (Biber & Reppen 2002a).

Conrad (2004) focuses on the frequencies and usage of linking adverbials of contrast
and concession in two registers (conversation and academic prose), comparing data
from the *Longman Grammar of Spoken and Written Language* (Biber et al. 1999) to
the coverage of the adverbial *though* in four American ESL textbooks. The study
concludes that textbook coverage does not match native corpus evidence. For
instance, Conrad (2004) notes that three out of the four textbooks fail to include the
use of *though* as a linking adverbial, and that the only textbook that mentions it
presents it as a means of showing contrast but neglects its usage as a means to express
concession. Although it occurs frequently in L1 conversation, all four textbooks fail
to mention *though* as a means of softening disagreement between speakers. Conrad
(2004) observes that only one textbook suggests a number of contrast linking
adverbials to use in conversation, but that this textbook misleads learners into
thinking that *however* and *on the other hand* are commonly used in conversation,
whereas they, in fact, occur far more frequently in academic prose than in any other
register. Indeed, a number of Textbook English studies have pointed to the
predominance of lexico-grammatical features typical of written registers in textbook
dialogues designed to emulate spontaneous spoken interaction (see 2.2.4 on spoken
grammar in Textbook English).

In a study following a very similar approach, Yoo (2009) compared the treatment of
definite articles in six EFL/ESL grammars with corpus findings reported in the
*Longman Grammar of Spoken and Written English* (Biber et al 1999). The results
suggest that whilst most ESL/EFL grammars extensively describe the anaphoric and
associative uses of the definite article (e.g., *Let's go to the Indian restaurant. The*
*food is delicious*.), its situational (e.g., *Can I have *the* chutney, please?*) and
cataphoric uses (e.g., *At *the* beginning of my PhD*) are neglected. The findings
potentially have important pedagogical implications since corpus data shows that the situation and cataphoric uses of the definite article are more common than its anaphoric use in a number of text registers that English learners are highly likely to be confronted with: namely, conversation, newspaper language and academic prose (Yoo 2009: 273–276).

In sum, the results of these case studies on the presentation of *though* and *the* in textbook grammars serve as a reminder as to the central importance of production modes, registers, and text types in the (contextual) use of specific lexico-grammatical features. It therefore follows that these factors must be taken into consideration, both in the elaboration and evaluation of Textbook English (see 3.1). The final case study on individual words reviewed as part of this section, Biber & Reppen (2002), illustrates how the proficiency levels that textbooks are targeted at must also be considered when modelling Textbook English.

Among other lexico-grammatical phenomena, Biber & Reppen (2002) focus on the role of adjectives in Textbook English. To this end, they compare the frequencies of different types of nominal premodifieds in a large general English corpus with how they are presented in six popular ESL/EFL grammar textbooks. The results suggest that the pedagogical materials over-emphasise the prevalence of participial adjectives (e.g., *an exciting game, an interested couple*) whilst underestimating the pervasiveness of nominal premodifieds (e.g., *a grammar lesson*) (Biber & Reppen 2002a: 201–202). As far as teaching beginner-level conversation is concerned, a focus on attributive adjectives (e.g., *the big house*) appears to be justified. At higher levels of proficiency, however, the authors argue that students would likely benefit from greater exposure to the use of nouns as nominal premodifieds since corpus-based findings have shown that these are conspicuously frequent in both newspaper and academic writing (Biber & Reppen 2002a: 202). Thus, these results point to the necessity of not only accounting for mode and/or register differences, as highlighted in the discussion of Conrad's (2004) and Yoo's (2009) results, but also textbook proficiency levels when describing and evaluating the language of textbooks. However, the vast majority of Textbook English studies reviewed as part of this chapter do not account for either of these potential sources of variation.

### 2.2.1.2 Multi-word units

Though 2.1.2.1 has shown that the study's methodology is not without its flaws, Zarifi & Munkundan (2012) certainly point to a disconcerting gap between the phrasal verbs featured in the 'spoken' sections of Malaysian ESL textbooks and the most frequent phrasal verbs in the BNC1994. For instance, they report that the most frequently occurring phrasal verbs in their textbook corpus, *clean up* and *melt down*, do not, in fact, belong to the most frequent 100 phrasal verbs in the BNC1994. The results also reveal that other highly frequent and more pedagogically valuable phrasal verbs –
such as *work out*, *turn over* and *go over* – do not appear at all in this corpus of textbook dialogues (Zarifi & Mukundan 2012: 13).

The comparative corpus-based methodology employed in Koprowski's (2005) exploration of multi-word lexical units in Textbook English was laid out in 2.1.2.4. Among the most striking results was the fact that more than 14% of the lexical phrases explicitly featured in the three EGP coursebooks examined were entirely absent from the selected reference corpus, the COBUILD (Sinclair et al. 1990). Such phrases include *cheap steak*, *mild cigarette*, *imprisoned man*, *recommend fully* and *on its last feet* (Mark Koprowski 2005: 328). Based on this small-scale investigation, Koprowski (2005: 329) draws the provocative conclusion that “the more lexical phrases in a course[book], the less useful the items tend to be on average”. It would appear that textbook authors frequently attempt to supply excessively comprehensive sets of lexical phrases of a single type or on a single topic thus resulting in the inclusion of some highly infrequent, sometimes outright implausible, collocations. In addition, and just like Catalán & Francisco’s (2008) study of individual words in school EFL textbooks, Koprowski (2005) points to a striking lack of consensus as to what constitutes a meaningful lexical curriculum at intermediate level since less than 1% of the lexical phrases collected are shared by any of the textbooks under study (Mark Koprowski 2005: 330).

Another strand of Textbook English studies is concerned with phrasemes, i.e., recurrent sequences of words such as *the fact that, I want you to* and *which is why* (also referred to as lexical bundles, lexical clusters and n-grams). Even though they often do not represent a complete structure nor are they necessarily idiomatic in meaning, these multi-word units nevertheless capture important discourse functions in both written and spoken registers (Biber 2006: 134–135; Biber & Barbieri 2007: 264) and are thus very relevant to the description and evaluation of Textbook English.

Siepmann (2014) compared the phrasemes featured in the vocabulary sections of two series of German secondary school EFL textbooks with a revised version of Martinez & Schmitt's (2012) list of the most frequent "non-transparent phrasemes" found in the BNC1994. Across these entire textbook series spanning five years of EFL instruction, only 12% (for *Green Line*) and 16% (for *G21*) of the phrasemes of the revised corpus-based list were mentioned at least once. Siepmann (2014) concludes that the selection of phrasemes in these textbooks is seemingly not based on frequency or, in fact, on any other systematic set of criteria. In addition, and contrary to expectations, it was also not the case that the number of phrasemes featured in these textbooks rose as students' proficiency level increased.

Aside from the aforementioned study, most Textbook English research to have taken a phraseme or lexical bundle perspective have examined English for Academic
Purposes (EAP) and English for Specific Purposes (ESP) textbooks. These materials are designed to equip non-native speakers of English with the necessary skills to cope with the demands of academic reading and writing at English-speaking universities and/or (future) professional activities in English. A number of studies have attempted to describe and/or evaluate the language of such textbooks by examining the types and frequencies of lexical bundles they feature (e.g., Biber et al. 2002; Biber 2006; Chen 2010; Grabowski 2015; Wood 2010; Wood & Appel 2014).

Wood (2010), for instance, investigates the frequency of lexical clusters in six intermediate and advanced EAP textbooks. This corpus-based analysis of the textbook materials reveals that textbook instructions feature considerably more lexical clusters than the reading passages. Wood (2010) advances the theory that publishers aim for a certain amount of consistency in the formulation of the tasks, thus leading to a high frequency of lexical clusters in the instructional texts. The reading passages, on the other hand, contain fewer lexical clusters and their frequencies of occurrence within any one textbook are such that the author believes that it is unlikely that learners can acquire them solely through reading. Wood's (2010) page-by-page analysis of the pedagogical treatment of formulaic language in the examined textbooks strengthens this hypothesis, as no attempt appears to be made to focus learners' attention on these lexical units.

The presentation of lexical bundles in EAP and ESP textbooks is also the focus of Wood & Appel's study (2014) in which they first extracted the most frequent three and four-word bundles in a corpus of ten first-year business and engineering textbooks, and then queried a corpus of five intermediate and advanced EAP textbooks to reveal which of those bundles appear in the EAP textbooks. Depending on the EAP textbook, between 35% to 47% of the most frequently occurring lexical bundles from the subject textbooks were found at least once in the EAP textbooks. However, the authors deplore that none of the formulaic sequences are dealt with pedagogically, i.e., presented as units worth learning or highlighted in any way that might raise learners' awareness of their potential.

Focusing on one discipline only, electrical engineering, Chen (2010) also compares the frequency and nature of multi-word units in entry-level university electrical engineering textbooks and ESP textbooks especially designed for students of this same discipline. In contrast to Wood & Appel (2014), however, she not only compiles a list of the most frequent lexical bundles found in the introductory subject-specific textbooks, but also one for the ESP textbooks. As a result, she is able to compare the types, frequencies and pragmatic functions of lexical bundles featured in both types of textbooks. Her results match those of Wood & Appel (2014) in that only a third of the lexical bundles identified in the electrical engineering introductory textbooks occur at least once in the corresponding ESP textbooks. Furthermore, a qualitative
analysis of the pragmatic functions of the bundles demonstrates that entire subcategories of stance bundles (e.g., *can be used to*, *it is important to*) are missing from the ESP textbooks. When it comes to referential bundles (e.g., *is referred to as*, *a great deal of*), Chen concludes that “the ESP textbooks underrepresent quantity and spatial specifications but overemphasize referential information which is not central in target language use, such as the introduction to new concepts/definitions and provision of time information” (Chen 2010: 123).

## 2.2.2 Tense and aspect

Having presented some of the observations derived from a range of studies on the lexis of Textbook English, this section now turns to the representation of tenses and aspects in EFL publications. To this end, it seems natural to begin with one of the earliest comparative corpus-based studies already mentioned in 2.1.2.3: Mindt’s (1987; 1992) study on the prevalence, functions and lexico-grammatical patterning of future constructions in German EFL textbooks.

### 2.2.2.1 Future constructions

As explained in 2.1.2.3, Mindt (1987; 1992) undertook to study the future constructions presented in textbooks designed for the first cycle of German secondary schools with those actually produced in speech by British native speakers. He concluded that the examined German EFL textbooks under-represent *will*, over-represent *going to* and leave out *shall* as a means of expressing future situations altogether (Mindt 1992: 189). Mindt (1992: 37) also observed that, compared to his reference corpus of (pseudo-)spoken English, the contracted forms of *going to* were considerably under-represented in the spoken passages of the textbooks. Furthermore, he interprets the absence of *gonna* and *ain’t* in the examined textbooks as a misrepresentation of English language usage at the time of the study (Mindt 1992: 35, 41). Mindt’s analyses go beyond simple comparisons of relative frequencies and also explore the context in which certain expressions of the future co-occur. For instance, he notes that *going to* co-occurs with a relatively narrow range of expressions in Textbook English and hypothesises that this is due to an over-generalisation of *going to* as a structure used almost exclusively to express the future (Mindt 1992: 190).

### 2.2.2.2 The present perfect

Following a similar approach, Schlüter (2002) conducted a book-length corpus-driven analysis of the use of the present perfect on the basis of a native-speaker corpus consisting of spoken and written English. From this data, Schlüter (2002) established a so-called ‘empirical grammar’ (see also Dirven 1990; Mindt 1995a) of the present perfect and contrasted it to existing traditional grammars that are known to mostly rely on introspection, as well as to the grammar sections of two popular series of
secondary school EFL textbooks used in Germany, together with their accompanying grammar and activity books. The textbooks are found to present the functions of the present perfect in substantially different ways (Schlüter 2002: 219–328). For example, textbooks fail to explain that the present perfect progressive is often used to refer to iterative actions or events rather than continuous ones.

In order to investigate Japanese EFL learners’ difficulties with the use of the present perfect, Fujimoto (2017) triangulated results from three corpora: a reference corpus of general American and British English, a Japanese L1 English L2 learner corpus and a corpus of EFL textbooks designed for Japanese high schools. Fujimoto (2017) reports that learners overuse the present perfect with temporal adverbials as compared to the reference corpus. The six textbooks examined vary greatly in their use of the simple present perfect both with and without temporal adverbials. However, the two textbooks that radically over-represent the simple present perfect with temporal adverbials, as opposed to without, do so principally in the exercises, rather than in the extended reading passages. Fujimoto (2017) suggests that this may explain why, in their own writing, Japanese learners of English are more comfortable using the present perfect with temporal adverbials than without. Their over-representation in EFL textbook exercises may mean that Japanese learners of English have internalised their use as lexical markers that trigger the use of the present perfect.

### 2.2.2.3 The progressive

In an in-depth, corpus-driven analysis of the progressive aspect comparable to Schlüter’s (2002) analysis of the present perfect (see 2.2.2.2), Römer (2005) examined how the progressive is represented in the dialogues of two popular textbook series also designed for German secondary schools. Her study broadly follows Mindt’s methodology described in 2.1.2.3. A noteworthy difference, however, is that Römer (2005) only examines occurrences of the progressives in the textbook passages intended to reflect spoken language use (printed dialogues, speech bubbles, transcripts of audio materials, etc.). By comparing these with how the progressive is used in everyday conversation among L1 speakers, her study is one of the few investigations of Textbook English to date that genuinely accounts for the fact that mode and register are likely to impact how such a grammatical construction is used in context.

For each occurrence of the progressive in her corpora, Römer (2005) surveyed a wide range of contextual features including tense forms, contraction, polarity, clause type, adverbial specification, verb lemma, subject and object of progressive verb phrases, as well as functional features including time reference, continuousness, repeatedness and framing (see 4.1.2 for details). Among other findings, Römer (2005: 244–245) reports that contracted forms of the auxiliary BE are under-represented among the progressive forms encountered in the textbook dialogues. Furthermore, she reports an
overuse of *he* and *she* in subject positions of progressives, whilst *I*, *it*, *we* and *they* are underused (Römer 2005: 246–248). With respect to the core functions of the progressive, Römer (2005: 260–266; see also Römer 2010: 22–24) notes that proportionally too few occurrences of textbook progressives convey the sense of “repeatedness”. Römer (2005) also attempts to compare the results of some of her analyses across the most frequent verb lemmas; however, the textbook corpus surveyed being relatively small (108,000 words), such comparisons of the contextual use of specific verb lemmas in the progressive are inescapably explorative in nature.

### 2.2.2.4 Modals

Römer (2004) also applies a comparative corpus-based methodology following Mindt (see 2.1.2.3) to compare the frequencies, co-occurrence patterns and functions of modal verbs in authentic spoken British English and in the German secondary school EFL textbook series: *Learning English Green Line New*. As electronic versions of the coursebooks were unavailable, Römer (2004) decided against the compilation of a “pedagogic corpus” containing all the texts featured in the coursebooks, opting, instead, for the non-random selection of 32 texts from the textbook units in which one or more modals are specifically taught. Combined with all the grammar sections from the same textbook series and the content of a grammar book for the same level (*Learning English Grundgrammatik*), the 32 texts were considered to represent “a sample of EFL textbook language – the kind of language pupils are exposed to in the EFL classroom” (Römer 2004: 190). Striking discrepancies are observed between the textbook data and the reference L1 corpus. For instance, whereas Römer’s analysis of the spoken BNC1994 reveals that the modal *would* (and its contracted form ‘d) is the second most frequent modal, it only comes in fifth position in the textbook data (Römer 2004: 193). On the whole, in Textbook English, modals more frequently refer to ability than in naturally occurring conversation. Thus, for *could* and *may*, the meaning of possibility tends to be under-represented in the textbook data. Furthermore, *must* expresses an inference/deduction in over a third of the BNC concordance lines examined, yet this meaning is only very rarely featured in textbooks (Römer 2004: 194). Textbooks also tend to over-represent certain negated modals whilst others are never presented in a negated form in the textbook materials examined (Römer 2004: 194). Römer (2004) notes further mismatches between the two corpora in the context of questions and *if*-sentences featuring modals (Römer 2004: 195).

### 2.2.2.5 Conditionals

Frazier (2003) surveyed eight ESL textbooks for their coverage of hypothetical and counterfactual conditionals. The study concludes that the textbooks largely neglect hypothetical and counterfactual *would*-clauses that are removed from their presumed *if*-clauses and rarely present such clauses in larger units of discourse. To demonstrate the prevalence of such *would*-clauses, Frazier (2003) also conducted a quantitative
and qualitative analysis of 467 instances of *would*-conditionals in one written and two spoken corpora of American English. Frazier (2003: 451) observes that “much more often than not, conditional and hypothetical clauses with the modal *would* are not accompanied by *if*-clauses anywhere near them, much less in the same sentence”. The author draws two major conclusions from his study. On the one hand, he encourages EFL/ESL textbook authors to “move away from the common practice of teaching hypothetical and counterfactual results in *would*-clauses only at the sentence level and only as adjacent to overt *if*-clauses” and on the other, he calls for a re-examination of the grammatical terminology used to describe *would*-clauses, starting with the term *conditional* itself (Frazier 2003: 464).

Taking Frazier’s (2003) corpus-driven data as her basis, Yoo (2013) investigated how *would*-clauses are presented in five Korean high school EFL textbooks which occupy around 70% of the EFL textbook market share in Korea. She extracted all 253 occurrences of *would*-clauses from the textbooks and annotated these for various linguistic variables. Yoo also compared the lexico-grammatical patterns in which the *would*-clauses are embedded and compared their frequencies with Frazier’s (2003) analysis of naturally occurring English corpus data. Yoo (2013) observes great disparities between the textbook sentences and naturally occurring language. For instance, she notes that despite its wide usage, all the textbooks explored fail to include the combination of *would* with the copula verb *seem* (Yoo 2013: 54), thus neglecting to expose learners to double hedging construction with a potentially highly useful pragmatic use. Moreover, no single example of an infinitive-*would* pattern is found in the textbook corpus (Yoo 2013: 81) despite attested frequent occurrence in naturally occurring English as demonstrated by Duffley’s (2006) corpus study.

Gabrielatos’ (2003; 2006; 2013) approach to evaluating the authenticity of *if*-sentences as described in EFL textbooks has already been presented in 2.1.2.4. These studies clearly demonstrate that a large proportion of *if*-sentences found in natural L1 speech and writing cannot be accounted for by the consensual typologies of conditionals typically taught in EFL/ESL textbooks. Remarkably, this is true even when taking account of all the rules and examples featured in the most advanced textbooks examined. Furthermore, Gabrielatos (2013: 158) notes that “ELT coursebooks generally adopt a naive and restricted approach to modal marking, as they tend to focus on central modals”. On the basis of these pedagogically-driven corpus-based studies, Gabrielatos (2013: 155) draws the worrying conclusion that “the pedagogical information in the coursebooks, taken collectively, presented learners not only with a partial picture of the variety of types of conditionals and their respective morphosemantic features, but also a distorted one”.

Using the same textbook corpus as for Römer’s (2005) analysis of the progressive, Römer (2004a) also conducted a study on conditionals in Textbook English. In this
study, the focus lies on the sequences of clauses and tenses in conditional sentences
and collocational patterns within *if*-clauses. Römer (2004a: 158) reports that a higher
proportion of *if*-sentences begin with the *if*-clause in the authentic data than in the
textbook corpus. The results also show that three tense form sequences are vastly
over-represented in the textbooks as compared to the corpora of naturally occurring
speech. These tense sequences correspond to what EFL textbooks and grammar books
usually refer to as Type 1, Type 2 and Type 3 conditionals. Conversely, Römer (2004a:
159–160) demonstrates that the most frequent tense combinations in *if*-sentences in
the spoken component of the BNC2014 (simple present + simple present), as well as
a number of other frequent tense sequences appear to be significantly under-
represented in Textbook English (for similar results, see also Gabrielatos 2003; 2006;
2013; Möller 2020; Winter & Le Foll forthcoming).

2.2.2.6 Reported speech

Applying a manual page-by-page approach (see 2.1.1.2), Barbieri & Eckhardt (2007) surveyed how reported speech is taught in seven popular ESL/EFL grammar textbooks. They report that the textbooks largely focus on indirect reported speech and find a general consensus on the ‘backshifting rule’ for pronouns, adverbials and tense. Whilst all the textbook authors seem to agree that, in general, the verb in the embedded clause should be “backshifted” to the past, there is no agreement as to which specific cases constitute exceptions to this rule. Barbieri & Eckhardt (2007) compared these results to two corpus-based studies on direct (Barbieri 2005) and indirect (Eckhardt 2001) reported speech in real language use, mobilising conversation and newspaper corpora as their data basis. They drew on the most striking discrepancies between the textbooks’ “grammar rules” and the patterns of use that emerge from the authentic data to make ten suggestions to improve the authenticity of EFL textbook’s portrayal of reported speech. Thought-provokingly, Barbieri & Eckhardt (2007) convincingly argue that indirect reported speech should not be taught as a transformation of direct speech (i.e., following the well-known backshifting rule), since the two constructions follow distinctive lexico-grammatical patterning and discourse functions and are used in different communicative situations and registers. Consequently, the authors suggest that indirect reported speech should be taught in the context of newspaper writing, whilst direct reported speech ought to be taught in the context of conversation. Barbieri & Eckhardt (2007) also make recommendations concerning the range of reporting verbs that ought to be associated with certain types of reported speech constructions in EFL/ESL textbooks and encourage textbook authors to highlight the grammatical patterns and discourse functions associated with less frequent tense sequences. Furthermore, they advocate the inclusion of informal quotatives such as *be like*, *go* and *be all* in textbooks, together with context regarding their discourse-pragmatic function and sociolinguistic associations.
### 2.2.3 Pragmatics

This section examines aspects of discourse and pragmatics in Textbook English. The selected studies are discussed in chronological order.

In English, doubt and certainty can be expressed in a variety of ways including using certain modal verbs, adjectives, tag questions, and specific intonation patterns, as well as paralinguistic and non-linguistic devices. The fact that many lexical markers of doubt and certainty are highly polysemous further adds to the complexity of the task. This motivated Holmes (1988) to conduct a survey on the coverage of lexical items commonly employed to express doubt in two EFL/ESL reference grammars and two coursebooks. To this end, Holmes (1988) compared the epistemic lexical items illustrated in the four textbooks with information on the range and frequency of the same lexical devices as retrieved from four different native corpora. In addition to these corpus-based comparisons, Holmes (1988) also referred to previous research findings and to native-speaker acceptability judgments to evaluate the choices of the pedagogical material designers. Holmes' (1988) study concludes that some textbooks paint an entirely misleading picture of epistemic modality compared to real-life English usage, whilst other textbooks neglect the topic altogether. Although corpus data clearly shows that when expressing doubt, native speakers of English do not confine themselves to modal verbs, Holmes (1988: 40) observes that the majority of textbook authors "devote an unjustifiably large amount of attention to modal verbs, neglecting alternative linguistic strategies for expressing doubt and certainty". As a result, she argues that textbooks ought to "present learners with alternative syntactic and lexical devices selected from those occurring most frequently in relevant spoken and written texts" (Holmes 1988: 40).

In academic writing, doubt is frequently expressed in the form of hedging. Hyland (1994) explored the representations of hedging devices in 22 EAP/ESP textbooks designed to help L2 English users acquire Academic English writing skills. Following a page-by-page approach (see 2.1.1.2), Hyland (1994) first drew a list of markers of uncertainty and tentativeness that a number of previous studies have found to be salient in academic writing and proceeded to manually check the EAP/ESP textbooks for evidence of coverage of these hedging devices. The evaluation of the textbooks' coverage of these devices is based on both the number of exercises devoted to these devices and the quality of the information provided on them (Hyland 1994: 244). Hyland's (1994: 250) study concludes that, in general, "the presentation of hedges in published [EAP/ESP] materials is not encouraging, with information scattered, explanations inadequate, practice material limited". Echoing Holmes' (1988) conclusion, Hyland (1994: 244) criticises the fact textbooks hardly present any alternatives to modal verbs for hedging.
For her cursory exploration of pragmatic information featured in English textbooks, Vellenga (2004) selected four EFL integrated skills textbooks and four ESL grammar textbooks that are frequently used in university settings in non-English-speaking countries. The study's methodology has already been detailed in 2.1.1.2. The results can only be considered exploratory but seem to point towards a general paucity of metapragmatic and metalinguistic information in EFL and ESL textbooks. The author notes that metalinguistic information is mostly presented in the form of imperative directives for learners to complete an activity in the textbook. Pronominal reference is often absent. Vellenga (2004) deplores the pragmatic inadequacy of the treatment of most speech acts in the textbooks explored. In particular, she points to the danger of providing unique speech act-grammatical form associations as they may prove misleading and restricting for learners. Based on informal observation, Vellenga (2004: n.p.) argues that the "distribution of speech act types across ESL and EFL textbooks did not appear to be patterned, nor based on frequency of speech act occurrence in natural language". Finally, interviews conducted with four experienced EFL/ESL teachers lead the author to the disconcerting conclusion "that textbooks do provide the majority of input, and that even professional teachers rarely have the time, inclination, or training to include supplementary pragmatic information in their lessons" (Vellenga 2004: n. p.).

Cheng (2007) examined a corpus of spoken English produced by competent English speakers from Hong Kong (the *Hong Kong Corpus of Spoken English*; Cheng, Greaves & Warren 2005) for instances of interruption in the form of initiation of simultaneous talk in conversations. Both the functions of the interruptions and their linguistic realisations were compared to phrases that eleven popular English textbooks used in Hong Kong secondary schools suggest are appropriate for interrupting. Cheng's (2007) results make clear that, by and large, the phrases suggested in the textbooks do not accurately reflect real-language use: the majority of the phrases taught in the textbooks (e.g., *Excuse me, but...*, *Sorry to interrupt...*, *If I could just come in here...*, *I want to say something, please*) do not occur even once in the 800,000-word corpus of naturally occurring Hong Kong English corpus queried.

Drawing on the same reference corpus, Cheng & Warren (2007) evaluated textbook authors' perception and presentation of strategies for monitoring and checking understanding in 15 upper secondary ESL/EFL textbooks. Their manual analysis of the pedagogic texts shows that the textbook authors emphasise the role of the listener in checking understanding, often providing example phrases encouraging the listeners to "seek clarification", "ask for repetition", "say they don't understand", etc. Only four of the 15 textbooks examined also propose strategies for the speaker to "check others' understanding" and "clarify". Contrary to textbook authors' intuition, the authentic conversation data reveal that the primary responsibility in ensuring that understanding has taken place rests, in fact, with the speaker. Cheng & Warren (2007)
also report that some of the phrases suggested by textbook authors to explicitly ask
the listeners whether they have understood (e.g., *Are you with me?* and *Do you
understand me?*) do not appear a single time in the *Hong Kong Corpus of Spoken
English*. Other phrases such as *Is that clear?* only feature in the corpus' academic
subcorpus and are only uttered by teachers speaking to students. Whilst the simple
backchannel *okay* is found to be the third most frequent form of checking
understanding in the corpus of real-life English, it is widely under-represented in
textbook examples. Thus, Cheng & Warren (2007: 202) conclude "that textbooks
contain language forms that are rarely, if ever, used in the real world and are overly
influenced by academic genres". Most interestingly from a pedagogical point of view,
they also conclude that "the most common forms found in the HKCSP [reference]
corpus are both simpler and less explicit than those included in the textbooks" (Cheng
& Warren 2007: 202).

In a similar vein and, again, relying on the same corpora, Cheng & Warren (2005; 2006) reveal that Hong Kong textbook authors' intuitions on forms used to express disagreement (Cheng & Warren 2005) and to give opinions (Cheng & Warren 2006) do not match real language use as documented in the *Hong Kong Corpus of Spoken English*.

In sum, the Textbook English studies surveyed in this section convincingly
demonstrate that textbook authors' intuitions and portrayal of the functions and
linguistic realisations of a wide range of learner-relevant speech acts largely fail to
correspond to evidence from corpora of naturally occurring English. It is worth noting
that, unlike the vast majority of corpus-based textbook language studies examined so
far, the aforementioned studies carried out in the Hong Kong context rely on a
reference corpus that consists of culture-specific spoken interactions between Hong
Kong Chinese speakers and speakers of languages other than Cantonese in a range of
communicative situations that English learners are likely to be confronted with. Thus,
contrary to many of the previously described Textbook English studies that rely on
general L1 corpora of sometimes doubtful relevance to anyone but highly advanced
learners of English (see Tono 2004), the *Hong Kong Corpus of Spoken English* can
reasonably be considered to constitute secondary school target learner language. As
a result of these apparent mismatches between textbook and target learner language,
Cheng & Warren (2007: 200) warn of "an urgent need for some realignment in learning
and teaching materials in terms of language forms and functions selected and
presented to language learners".

2.2.4 Spoken grammar

The final section of this literature review focuses on how spoken grammar is
represented in EFL/ESL textbooks. Attitudinal research indicates that learners of
English are generally very interested in possessing at least receptive knowledge of
spoken grammar, whereas teachers and textbook authors are more divided on the
subject (Timmis 2003). Although he refrains from citing specific textbooks, Timmis
(2003) highlights a number of spoken grammar features that he considers to be largely
ignored by ELF materials designers. These include the *get*-passive, discourse
dimensions of past-aspect choices and certain reported speech forms, in addition to
other grammatical features which analyses of L1 speech corpora have shown to be
very frequent, such as heads, tails and ellipsis structures (see, e.g., Aijmer 2002;
Carter & McCarthy 2006b; Carter & McCarthy 1995; McCarthy & Carter 1995;
Carter, Hughes & McCarthy 1998; McCarthy 1998; Biber & Quirk 2012; Biber et al.
2021). This claim would certainly merit quantitative verification and, in fact, a
number of Textbook English studies are concerned with the specific linguistic features
of spoken English.

One original methodological approach to evaluating the linguistic features of spoken
interactions in EFL textbooks was already presented in 2.1.2.5. As a reminder,
Gilmore's (2004) evaluation of the authenticity of spoken textbook language involved
the comparison of re-creations of communicative situations simulated in textbooks
with the textbook dialogues themselves. Nine discourse features were selected for
comparison: lexical density, false starts, repetition, pauses, terminal overlap, latching,
hesitation devices and backchannels. Gilmore (2004) reports that the 'authentic'
enactments of the dialogues (see caveats in 2.1.2.5) are almost twice as long as their
textbook counterparts. Additionally, the lexical density of textbook dialogues is
higher than that of authentic conversations (though it may be argued that the
difference, based on 200-word samples from only seven examples, does not appear to
be great, nor is this claim backed by the results of any statistical significance testing).
In particular older textbooks were found to hardly feature any of the discourse
features typical of spontaneous spoken interactions.

In a page-by-page survey, Cullen & Kuo (2007) focused on explicit mentions of
features of spoken grammar in 24 global EFL textbooks covering all levels from
beginner to advanced. Drawing on specific examples from Carter (2004), they
investigated three categories of features of spoken grammar. Cullen & Kuo (2007:
361) conclude that “where spoken grammar is dealt with at all, there tends to be an
emphasis on lexico-grammatical features, and common syntactic structures peculiar
to conversation are either ignored or confined to advanced levels as interesting
extras”. One could argue, however, that such structures are more likely to be taught
implicitly than in an overt manner and that, as a consequence, a corpus-based
textbook study that includes the written dialogues and transcripts of the listening
materials accompanying the textbooks may paint a rather different picture.

The results of the studies outlined above confirm that, despite a strong focus in syllabi
on speaking skills and communicative language learning, textbooks seemingly
continue to present a misleading picture of spoken language. Barbieri & Eckhardt (2007: 321) reflect that even more recent publications “neglect important and frequent features of the language spoken by real language users, present a patchy, confusing, and often inadequate treatment of common features of the grammar of the spoken language, and, in sum, do not reflect actual use”. However, this section has shown that studies on representations of spoken English in EFL/ESL textbooks have either only focused on individual linguistic features (e.g., Römer on *if*-conditionals [2004; see 2.2.2.5] and the progressive [2005; see 2.2.2.3]), on explicit mentions of spoken grammar features only (e.g., Cullen & Kuo 2004), or on a very small sample of textbook dialogues (e.g., Gilmore 2004).

## 2.3 Conclusion

This review leaves no doubt that, as suspected by many (former) learners of English as a foreign language in instructional contexts (see 1.1), Textbook English does indeed constitute a distinct variety of English that, in many respects, differs substantially from real-life, naturally occurring English. Section 2.2.2.5 highlighted the fact that some grammar rules promulgated in textbooks are, in fact, not even respected in the extended written passages of the textbooks themselves; thus, pointing to striking intra-textbook inconsistencies and, more generally, to a genuine gap between prescriptive grammars of English and real-life usage.

Section 2.2.3 showed that the results of most studies examining pragmatics in Textbook English stress that textbooks are not providing learners with the right kind of input to develop their pragmatic competence. In spite of some improvements found in studies comparing older with more recent publications (e.g., Gilmore 2004; Jiang 2006; Usó-Juan 2008), critics argue that newer publications do little more than simply lengthen the list of linguistic structures to be used in the context of specific speech acts, yet provide next to no contextual information as to their use in real language use (Usó-Juan & Martínez-Flor 2010: 426).

In terms of methods, we have seen that linguists have, thus far, mostly strived to compare Textbook English to naturally occurring English as produced by native speakers; however, even comparisons of local ESL textbooks with that of English produced by local proficient ESL speakers have shown that the language input learners obtain from their textbooks remains far removed from what they are expected to later engage in outside the classroom (Cheng 2007; Cheng & Warren 2005; 2006; 2007).

In order to better grasp this apparent mismatch between the language of pedagogical materials and target learner language, it is important to consider the factors that may contribute to this gap. The first argument that textbook authors would presumably advance is that their task is to simplify real-life language use to make it accessible to
language learners. In fact, a distinct limitation of many of the studies surveyed as part of this literature review is that they examine Textbook English as one single variety of English, ignoring potential variation related to the different proficiency levels of the textbooks. Whilst there is no doubt that proficiency level must be accounted for in future descriptions of Textbook English (see 3.1), advocates of usage-based L2 instruction models would counter that the most important factors in the construction of Textbook English ought to be the relative frequencies of occurrence and salience of linguistic features (see 1.5). In reality, however, it would appear that the majority of textbook authors still rely on long-established grammar conventions and/or their intuition, rather than on (corpus-based) insights from actual language usage (see 1.8). Many of the conclusions of the Textbook English studies summarised in this chapter thus reminded us that: “Tradition, even if it is most venerable, cannot serve as a substitute for research” (Mindt 1997a: 41). This was particularly obvious in the studies that compared the coverage of key lexical items in different textbooks targeted at the same proficiency level and found very little agreement across different textbook series (e.g., Koprowski 2005; see 2.2.1.2).

Furthermore, many of the studies surveyed as part of this literature review have concluded that textbooks tend to present lexico-grammatical patterns as if they were generalisable across all registers, thus failing to acknowledge crucial differences between different production modes, text types and discourse-context-specific uses (see also Barbieri & Eckhardt 2007: 321). The studies summarised in 2.2.3 and 2.2.4, in particular, suggest that the language of EFL textbooks is predominantly based on norms pertaining to written text registers. Indeed, to a large extent, textbook dialogues fail to account for the processing conditions of spontaneous spoken interaction (e.g., lack of planning, reciprocity, shared environment). The resulting disregard of syntactic and lexico-grammatical features that are typical of unplanned speech therefore also contributes to this prevailing lack of fit between many aspects of Textbook English and real-life language use. At the same time, however, this criticism can be turned on its head. As this chapter has shown, numerous Textbook English studies suffer from severe methodological limitations. Crucially, many of the conclusions drawn on the basis of comparisons between Textbook English and naturally occurring English are likely flawed due to the use of inappropriate or only partially suitable reference corpora (e.g., comparing textbook dialogues to the entire BNC1994).

In spite of their limitations, however, the studies outlined in this chapter, collectively, provide substantial evidence for the frequently idiosyncratic use of specific lexico-grammatical features in Textbook English. They reveal pedagogically questionable gaps between the language input textbooks provide and what learners are expected to eventually engage with. However, in spite of the considerable scope of linguistic features explored thus far, our understanding of Textbook English remains patchy –
not least because no study has yet attempted to provide a comprehensive linguistic description of Textbook English across a broad range of linguistic features and registers, based on appropriate reference corpora and a sufficiently large corpus of EFL textbooks. The present study sets out to contribute to bridging this gap by investigating a broad range of lexico-grammatical features in a large corpus of contemporary EFL textbooks widely used in secondary schools in France, Germany and Spain. The following chapter explains how this literature review informed both the methodological approach of the present study (3.1–3.2) and the design of the corpora used in its comparative corpus-based analyses (3.3).
# 3 Research aims and data

*One does not study all of botany by making artificial flowers.*

❖ John McH. Sinclair (1991: 6)

The present chapter begins by drawing conclusions from the studies discussed as part of the literature review in Chapter 2. On the basis of these insights, the overarching methodological framework for the present study is briefly outlined in 3.2. Four broad sets of research questions are formulated in 3.2.1. These concern the linguistic nature of Textbook English and the various factors that may mediate variation within this variety of English. They will be further specified in each of the four analysis chapters that follow. Since these analysis chapters, Chapters 4–7, can be read as individual studies, they each begin with a set of specific research questions and their own methods sections. However, the analyses presented in Chapters 4–7 are united by both the overarching research aims and questions spelt out in 3.2 and by the fact that they rely on the same corpus data. The second half of the present chapter outlines the various decision-making processes involved in the compilation of these corpora and provides key statistics on their compositions.

## 3.1 Insights from the literature review

Taken together, the studies surveyed in Chapter 2 provide valuable insights into “the kind of synthetic English” (Römer 2004b: 185) that EFL learners are exposed to via their English textbooks. However, the literature review revealed some problematic aspects that have commonly been neglected in past endeavours to study the language of textbooks and which the present thesis aims to address.

First, throughout the literature review, concerns were raised as to the suitability of the reference corpora used in comparative corpus-based analyses. Whilst some have argued that English native speaker standards are not suitable for most EFL learners (for more on this, see 3.3.2.1), the more pressing issue resides in the fact that many of the reference corpora used in previous Textbook English studies do not match the communicative aims and/or target audiences of the textbook texts. Many of the surveyed studies relied on general English corpora such as the British National Corpus (BNC), which is made up of 90% written language, mostly penned by professional writers with an adult readership in mind and edited by professionals for publication. Thus, the present study aims to find and, if necessary, compile the most appropriate reference corpora possible for maximally meaningful comparisons.
Second, though it has long been established that situational characteristics of texts are a major driver of linguistic variation (see, e.g., Biber 2012; Gray & Egbert 2019; Goulart et al. 2020), Chapter 2 repeatedly showed that potential register differences between the various types of texts typically featured in school EFL textbooks have largely been brushed aside. Given that school EFL textbooks may feature, for example, extracts of a short story, a dialogue, instructions, and exercises on any double page, the present thesis hypothesises that Textbook English cannot be meaningfully examined without taking a register-based approach. Up until now, however, register variation within EFL textbooks has largely been ignored (but see Miller 2011 for an exception with respect to university-level ESL textbooks). In the few cases where register has been considered in the analysis of EFL textbooks, the focus has almost exclusively been on representations of spoken language, e.g., Mindt (1987; 1995a) and Römer (2004a; 2005) who compared the dialogues of secondary school EFL textbooks to corpora of spoken and pseudo-spoken native speaker English. However, to the author's best knowledge, other sources of register variation in school EFL textbooks have yet to be explored. Other registers of interest in the analysis of secondary school EFL textbooks may include fiction, task instructions & explanations, informative texts, etc. (see 3.3.1.4).

Another frequently neglected aspect concerns interactions between the frequencies of individual linguistic features. This is important because usage-based approaches to language acquisition (see 1.4–1.5) postulate that the co-occurrence information that learners perceive in language input:

> is stored as points in a multi-dimensional space at coordinates, and that speakers process this stored linguistic information in ways that allow them to identify (under certain conditions and defined by various types of frequency occurrences) abstract linguistic patterns" (Rautionaho & Deshors 2018: 229).

Thus, whilst some influential studies have helped us to understand how EFL/ESL learners can be misled by their textbooks to make unidiomatic use of specific linguistic features (e.g., the progressive aspect; Römer 2005), only a multivariable approach can paint the full picture as to how Textbook English – as a whole – differs from the English that English learners will later encounter and be expected to produce outside the classroom. In fact, even within a case-study approach describing and/or evaluating the representation of a single linguistic feature in Textbook English, potential interactions between different variables ought to be taken into consideration. For instance, if a study reports that certain lexical verbs in the present progressive are under-represented in textbook dialogues as compared to naturally occurring conversation among L1 speakers, this could mean that these verbs are, overall, under-represented in EFL textbooks, across all tenses and aspects. It could, however, equally mean that the present progressive is under-represented as compared to other tenses and aspects. Alternatively, it could point to a genuine under-representation of a specific combination of tense, aspect and verb type. Similarly, if
such a study additionally reports that progressives with contracted auxiliary forms
are under-represented, the question arises as to whether, in natural speech, present
progressives with these particular verbs are proportionally more likely to feature in
contracted forms than other verbs in the progressive, in which case these two findings
can be accounted for by one and the same phenomenon. The problem is that Textbook
English studies, to date, have not accounted for such possible interactions between
the linguistic variables that they have measured, compared and reported.

As already hinted at by these examples, such multi-variable analyses of corpus data
call for appropriate statistical methods. In the methodological review in Chapter 2,
however, it was reported that statistical tests have only rarely been conducted in the
context of quantitative comparative corpus-based analyses of Textbook English.
When they have, chi-squared tests have been favoured (see, e.g., Römer 2005: 60).
These statistical tests, together with other tests popular in corpus linguistics such as
log-likelihood and Fisher's exact tests (Brezina 2018: 112–115), can help researchers
conclude with a certain degree of certainty that frequencies observed in textbook
language differ from the probabilistically expected frequencies (drawn from, e.g., a
reference corpus) because of genuine differences between the corpora, rather than due
to random, chance variation (Levshina 2015: 201–213; Wallis 2020: chap. 8). Many
studies that report tests such as chi-squared tests, however, only report whether the
test returned a *p*-value below a pre-defined threshold (usually 0.05, corresponding to
5% probability that the same test result or a more extreme one could have been
obtained if there were no actual difference between the compared populations). Here,
the problem is twofold. First, *p*-values do not inform the reader as to how large (let
alone: relevant!) these supposed differences actually are and, second, they are
dependent on sample size. Thus, given very large corpus datasets, statistically
significant results (i.e., with very small *p*-values) will almost inevitably be returned
(Baroni & Evert 2009: 787; Gries 2005; or simulate some data to observe this effect
on https://shinyapps.org/apps/p-hacker/ [Schönbrodt 2016]). With small datasets,
by contrast, only extremely large differences between two sets of frequencies will
return significant results. In other words, studies with small sample sizes are often
underpowered and therefore cannot be used to reliably detect anything but huge
effects (Winter 2019: 171–175). Another issue is that when conducting individual tests
on potential differences in the frequencies of many different linguistic variables, *p*
values ought to be corrected to account for multiple comparisons (Wallis 2020: 274–
275; Winter 2019: 175–177). Again, so far, this has rarely been done in corpus-based
analyses of Textbook English.

On a related matter, previous quantitative corpus-based studies of textbook language
have usually been undertaken at the corpus level (rather than at the textbook volume,
chapter, unit or individual text level), thus implicitly assuming that Textbook English
is a homogenous variety of English in which the linguistic features under study are
dispersed evenly within the textbook corpora. Just like learner Englishes have been shown to vary across different registers, tasks, proficiency levels, individual learners, etc., so can Textbook English be expected to vary across different textbook series, targeted textbook audience/instructional setting, proficiency levels, text registers, etc.

## 3.2 The present thesis

The present study is based on Mindt’s comparative, corpus-based methodology (see 2.1.2.3). As explained in the previous chapter, such an approach requires the compilation of two corpora: a textbook corpus and a reference corpus of naturally occurring English representative of the target language students can be expected to aspire to. In addition, various methods that have been proposed to account for the multifactorial/-variate and multilevel structure of learner corpus data (Gries 2013; 2018; Gries & Deshors 2020; Paquot & Plonsky 2017; Möller 2017; Wulff & Gries 2021) have inspired the methodological approaches followed in the present study in an attempt to account for the nested nature of textbook language data. Chapters 6 and 7, in particular, attempt to model the potential impact of the different text registers typically found in EFL textbooks, their varying target proficiency levels, and any potential idiosyncrasies of textbook authors, editors or publishers, as well as how these variables may interact with each other.

### 3.2.1 Research aims and questions

In sum, the present study attempts to describe both the linguistic specificities of Textbook English as a variety of English, as well as to model its internal variation. More specifically, the project addresses the following broad research questions:

1. How homogenous is Textbook English as a variety of English? Which factors mediate intra-textbook linguistic variation?

2. To what extent are French, German and Spanish secondary school pupils confronted with varying English input via their textbooks?

3. To what extent is the language of current EFL textbooks used in secondary schools in France, Germany and Spain representative of ‘real-life’ English as used by native/proficient English speakers in similar communicative situations? To what extent are some registers more faithfully represented than others?

4. Which (clusters of) lexico-grammatical features are characteristic of Textbook English? To what extent are these stable across entire textbook series? To what extent are some of these defining features specific to certain proficiency levels?

This study thus aims to provide an empirical, multi-variable description of the language of a large sample of secondary school EFL textbooks. As such, it is hoped that it can contribute to raising awareness of what constitutes the main variety of English that secondary school students are formally exposed to: “Textbook English”. Ultimately, both the results and some of the methods employed may be used to evaluate future EFL teaching materials. In addition, the results may help EFL
teachers, textbook authors and editors to improve existing teaching materials in order
to better equip learners with the necessary linguistic skills to succeed in the English-
speaking world outside the classroom.

3.2.2 Open Science statement

Another important insight from the methodological part of the literature review (Chapter 2) is that, to the author’s best knowledge, no Textbook English study published so far has included (as an appendix or supplementary materials) the data and code necessary to replicate the published results. This means that it is very difficult to evaluate the reliability or robustness of the results reported. Granted, a major issue in (corpus) linguistic research is that it is often not possible for copyright or, when participants are involved, data protection reasons to make linguistic data available to the wider public. However, both research practice and the impact of our research can already be greatly improved if we publish our code or, when using GUI software, methods sections detailed enough to be able to successfully replicate the full procedures. This step can enable others to conduct detailed reviews of our methodologies and conceptual replications of our results on different data.

Aside from data protection and copyright regulations, there are, of course, many
reasons why researchers may be reluctant to share their data and code (Berez-Kroeker
et al. 2018; McManus 2021). It is not within the scope of this thesis to discuss these;
however, it is clear that, in many ways, such transparency makes us vulnerable. At
the end of the day: to err is human. Yet, the risks involved in committing to Open
Science practices is particularly tangible for researchers working on individual project,
like the present author on this doctoral thesis, who have had no formal training in
data management or programming and have therefore had to learn “on the job”.
Nonetheless, the author is convinced that the advantages outweigh the risks. Striving
for transparency helps both the researchers themselves and others reviewing the work
to spot and address problems. As a result, the research community can build on both
the mishaps and successes of previous research, thus improving the efficiency of
research processes and ultimately contributing to advancing scientific progress.

It is with this in mind that the author has decided, whenever possible, to publish all
the raw data and code necessary to reproduce the results reported in the present
thesis following the FAIR principles (i.e., ensuring that research data are Findable,
Accessible, Interoperable and Reusable; Wilkinson et al. 2016). For copyright reasons,
the corpora themselves and annotated corpus data in the form of concordance lines
cannot be made available. However, the outcome of both manual and automatic
annotation processes is published in tabular formats in the *Online Appendix*. These
tables allow for the reproduction of all the analyses reported on in the following
chapters using the reproducible data analysis scripts also published in the *Online
Appendix* and on GitHub (https://github.com/elenlefoll/TextbookEnglish). In all
chapters of this thesis, full transparency is strived for by reporting on how each sample
size was determined and on which grounds data points were excluded, manipulated
and/or transformed. Most of these operations were conducted in the open-source
programming language and environment R (R Core Team 2020). Most of the data
processing and analysis scripts therefore consist of R notebooks. These were rendered
to HTML pages (viewable in the *Online Appendix*) thus allowing researchers to
review the procedures followed without necessarily installing all the required packages
and running the code themselves. These scripts also feature additional analyses, tables
and plots that were made as part of this study but which, for reasons of space, were
not reported on in detail in the present thesis. Whenever additional software or open-source code from other researchers were used, links to these are also provided in the
*Online Appendix* (in addition to the bibliographic references in the corresponding
sections of the thesis).

## 3.3 Corpus data

Having outlined the overarching aims and research questions of the present thesis (3.2.1) and its commitment to Open Science (3.2.2), the second half of this chapter describes the corpora drawn upon to explore the characteristics of Textbook English as a variety of English.

The first part (3.3.1) explains the rationale behind the many decision processes
involved in the creation of the Textbook English Corpus (TEC), e.g., the selection of
the textbooks to be included in the corpus (3.3.1.1) and more technical aspects of the
corpus compilation process such as the software used to process scans of physical
copies of the textbooks obtained in print and those used to process digital copies of
various formats (3.3.1.2), as well as the XML syntax used for the mark-up of textbook
metadata (3.3.1.3) and register annotation (3.3.1.4).

The second part (3.3.2) is devoted to the three reference corpora used to compare and
contrast Textbook English with naturally occurring ‘real-life’ language deemed to be
representative of the kind of English that secondary school L2 English learners can
be expected to understand and produce once they have completed their secondary
school education. The reasoning behind the choice of the Spoken BNC2014 and how
the untagged XML version of the corpus was processed for the purposes of this study
are explained in 3.3.2.1. This section also includes a brief excursus on the choice of
British English as the reference norm for the present analyses. The remaining two
reference corpora – the Youth Fiction Corpus and the Informative Texts for Teens
Corpus (abbreviated to Info Teens) – were compiled especially for the present project.
Their composition and respective corpus building processes are described in 3.3.2.2
and 3.3.2.3.
### 3.3.1 The Textbook English Corpus (TEC)

The present study aims to examine the English language content EFL learners are exposed to in secondary school settings. To conduct a corpus-based analysis of this input, it is necessary to compile a pedagogic corpus, which Hunston (2002: 16) defines as:

> A corpus consisting of all the language a learner has been exposed to. For most learners, their pedagogic corpus does not exist in physical form. If a teacher or researcher does decide to collect a pedagogic corpus, it can consist of all the course books, readers, etc., a learner has used, plus any tapes etc they have heard. [...] It can also be compared with a corpus of naturally occurring English to check that the learner is being presented with language that is natural-sounding and useful.

Collecting data for a pedagogic corpus as defined by Hunston is undoubtedly a highly ambitious project. Although not explicitly mentioned, her definition implies that it should also include all teacher-student and student-student interactions in the L2 and would thus be specific to each and every class group and, even, learner. If, however, in “input-impoverished EFL context[s]” (Meunier & Gouverneur 2007: 122), textbooks do indeed account for such a large proportion of the learner language input, it follows that the textbooks themselves can be considered as a kind of “learner input corpus” (cf. Gabrielatos 1994: 13). In 1.6, textbooks were shown to be a key source of language input in school EFL classroom settings. This formed the starting point for investigating the language of EFL textbooks used at lower secondary school level in France, Germany and Spain. In designing and compiling any corpus, several aspects must be carefully considered. These include the corpus specification, the data collection sampling frame and considerations pertaining to corpus size, representativeness, and balance, all which are explained in the following sections.

#### 3.3.1.1 Selection of textbooks

No matter how large, corpora tend to represent only a sample of a target population, with few exceptions such as corpora of individual authors’ complete published works. In this study, however, the target population is defined as the English language content of all the textbooks from which all lower secondary school students in France, Germany and Spain were learning English as a second or foreign language between 2016 and 2018. Since the school systems are organised differently across the three countries of interest, lower secondary school is defined here for comparison as the equivalent to the German *Sekundarstufe I* or ISCED 2 (Unesco 2012; 2015), i.e., the stage where pupils are usually expected to be aged between 11–12 to 15–16 years. In most OECD countries, this period coincides with compulsory secondary education. The corresponding educational levels and year groups for France, Germany and Spain are displayed in Table 3 (data from Fournier, Gaudry-Lachet & DEPP-MIREI 2017). To compare textbooks aimed at similar levels and year groups across these different educational systems, an additional universal “country-neutral” textbook level variable is used throughout this study (see first column of Table 3). Textbooks for more
advanced secondary school English L2 learner were not included in the corpus because textbooks are used more sparingly beyond the first four to five years of secondary school EFL instruction. Although still present in many European classrooms, they are often supplemented with other “real-life” materials (see, e.g., Leroy 2012: 72 for the French context).

<table><caption>Table 3: The levels of the Textbook English Corpus (TEC)</caption><thead><tr><th>TEC Level</th><th>France</th><th>Germany</th><th>Spain</th><th>Pupil age (approx.)</th></tr></thead><tbody><tr><td>A</td><td></td><td></td><td>Educación Primaria</td><td>11</td></tr><tr><td>B</td><td rowspan="2">Collège</td><td>5<sup>e</sup></td><td>6. Klasse</td><td>1° ESO</td><td>12</td></tr><tr><td>C</td><td>4<sup>e</sup></td><td>7. Klasse</td><td>2° ESO</td><td>13</td></tr><tr><td>D</td><td>3<sup>e</sup></td><td></td><td>8. Klasse</td><td>3° ESO</td><td>14</td></tr><tr><td>E</td><td>Lycée</td><td>2<sup>e</sup></td><td>9./10. Klasse</td><td>Ciclo</td><td>4° ESO</td><td>15</td></tr></tbody></table>

Since the French, German and Spanish educational authorities do not prescribe the use of any particular textbooks in state schools, a vast number of different textbook series from a range of publishers are currently in use. For a textbook corpus to capture the full variability of lower secondary school EFL textbooks used in France, Germany and Spain, it would have to include all the textbooks in use, including possibly some older or little-known editions favoured by individual schools or teachers. However, the principle of representativeness, as described by Biber (1993a), also implies that the corpus ought to be representative of the textbook language to which as many pupils as possible are exposed, so that conclusions drawn from the sample of textbooks contained in the corpus may be confirmed in a larger sample of textbooks. This, in turn, implies that the most popular textbooks used in the majority of classrooms ought to be included in the corpus. Since textbook sale figures are not publicly available, informal surveys were conducted with local teachers (in EFL teacher Facebook groups), bookshop assistants and publisher representatives to establish a list of the most widely used school EFL textbooks in France, Germany and Spain.

Table 4 summarises the results of these informal market surveys, which revealed differences between countries in school textbook market dynamics. In Germany, the textbook market is dominated by three publishers (Klett, Cornelsen and Diesterweg), which each offer one major English textbook series per school form (usually: *Hauptschule*, *Realschule*, *Gesamtschule* and *Gymnasium*) and which, to a lesser extent, may be adapted to match the requirements of specific *Länder* (note that in Germany, education falls under the responsibility of the *Länder* as opposed to the federal ministry). By contrast, France has a centralised national educational system.
That said, schools and teachers are also free to choose whether to use textbooks at all, and if so, which. In general, there is a tendency to be more critical of textbooks, with some trainee teachers instructed not to use any commercial textbooks, or at least to design their own lesson units selecting suitable materials from a range of textbooks and authentic materials, rather than religiously following one series. In practice, however, it would appear that the majority of English teachers in lower French secondary schools do largely rely on one textbook per year group (Leroy 2012: 62) and, in fact, the school textbook market in France continues to show record growth in spite of the concerns voiced by critics (Syndicat national de l'édition 2021).

Whilst the textbooks used in French and German secondary schools are usually published in France and Germany, Spanish schools, teachers (and parents?) seem less convinced of the quality of their locally published textbooks and, as a result, the textbook market is largely dominated by Anglo-Saxon publishers. It may be speculated that such “imported textbooks” are favoured for the same reasons as they are in Southeast Asia, where Dat (2008) reports that they are perceived as being more visually and thematically appealing, linguistically accurate, and systematic in their pedagogical approaches compared to “domestic textbooks”. Since the only Spanish publisher of school English textbooks featured in the informal survey list did not wish to contribute textbooks to the present project and was technically unable to sell digital textbooks without a Spanish ID number, only Anglo-Saxon published textbook series could be included in the Spanish textbook sub-corpus. These textbook series are generally the result of a core “global coursebook” (Gray 2002: 151, though the article dates from a time where this phenomenon was largely restricted to adult EFL/ESL textbooks series) sold to a number of target countries with “differentiated supplementary materials [...] often written by local authors with specific local knowledge [...] to give the teachers ‘a better fit’” (Gray 2002: 165).

<table><caption>Table 4: Most widely used lower secondary school textbook series (and publisher in brackets) according to the informal market surveys conducted with teachers, bookshop assistants and publishers in France, Germany and Spain</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>France</td><td></td></tr><tr><td>Hi there! (Bordas)</td><td></td></tr><tr><td>Join the team (Nathan)</td><td></td></tr><tr><td>New Enjoy (Hatier)</td><td></td></tr><tr><td>E for English (Didier)</td><td></td></tr><tr><td>Piece of Cake (Le Livre Scolaire)</td><td></td></tr><tr><td>New Connect (Hachette)</td><td></td></tr><tr><td>Spain</td><td></td></tr><tr><td>High Achievers (Richmond)</td><td></td></tr><tr><td>Fast Track (Richmond)</td><td></td></tr><tr><td>Action! (Burlington)</td><td></td></tr><tr><td>Real English (Burlington)</td><td></td></tr><tr><td>English in Use (Burlington)</td><td></td></tr></tbody></table>
<table><tr><td>English in Mind for Spanish Speakers (Cambridge University Press)</td></tr><tr><td>English File (Oxford University Press)</td></tr><tr><td>Germany</td></tr><tr><td><em>Gymnasium</em></td></tr><tr><td>Green Line (Klett)</td></tr><tr><td>Access G (Cornelsen)</td></tr><tr><td>Camden Town (Diesterweg)</td></tr><tr><td><em>Gesamtschule</em></td></tr><tr><td>Orange Line (Klett)</td></tr><tr><td>Lighthouse (Cornelsen)</td></tr><tr><td><em>Hauptschule</em></td></tr><tr><td>Blue Line (Klett)</td></tr><tr><td><em>Realschule</em></td></tr><tr><td>Red Line (Klett)</td></tr></table>

In designing the sampling frame of the Textbook English Corpus (TEC), the aim was to select three recently published textbook series per country, ideally by three different major publishers, from the list compiled from the informal surveys (see Table 4). The selection was based on the following opportunistic criteria:

1. Availability of the textbooks in

a. Text/PDF format

b. Other digital formats and

c. Print (in that order)

2. Price

Price was particularly relevant for French and Spanish textbooks as some publishers were only willing to sell digital textbooks in bundles of 20 textbooks or more. Availability may seem an odd criterion in an interconnected globalised world, but in some cases buying even a single digital copy of a Spanish textbook requires a valid Spanish ID number. Digital textbook formats also raised technical issues. Whilst PDF textbooks are relatively easy to convert to text using standard optical character recognition (OCR) software, many digital textbooks are only available as complex flash files designed for use with smartboards and/or tablets. These had to be converted to PDF on a page-by-page basis (though this was automated with a script) before they could be converted to text. Finally, two textbook series were obtained in print and scanned to PDF for further OCR processing.

The textbooks included in the TEC are listed in Table 5 (the full bibliographic metadata is available on doi.org/10.5281/zenodo.4922819). To ease comparisons across different educational systems, whenever possible, five textbooks per series were included. As a result, two French textbook series designed for use in *collèges* (corresponding to the first four years of secondary school education, see Table 3 – note, also, that French school years are counted backwards) are complemented with
a fifth textbook aimed at first year *Lycée* students in Seconde (2ᵉ; see Table 3) from
the same publisher. This was not possible in the case of the most recent textbook
series, *Piece of Cake*, since this relatively new publisher had, as of September 2018,
not yet entered into the *lycée* textbook market. In terms of its marketing concept,
*Piece of Cake* is rather different from the other textbook series featured in the TEC
since it was co-authored by, at the time of writing, over 100 school English teachers,
is published under a Creative Commons license for free modification and use, and is
available online in its entirety for free on https://www.lelivrescolaire.fr. Though it
was still relatively new when the TEC was compiled, the informal market study
revealed it to be a very popular series in French secondary schools already.

Every effort was made to also include all the (tran)scripts of the audio and video
materials belonging to the textbooks of the corpus. When they were not provided by
the publishers themselves, this involved trawling through teachers' books and
textbook home pages to access the materials. Unfortunately, transcripts could not be
sourced for the older version of *Green Line* (Klett, 2006–2009 edition) or *Achievers*
(Richmond). None of the textbooks' accompanying workbooks were included. This
decision was based on pragmatic time and resource constraints and is justified by the
fact that many schools do not require parents to buy the workbooks and many
teachers do not, or only rarely, use them.

Though the TEC was constructed as a balanced corpus with three textbook series for
each country of use, as shown in Fig. 1, the total word count is not spread equally
across the three “national” subcorpora. In terms of quantity, the French textbooks
feature considerably less English input than the German and Spanish ones.

<figure><img src="image_3.png" /><figcaption>Fig. 1: Proportion of tokens in the three "national" subcorpora the TEC (as displayed by Sketch Engine; Kilgarriff et al. 2014)</figcaption></figure>

Of the textbooks featured in the TEC, the only series advertised as corpus-informed
is *English in Mind* from Cambridge University Press. However, many of the other
textbook series claim to include a large proportion of “authentic materials”.
<table><caption>Table 5: Composition of the Textbook English Corpus (TEC) (the full bibliographic metadata is available on doi.org/10.5281/zenodo.4922819)</caption><thead><tr><th>Country of use</th><th>Publisher</th><th>Textbook series</th><th>Volume</th><th>Level</th><th>Publication date</th></tr></thead><tbody><tr><td rowspan="10">France</td><td rowspan="4">Bordas</td><td rowspan="4">Hi There</td><td>6<sup>ème</sup></td><td>A</td><td>2012</td></tr><tr><td>5<sup>ème</sup></td><td>B</td><td>2013</td></tr><tr><td>4<sup>ème</sup></td><td>C</td><td>2014</td></tr><tr><td>3<sup>ème</sup></td><td>D</td><td>2015</td></tr><tr><td rowspan="2">New Mission</td><td>2<sup>nde</sup></td><td>E</td><td>2014</td></tr><tr><td>6<sup>ème</sup></td><td>A</td><td>2010</td></tr><tr><td rowspan="4">Nathan</td><td rowspan="4">Join the Team</td><td>5<sup>ème</sup></td><td>B</td><td>2011</td></tr><tr><td>4<sup>ème</sup></td><td>C</td><td>2012</td></tr><tr><td>3<sup>ème</sup></td><td>D</td><td>2013</td></tr><tr><td>2<sup>nde</sup></td><td>E</td><td>2010</td></tr><tr><td rowspan="4">Le Livre Scolaire</td><td rowspan="4">Piece of Cake</td><td>6<sup>ème</sup></td><td>A</td><td rowspan="4">2017</td></tr><tr><td>5<sup>ème</sup></td><td>B</td></tr><tr><td>4<sup>ème</sup></td><td>C</td></tr><tr><td>3<sup>ème</sup></td><td>D</td></tr><tr><td rowspan="10">Germany</td><td rowspan="4">Klett</td><td rowspan="4">Green Line</td><td>1</td><td>A</td><td rowspan="4">2006</td></tr><tr><td>2</td><td>B</td></tr><tr><td>3</td><td>C</td></tr><tr><td>4</td><td>D</td></tr><tr><td rowspan="4">Klett</td><td rowspan="4">New Green Line</td><td>5</td><td>E</td><td>2008</td></tr><tr><td>1</td><td>A</td><td>2014</td></tr><tr><td>2</td><td>B</td><td>2015</td></tr><tr><td>3</td><td>C</td><td>2016</td></tr><tr><td rowspan="4">Cornelsen</td><td rowspan="4">Access G</td><td>4</td><td>D</td><td>2017</td></tr><tr><td>5</td><td>E</td><td>2018</td></tr><tr><td>1</td><td>A</td><td>2013</td></tr><tr><td>2</td><td>B</td><td>2014</td></tr><tr><td rowspan="4">Richmond</td><td rowspan="4">Achievers</td><td>3</td><td>C</td><td>2015</td></tr><tr><td>4</td><td>D</td><td>2016</td></tr><tr><td>5</td><td>E</td><td>2017</td></tr><tr><td>1</td><td>A</td><td>2015</td></tr><tr><td rowspan="10">Spain</td><td rowspan="4">Cambridge University Press</td><td rowspan="4">English in Mind</td><td>A1+</td><td>A</td><td rowspan="4">2010</td></tr><tr><td>A2</td><td>B</td></tr><tr><td>B1</td><td>C</td></tr><tr><td>B1+</td><td>D</td></tr><tr><td rowspan="4">Oxford University Press</td><td rowspan="4">Solutions</td><td>B2</td><td>E</td></tr><tr><td>Starter</td><td>A</td></tr><tr><td>1</td><td>B</td></tr><tr><td>2</td><td>C</td></tr><tr><td rowspan="4">Oxford University Press</td><td rowspan="4">Solutions</td><td>3</td><td>D</td><td rowspan="4">2011</td></tr><tr><td>4</td><td>E</td></tr><tr><td>Elementary</td><td>A</td></tr><tr><td>Pre-Intermediate</td><td>B</td></tr><tr><td rowspan="3">Oxford University Press</td><td rowspan="3">Solutions</td><td>Intermediate</td><td>C</td><td>2016</td></tr><tr><td>Intermediate Plus</td><td>D</td><td>2017</td></tr><tr><td>Intermediate</td><td>D</td><td>2017</td></tr></tbody></table>
### 3.3.1.2 Textbook processing

The nine selected textbook series, or 42 textbook volumes (see Table 5), were processed so as to include as much of the textual content as possible. All PDF files were processed with high-performing OCR software (ABBYY FineReader 14 Corporate). The results were saved as text (.txt) files for future processing with corpus analysis software. All non-text elements such as images, symbols, font specifications, etc. were discarded.

### 3.3.1.3 Corpus mark-up: headers

To ensure maximum compatibility across operating systems and software, the corpus files were saved with Unicode UTF-8 encoding. In keeping with standard corpus practice, eXtensible Markup Language (XML) was used for the markup and annotation. However, in line with Atkins et al.’s (1992) advice to aim for “a level of mark-up which maximizes the utility value of the text without incurring unacceptable penalties in the cost and time required to capture the data” (Atkins, Clear & Ostler 1992: 9; see also Hardie 2014), the standards usually advocated for XML corpus mark-up, such as the Corpus Encoding Standard (CES; Ide 1996) and the Text Encoding Initiative (TEI; Burnard, Lou & Bauman 2021), were deemed unnecessarily detailed and too labour intensive. Since it was not relevant to the research question at hand, the textbooks’ structural and formatting elements (e.g., paragraphs, font formats, section dividers, etc.) were not annotated and the design of the XML schema was therefore kept as simple as possible.

The metadata associated with each textbook was encoded in a simple XML header at the start of each textbook file, e.g.:

```xml
<doc sign="POC4" series="Piece of cake" level="C" publisher="Livre scolaire" year="2017" country="France">
```

Each file header includes:

i. A unique file name (doc sign)

ii. The name of the textbook series

iii. The textbook proficiency level (according to the country-neutral scale introduced in 3.3.1.1 and Table 3)

iv. The publisher

v. The date of publication

vi. The country in which the textbook is used (i.e., France, Germany or Spain).

This simple metadata markup schema makes it possible to restrict corpus searches to subsections of the corpus, choosing for instance one or more level(s), publisher(s), series, country, a publication date range, or any combination of these parameters in off-the-shelf corpus software such as Sketch Engine (Kilgarriff et al. 2014), as well as via custom scripts in, e.g., R or Python.
3.3.1.4 Register annotation

A major pedagogical implication to emerge from corpus-based research is “the centrality of *register* for studies of language use” (Conrad & Biber 2013: 334, emphasis added). Thus, before outlining the register annotation process, a brief excursus on register is called for. Indeed, corpus studies have consistently shown that, at all linguistic levels, lexico-grammatical patterns are distributed systematically differently according to the communicative purposes and situations of use of the texts under study (e.g., Biber 1988; Biber et al. 1999). Pedagogical approaches to language description, however, have, if at all, tended to focus on linguistic variation across national (and more rarely regional and sociocultural) varieties of English only. This is highly problematic given that the few studies that have attempted to quantify both generic and geographic dimensions of variation have consistently found that text genre/register is a much more powerful predictor of variation than geography (e.g., Bohmann 2017). Similarly, textbook language evaluations have traditionally either considered textbook language as one register (e.g., Ljung 1990), thus disregarding major intra-textbook register variations, or focused solely on textbook dialogues (e.g., Mindt 1995b; Römer 2005). Aiming for a comprehensive lexico-grammatical analysis of Textbook English, this study accounts for the different registers featured in school EFL textbooks, in recognition of the fact that “[s]trong patterns in one register often represent only weak patterns in other registers, and, consequently, few descriptions of language are adequate for a language as a whole” (Barbieri & Eckhardt 2007: 325). As a result, the present study aims to explore the lexico-grammatical specificities of a range of typical textbook registers. To this end, the main textbook registers featured in school EFL textbooks were first identified. Before this procedure is described, a few definitions are in order.

Numerous attempts have been made to tease apart the often partly overlapping terms *genre*, *register* and *text type*. For many researchers, the use of the term *genre* or *register* is a matter of tradition or personal choice; both have been used to refer to text varieties associated with specific situations of use and communicative purposes (Egbert, Biber & Davies 2015). Biber & Conrad's (2019) framework, however, distinguishes between the two terms. *Genres* are text categories whose definitions are based on their conventional structures. In general, *genre* studies have tended to focus on socio-cultural aspects (Biber 2006: 11). *Registers*, on the other hand, are text categories that are defined according to their situational characteristics, such as their communicative purpose, the type of interaction and participants they involve, and topic (though the latter is controversial, see Lee 2001) (e.g., Biber 2006; Egbert, Biber & Davies 2015; Biber & Conrad 2019).<fnref n="13" /> *Register* studies home in on the typical

<footnote n="13">¹³ It is worth mentioning, however, that in pedagogically-motivated systemic functional linguistics (SFL), the term *genre* has frequently been used to refer to what most other SFL research refers to as</footnote>
lexico-grammatical features of particular registers, thus revealing the systematic use
of specific features in particular contexts of use. Finally, the term *text type* refers to
text varieties which are initially defined according to similarities in linguistic form,
i.e., in the co-occurrences of lexico-grammatical features. In other words, the terms
genre, register and text type refer to different, external or internal, yet
complementary, perspectives on text varieties (for a more detailed discussion, see Lee
2001). For the purposes of this study, the term *register* is preferred because the
different text varieties found in the textbooks are initially distinguished according to
their situational features, before being functionally analysed on the basis of their
specific linguistic features. Put differently, it is assumed that a register’s defining
lexico-grammatical features serve a functional purpose within a particular situational
context of use. For instance, it is known that most face-to-face conversations call for
the frequent use of first and second personal pronouns (Biber 1988).

As mentioned earlier, textbook corpus research has largely evaded the question of
register in textbook language. This is possibly due to the emphasis placed on the text
unit in text-linguistic research that has traditionally been based on the assumption
that "texts are nested within registers, but the opposite is not true: registers are not
nested within texts" (Egbert & Mahlberg 2020: 75). In an article entitled 'Fiction –
one register or two?', Egbert & Mahlberg (2020) break away from this tradition by
analysing the linguistic variation within novels whilst distinguishing between passages
of narration and fictional speech in fictional writing. As a result, coherent texts were
divided into text segments that, while situationally different, remain contextually
interdependent. To a certain extent, subdividing textbooks into texts of different
registers may also be interpreted as dividing whole, coherent texts into text segments.
In actual fact, defining text units, as required in text-linguistic corpus approaches
where each text represents one observation (Biber et al. 2016: 357), is particularly
tricky when it comes to textbooks. Indeed, typical school textbooks offer a range of
plausible units of observation: the textbook series, textbook volume, chapter/unit,
subchapter/unit, right down to the individual text (Le Foll 2020c). This study applies
the smallest of these units where any one exercise, reading passage, explanation,
instruction, dialogue or transcript corresponds to one observation. At the same time,
each text observation is nested within one of the 43 textbook volumes and nine
textbook series (see Table 5).

Text subdivision and annotation was performed manually. As part of this process,
each text was manually annotated for register. First, however, it was necessary to
identify meaningful register categories for all the texts featured in the 43 textbooks
of the corpus. To the author's best knowledge, this had not been attempted in this
form before. The most detailed textbook mark-up scheme the author is aware of is

<footnote>register (this is particularly true of the Sydney School, e.g., Martin & Rose 2008; Martin 2009; Rose & Martin 2012).</footnote>
that of the TeMa project (Meunier & Gouverneur 2009). The TeMa corpus consists
of 32 English for General Purposes (EGP) textbook volumes. Both the coursebooks
and workbooks of each textbook are subdivided into four subcorpora: texts,
transcription of the tape scripts, vocabulary exercises and the guidelines to these
exercises (Meunier & Gouverneur 2009: 7). Frequently, however, textbook
instructional language has either been annotated for separate analysis or entirely
removed from textbook corpora. For instance, in an exploration of lexical clusters in
EAP textbooks, Wood (2010) organised his textbook corpus into two subcorpora: one
containing the main textual elements of textbooks, and the other capturing the
instructional material. He also pruned the raw text data of titles, headings, tables of
content, prefaces, etc., thus obtaining a textbook corpus of approximately 580,000
tokens, of which 68% consisted of instructional material. In a different study
comparing business and engineering textbooks with EAP textbooks, Wood & Appel
(2014) acknowledged the difficulty arising from the wide range of registers found in
textbooks. They therefore removed all instructional language from their business and
engineering textbooks and annotated their EAP textbook corpus so as to create two
subcorpora: one for the reading texts, and a second for all “instructional language”
including vocabulary and comprehension exercises (which accounted for ca. 42% of
the total word count).

This study aims to account for, among other factors, register-based variation in
modelling Textbook English so that a simple division of (reading/listening) text vs.
instructional language is not satisfactory. To determine suitable textbook text register
categories, the following cyclical categorisation process (inspired by Mayring 2010:
84–85; Kuckartz 2014: 43–44) was applied:

1. Define the aim of the categorisation according to the research questions.

2. Set selection criteria according to the research aim and research questions.

3. Set appropriate abstraction levels for the register categories.

4. First cycle coding: Begin annotating one textbook from each series.

5. Proceed with category differentiation and subsumption.

6. End first cycle coding and verify all categories for duplication and/or incoherence in abstraction levels.

7. If necessary, re-define categories.

8. Second cycle coding: Re-annotate the textbooks that have already been annotated using the up-dated categorisation system.

9. Annotate the remaining textbooks.

10. Spot check the files for any annotation errors.

In order to reduce the manual annotation workload as well as the risk of inattention
errors, short scripts were inputted into Keyboard Maestro 7.3.1 which enabled the
two annotators to simply highlight each text in the raw textbook files and then press
two keys for the shortcut corresponding to the correct register category (e.g., cmd + I
for 'Instructional') for the section to be automatically annotated with the appropriate
XML syntax, e.g., adding `<div type="instructional">` before the highlighted passage and `</div>` at the end (see Le Foll 2020c for details and video demonstration). The final annotated version of the TEC files contains over 52,000 individual div type tags.

(1) <div type="instructional"> Try to guess what each piece of information refers to. </div><div type="individual words or sentences">Olivia Timothy Liverpool London nd of March Fiona Loudon 4 Ella Steven Spielberg Casino Royale going to the theatre </div><div type="instructional"> Then listen to him speak about his life and check if you were right. </div>

Although the annotation process was undertaken directly in the raw text files, it was frequently necessary to refer to the PDF versions of the textbooks in order to decide on the appropriate register. This manual register annotation process was very time-consuming; however, it also served to thoroughly check the OCR process across all the textbooks. This was important because the use of original fonts and the frequently complex formatting of blocks of texts on individual pages of the textbooks meant that it was often necessary to correct OCR mistakes and in some cases re-type or re-organise sections of the processed text files. In other words, the register annotation process also contributed to the cleaning of the raw data. In the rare cases where text was simply not retrievable (i.e., due to poor scanning), it was not annotated and therefore not included in the TEC so as not to compromise future analyses.

The register annotation was carried out by the author and a student research assistant. The reliability of the annotation scheme and method was tested by having both coders blind-annotate three full textbook volumes and comparing the results. Inter-rater agreement rate was found to be satisfactorily high (96.65%). The only notable difficulty consisted in distinguishing between individual sentences and isolated words/phrases; hence these two categories were merged into one in the final annotation scheme.

Ultimately, the cyclical categorisation process outlined above led to the creation of eight textbook register categories (see Table 6 and Fig. 2): Conversation, Informative writing, Fiction, Personal correspondence (letters, diary entries, social media posts, and e-mails), Instructional (instructions and explanations), Poetry (songs and poems), Other texts (timetables, shopping lists, etc.) and Individual words or sentences. Tags were also added to identify textbook passages in languages other than English (i.e., explanations or translations in the students’ L1/school language). In the following, each register category and its rationale are briefly outlined.
<table><caption>Table 6: Number of words per textbook register categories in the TEC (as calculated by Sketch Engine Kilgarriff et al. 2014)</caption><thead><tr><td>Register</td><td>Words</td></tr></thead><tbody><tr><td>Conversation (spoken)</td><td>508,370</td></tr><tr><td>Fiction (narrative)</td><td>253,836</td></tr><tr><td>Individual words or sentences</td><td>913,331</td></tr><tr><td>Informative texts</td><td>302,739</td></tr><tr><td>Instructional texts</td><td>591,743</td></tr><tr><td>Personal correspondence (personal)</td><td>67,050</td></tr><tr><td>Poetry & rhyme (poetry)</td><td>26,174</td></tr><tr><td>Other texts in English (other)</td><td>14,379</td></tr><tr><td>Non-English texts (foreign)</td><td>346,336</td></tr><tr><td>Total</td><td>3,023,958</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>Fig. 2: Proportion of tokens in each register category on the TEC</figcaption></figure>

In Chapter 2, we saw that some Textbook English studies have focused exclusively on the dialogues and other spoken-like language featured in EFL textbooks (e.g., Mindt 1992; Römer 2005). In this study, all dialogues, scripts and transcripts of the audio and video materials accompanying the textbooks purporting to be unscripted were annotated as one register labelled ‘Conversation’, e.g.:

(2) Nice of you to let us come to your barbie, Mike.
No worries. Great you're here. - Hey Cam! Come and meet a couple of new
mates. They're staying at the hostel. Hey, how're you doing?
Hi. I'm Tanya. Nice to meet you.
You're a Kiwi, right? From your accent? And you're, let me guess ...
American? No, I'm from Israel. Moshe.
OK, cool. You know Mike can trace his ancestors right back to the first
British convicts in Australia?
Come on, Cam. Not that joke again! The story is: My ancestor Bill was
walking down the road when a man bumped into him. The man was being
chased by a police officer because he'd stolen a gold necklace from a
The ‘Informative’ register tag was used to annotate factual articles, newspaper-like writing, reports (including scripted oral reports featured in video and audio materials) and texts from informative websites, e.g.:

(3) English is an official language in over seventy-five countries in the world.
More than two billion people speak English. Fifty-four English-speaking countries are members of the Commonwealth of Nations, an association of independent countries. Queen Elizabeth II is head of the Commonwealth.
31 % (percent) of the world's population live in the Commonwealth.
Six people out of ten in the United Kingdom have a relative in a Commonwealth country. <TEC: Hi There 5>

‘Fiction’, or narrative, texts in lower secondary school EFL textbooks are mostly
found in the form of short stories and extracts of novels, e.g., (4). To keep the integrity
of the texts, direct speech passages in these narrative texts were not annotated
separately (for a different approach, see Mindt 1995a: 7).

(4) With my backpack in my hands, I stepped off the train onto the crowded platform. It was 7:30 in the evening. People were hurrying home. A mother and her two young children were sitting on a bench. The mother was talking to the boy, but he wasn't looking at her. The girl was singing quietly and playing with a toy. Around them, travellers were shouting greetings, waving goodbye, carrying heavy bags or running to catch trains. A very tall man was standing completely still near the exit. Why was he wearing summer clothes in this weather? And why was he looking straight at me? [...] <TEC: Solutions Pre-intermediate>

Singapore Wala (2013: 134) describes textbooks' "narratorial voice [...] as the formal and most powerful means of structuring the relationship between coursebook and learner". As such, the coursebook narrator is responsible for informing, instructing learners to carry out learning tasks and asking questions to seek information in order for learners to meet their learning objectives. Here, the textbooks' "narrative voice" was annotated as 'Instructional', regardless of the exact function of the narrative voice at any point in time, e.g., (5). As has been shown by Wood (2010) and Wood & Appel (2014), this register is likely to make up a large proportion of school EFL textbooks and since textbook metalanguage is a major source of linguistic input for EFL learners (Kim & Hall 2002), it seemed essential to include it in this comprehensive exploration of Textbook English.

<footnote n="14">¹⁴ Throughout this thesis, references to corpus excerpts are provided in angle brackets. The corpus abbreviation is followed by a colon and then a file identifier. In the case of the TEC, each corpus file corresponds to a textbook, hence the name of the textbook volume is printed.</footnote>
(5) Plymouth, my hometown
a) In the film, the girl shows us her hometown. Watch the film. What did she show us? Choose A or B.
b) Watch the film again.
What other things and places can you see in the film?
Make a list. <TEC: Access 1>

The register category ‘Personal correspondence’ includes diary and blog entries, personal e-mails and letters, e.g., (6). Since formal letters and e-mails may serve somewhat different communicative purpose, they were originally allocated to a separate category. However, as lower secondary school textbooks contain very few such texts, they were later relegated to the category ‘Others’ as part of the second annotation cycle.

(6) Ally McKoene > WestHigh Bros
December 1 near University Heights, IA via mobile
Your best feature is definitely your kindness and I'm sure everyone else agrees! You have tons of kindness in your heart and your compliments can light up anyone's face. You guys are some of the kindest people I've met and I'm so glad that you guys do what you do. Your compliments can make anyone's day :) keep it up!
Like - Comment
nh [OCR error: Facebook-style thumbs up symbol] West High Bros likes
this. <TEC: New Mission 2>

Though not included in the original annotation scheme, the first cycle of the categorisation process revealed that songs, poems and rhymes feature heavily in lower secondary school EFL textbooks, thus justifying the need for a separate ‘Poetry’ register category, e.g.:

(7) School friends
Welcome to my school!
Welcome to my school!
Come in, and be cool!
Good morning, you can all sit down!
I'm Mister Parker
Yes, I'm your teacher
Good morning, Good morning Sir!
Can you repeat? I think I don't know I don't understand... Can I open the window?
Can you come to the board? Can you write the date? Yes sir! Yes sir!
It's a piece of cake!
Welcome to my school!
Let's all be cheerful!
Take your pens and write this down I'm Mister Parker!
Listen to your teacher!
Yes Sir! No Sir! Thank you Sir! <TEC: Join the Team 6>

Finally, the register tag “Individual words or sentences” was used to label example words, phrases or sentences designed to illustrate particular lexico-grammatical phenomena, as well as the contextless words, phrases and sentences featured in
exercises, e.g., (8). Note that glossaries found at the back of many coursebooks were
not included in the TEC.

(8)

a) You are in a home, not a hotel!
b) Questions are better than mistakes.
c) It's important to be polite.
d) Your host family will get worried.
e) It can be very tasty!
f) Maybe there is a queue outside! <TEC: Green Line 2>

In addition, the manual annotation process revealed that the textbook series differ considerably in their use of students’ L1/school language to provide instructions and explain grammatical points. As a result, it was thought worthwhile to keep track of the use of a language other than English across the various textbooks series and different proficiency levels. All extended passages in languages other than English were thus annotated with the register tag ‘Foreign’, e.g., (9). Note that, in practice, these passages are almost exclusively either in French or in German since the Spanish subcorpus of the TEC only includes series from Anglo-Saxon publishers with the international market in mind (see 3.3.1.1).

(9)
Regarde la légende du document. a. Identifie la date. b. Cherche des informations à propos de l'artiste sur internet. B c. Lis le titre, regarde le tableau et devine le sens du mot shiner. <TEC: Hi There 3e>

In the following sections and chapters, the register subcorpora of the TEC are frequently referred to as separate entities. To differentiate the general reference to typical texts of a particular register found in textbooks from, specifically, a register subcorpus of the TEC, the word ‘Textbook’ and the first word of the register category are capitalised in Table 6 are used to denote the latter, i.e., ‘Textbook Conversation’ and ‘Textbook Personal’ refer to the Conversation and the Personal correspondence subcorpora of the TEC, respectively.

**3.3.2 The reference corpora**

As has been noted in 2.3, when analysing the frequency, use and function of individual lexico-grammatical features of Textbook English, a realistic reference benchmark is of the upmost importance. It has been argued that target learner language is, for instance, unlikely to resemble professional journalistic writing, and that such comparisons are thus unhelpful indicators when evaluating the lexico-grammatical content of Textbook English. It is also an argument advanced by Harwood (2005), in his criticism of studies that compare ESL textbooks to corpora. Striking a similar tone, Miller (2011: 34) rightly argues that “we must carefully consider measures (e.g., comparison corpora) upon which we are gauging our evaluation so that conclusions drawn are indeed fair and useful”. Thus, it is not necessarily meaningful to compare learner language to professional native-speaker writers and radio presenters, using such general corpora as the British National Corpus (Burnard 2007), since this is not what secondary school students are expected to aspire to. On the basis of this claim,
Tono (2004) somewhat counterintuitively argues that Textbook English itself may be a useful benchmark to compare learner language to. However, if we are concerned with the authenticity and relevance of the language input that EFL learners receive in the EFL classroom, this approach is evidently cyclical. It certainly runs the risk of learners achieving only “textbook proficiency” (to borrow a term from Dörnyei, Durow & Zahran 2004: 87) rather than the language competences required beyond the EFL classroom.

That said, setting out to compile a more realistic target learner language reference corpus is not without its issues. In the field of tertiary-level learner English, however, such a project has already been undertaken. Indeed, when Biber et al. (2002; 2004) embarked on the TOEFL 2000 Spoken and Writing Academic Language Project (T2K-SWAL), their initial motivation grew out of the need to create “an external standard to evaluate the representativeness of ESL/EFL materials” (Biber 2006: 20). The T2K-SWAL corpus emerged from this large-scale project. In an initial phase, it served as a basis to identify salient lexico-grammatical features and patterns used across a range of university registers and academic disciplines (for a detailed report on the project design, corpus compilation and its subsequent linguistic analysis, see Biber et al. 2004). Drawing on these results, several diagnostic tools were developed to evaluate the extent to which a text can be considered representative of its target reference register.

The difficulty of finding an appropriate benchmark corpus for school EFL textbooks is compounded by the fact that, as shown in 3.3.1.4, textbooks comprise several, quite distinct registers, as well as text passages that consist of individual, contextless phrases and sentences of no identifiable register category. Instead of attempting to compile a single reference corpus for the TEC, it was decided to focus on three of the major registers identified in school EFL textbooks: Conversation, Fiction and Informative texts. The choice of the Spoken BNC2014 as the reference corpus for the conversation transcripts and conversation-like dialogues featured in the TEC is justified in 3.3.2.1. Section 3.3.2.2 explains how a corpus of modern fiction literature aimed at children, teenagers and young adults was compiled to match the Fiction subcorpus of the TEC. Finally, Section 3.3.2.3 outlines the rationale and design of the Informative Texts for Teen Corpus, compiled from web data as a reference corpus for the Informative texts featured in the EFL textbooks of the TEC.

### 3.3.2.1 The Spoken BNC2014

A number of earlier Textbook English studies that focused on the spoken or spoken-like passages of EFL textbooks relied on the demographically-sampled section of the British National Corpus 1994 (hereafter BNC1994; 4.2 million words of transcribed conversation recorded in the early 1990s in the UK) as a reference corpus for comparisons between textbook vs. authentic language use (e.g., Römer 2005). In the
present study, Textbook Conversation is compared to the equivalent component of
the latest version of the British National Corpus, the Spoken BNC2014 (Love et al.
2017; 2018). The new Spoken BNC2014 is an 11.4-million-word corpus of
orthographically transcribed conversations among L1 speakers in the UK (covering a
range of self-reported regional dialects). The recordings were made by the speakers
themselves using their own smartphones between 2012 and 2016 (Love, Hawtin &
Hardie 2018: 4–5). In total, the corpus features 668 speakers in a total of 1,251
recordings (Love, Hawtin & Hardie 2018: 1).

The Spoken BNC2014 was chosen because, to date, it is the largest publicly accessible corpus of contemporary spoken English and one of the very few to reflect unscripted, informal conversation on everyday topics, which was identified as a key register in Textbook English (see 3.3.1.4). In choosing the Spoken BNC2014 as a reference corpus for Textbook Conversation, two additional choices were made: the choice of a native English variety as a reference for Textbook Conversation on the one hand, and the choice of British English over US-American or any other L1 English variety, on the other. The thoughts and reflections that motivated these two choices are explained in the following brief excursus.

On the use of L1 norms in English language teaching

Traditionally, EFL instruction in Europe has largely relied on British English norms
(see, e.g., Bieswanger 2012; Forsberg, Mohr & Jansen 2019; Gilquin 2018). However,
over the past few decades, “the whole notion of *nativeness* has become murky, if not
downright controversial” (Moyer 2013: 91, emphasis original). In ESL contexts, the
hegemonic, colonial implications that lie at the heart of many native vs. non-native
distinctions are no longer tenable. But even in EFL instructional contexts, such as
those found in most of continental Europe, the relevance of native speaker models
has increasingly been questioned as a result of the ever-growing use of English as a
*lingua franca* (ELF) and English as an International Language (EIL) in the business
world and beyond (e.g., Jenkins 1998; 2000; 2003; Gnutzmann 1999; Gnutzmann &
Intemann 2008; Prodromou 1992). If 21st century English teachers aim to follow a
communicative language teaching approach, they will evidently need to equip their
students with the (socio-)linguistic and pragmatic knowledge to communicate with
both native and non-native speakers of English from various regional and socio-
cultural backgrounds (Bieswanger 2012: 27). This is why many have called for a
definition of “authentic” foreign language exposure that recognises “the reality of
language use which learners will encounter outside and after their course[s]”
(Tomlinson 2013b: 476). In the case of EFL teaching in continental European schools,
this post-instruction context will likely involve interacting with native and non-native
speakers of English in both professional and personal ELF contexts. Thus, there are
many convincing arguments for deciding against the exclusive reliance on native-
speaker norms in EFL instruction.
Correspondingly, Mauranen (e.g., 2003; 2004a; 2004b; 2010) advocates for the use of ELF reference corpora for pedagogic purposes (such as the English as a Lingua Franca in Academic Settings Corpus [ELFA]) which she claims contains “good international English spoken in academic and professional contexts” (Mauranen 2004a: 207). However, compiling such a reference corpus is not without its issues. It goes without saying that selecting “good speakers” (see also Prodromou 2003 on the notion of “successful users of English”) to include in such a corpus will inevitably involve some subjective and/or normative judgements which will, themselves, be based on the researchers’ own (naturally biased) norms and standards (cf. Mauranen 2004a: 207). This problem is, of course, not limited to the compilation of non-native language corpora. It can easily be argued that many native speakers of English are, in fact, not particularly eloquent speakers (cf. McCarthy & Carter 2001: 339). And when it comes to written registers, even fewer L1 users are expert writers of the kinds of professionally written texts featured in most general L1 corpora such as the BNCs or COCA. Suffice to say that the idea of capturing the language of proficient or expert language users regardless of their native-speaker status is – no matter how theoretically or pedagogically meaningfully – rather difficult in practice.

Whilst recognising the need to raise awareness of and embrace the diversity of plural native, non-native, standard, and non-standard English varieties in use around the globe, some English education scholars argue that EFL teachers and their students nevertheless need “a standard for pedagogical consistency” (Moyer 2013: 92). Thus, although native speaker norms are no longer explicitly mentioned in European secondary school EFL curricula (see 1.2), they nevertheless remain the most practical and reliable way of evaluating whether students are exposed to and themselves produce “authentic” and “correct” English – to quote two adjectives still frequently found in secondary school EFL curricula (see 1.2). On the other hand, staunch advocates of communicative foreign language teaching approaches argue that such a strict understanding of “authentic” and “correct” language use writes off too many unidiomatic learner usage cases as “inauthentic” and “incorrect” when they are, in fact, frequent in ELF contexts and do not hinder communication among native or non-native speakers of English. Mauranen (2004a: 208) illustrates this with an example from Altenberg and Granger’s (2001) investigation of L2 speakers’ use of collocations with the verb MAKE: she concludes that many, when compared to L1 use, so-called “collocation errors” are actually irrelevant for daily communicative needs and should therefore not be highlighted as deviant.

At the opposite end of the spectrum, Siepmann et al. (2011: 4) remind critics of native speaker norms in (advanced) foreign language teaching of “the age-old insight that the lower you set your sights, the less you will ultimately achieve”. Returning to the example of collocates of MAKE mentioned above, they convincingly argue:
the word combination ‘make a claim’ could theoretically mean ‘invent a claim’, but there is a common-sense convention which assigns to it the meaning ‘utter an assertion’. There is, of course, nothing that prevents foreign-born writers [in the sense of: L2 English users] from using ‘make a claim’ creatively to mean ‘invent a claim’; the snag is that their (unidiomatic) use of the word combination is certain to be misinterpreted by both native and non-native speakers of English (Siepmann et al. 2011: 4).

Going further, Siepmann et al. (2011: 4) warn that “[o]nce you start turning a blind eye to [standard L1 norm infringements], it is difficult to say where to draw the line” – an argument that echoes that of the difficulty of reliably distinguishing between (very) proficient and so-called “non-proficient” speakers of English (regardless of their native-speaker status). Gilmore (2007: 106) adds that taking the production of even highly proficient (however this may be defined) L2 speakers as the reference norm runs “the risk of providing learners with ‘dumbed down’ models of English which, although perhaps meeting their transactional needs, fail to illustrate the true expressive potential of the language”.

A further argument in favour of L1 norms can be found in attitudinal research, which has repeatedly shown that both EFL teachers and, crucially, EFL learners still largely aim for native-speaker norms in spite of the generalisation of ELF/EIL (e.g., Edwards 2016; Forsberg, Mohr & Jansen 2019; Mohr, Jansen & Forsberg 2019; Scales et al. 2006; Timmis 2003). Even when their foreseen use of English is more likely to be with other non-native speakers in international ELF contexts, English L2 learners nevertheless claim to aim for norms aligned with those of Inner Circle English varieties, i.e., from regions with largely monolingual English-speaking populations (Kachru & Smith 2008). Of course, there is no denying that students’ opinions will be shaped by all kinds of societal pressures including their teachers’ (perceived or presumed) preferences. Nonetheless, the results of such attitudinal research should be taken into consideration if we are to attempt to break away from the customary paternalistic approach that tends to cast aside the opinions and wishes of learners as irrelevant (Wain 1992: 24).

In addition, some ELF agendas (as laid out, for instance, in Jenkins 2000; Seidlhofer 2001) risk dissociating learning to communicate in English from the (arguably also highly relevant) sociocultural contexts of the language – a critical aspect repeatedly highlighted as “intercultural (communicative) competence” in the EFL curricula of the educational authorities of France, Germany and, to a lesser extent, Spain (see, e.g., Conseil supérieur des programmes 2015; Consejería de Educación, Juventud y Deporte de Madrid 2015; Kultusministerkonferenz 2003; Kultusministerkonferenz 2012), as well as the CEFR (Council of Europe 2001; 2020) on which European national and regional school curricula are increasingly based on.

Bearing these three factors in mind – the pedagogical need for (at least some) consistency, respecting learners’ wishes (though there is no doubt that these are
influenced by those of their peers, teachers, parents and society as a whole) and the
curricular requirements to also teach English for intercultural competence, in addition
to the pragmatic considerations mentioned above (in particular: what constitutes a
"good/proficient" ELF speaker?), no attempt was made to create ELF reference
corpora for this project. This is not to say that students' language production cannot
or should never be assessed against an ELF norm (however hard that may be to
define!). Indeed, the present study of Textbook English focuses on the evaluation of
students' language *input* rather than *output* and this input is expected to be largely
shaped by 'standard' L1 usage.

Given that the three educational systems represented in the TEC are situated in
Europe, the choice of British English over the other dominating L1 English variety
in EFL instructional contexts – American English – seemed most natural. As
mentioned at the beginning of this section, British English has been, and continues
to be, the most commonly used target English variety in Europe, even though the
relevant national/regional curricula no longer explicitly refer to a single British
English norm (see 1.2). In the Netherlands, a large survey concluded that British
English remains the English model of choice for over half of the Dutch population,
whilst just 15% claimed to aim for an American English norm (Edwards 2016: 81).
Similarly, the results of an attitudinal study conducted in Spain suggest that learners
prefer Standard British English (RP) rather than American English and aim to
emulate this variety themselves (Carrie 2017). In practice, all the textbook series
featured in the TEC focus on England, both in terms of language use (most
noticeably, of course, in the pronunciation in the audio and video materials – though
this is not the focus of this thesis) and cultural contextualisation. That said,
individual activities and textbook units do attempt to feature other L1 speaker
varieties, most notably US-American<fnref n="15" />, Australian, Irish, and South African Englishes
(see Scheiwe in preparation on how realistic the portrayals of such accents, usually
produced by British actors, are in German textbooks). Across all the textbooks of the
TEC, however, British English is clearly the target norm since it is deviations from
British English that are indicated; for example, in most textbook glossaries, the terms
*movie* and *cellphone* are followed by "American English", "AmE" or similar.

As its name suggests, the Spoken BNC2014, which was chosen as the reference corpus
for Textbook Conversation, is most representative of British English – though it
covers a range of regional dialects. The corpus features 566 English L1 speakers from
all over the UK, 17 from outside the UK, and this optional speaker metadata is
unavailable for 88 speakers (who, together, account for about 10% of the total word
count) (Love, Hawtin & Hardie 2018: 24). The remaining two reference corpora,
Youth Fiction and Info Teens, are also biased towards British English but

<footnote n="15">¹⁵ In fact, in the G8 [*Gymnasium* in eight years] system in Germany, Year 8 is devoted to US-American English and culture.</footnote>
considerably less so than the reference corpus for spoken English. Thus, 55% of the
novels of the Youth Fiction corpus are by British authors, 31% by US-American
authors, and the remaining 14% by authors from eleven different countries. The
reference corpus for the informative texts of the TEC mostly contains texts of
unknown authorship. However, the web domains from which they were sourced are
principally from the UK and the USA, with a smaller percentage of texts from
Australia and New Zealand. Hence, both these reference corpora can also be said to
largely represent L1 English usage. Details of the composition of these corpora can be
found in 3.3.2.2 and 3.3.2.3.

Processing of the Spoken BNC2014

The Spoken BNC2014 is richly annotated with detailed metadata on the speakers and
the context of each conversation. In the untagged XML version of the corpus, which
served as the basis for the preparation of the version used in the present thesis, the
metadata is listed in the files' headers. The analyses carried out in the context of this
study do not take this metadata into consideration; hence the headers were removed.
The corpus also includes numerous other metatags, e.g., for paralinguistic sounds,
pauses and overlaps. These were also removed. Table 7 summarises the regular
expressions featured in the R script which was used to pre-process the untagged XML
files of the Spoken BNC2014 (see also *Online Appendix 3.3*). Many of these pre-
processing steps were necessary because the data contributed to the Spoken BNC2014
has been fully anonymised and therefore contains many tags of anonymised words
and phrases. These tags were replaced with placeholders designed to ensure that the
POS-tagger and dependency parser used to further process the corpus would correctly
label them for word class and function. In addition, truncated words, which are rarely
correctly identified by lemmatisers and POS-taggers, were removed.

As opposed to the BNC1994, the transcription scheme of the Spoken BNC2014 makes
minimal use of punctuation and in fact only allows for question marks (Love, Hawtin
& Hardie 2018: 37–38). Since automatic taggers and parsers are usually trained with
punctuated texts, placeholder full stops were added at utterance boundaries that did
not end in a question mark in order to reduce the potential for tagging errors resulting
from a lack of punctuation. However, it is worth noting that these full stops markers
were not used in any further linguistic analyses. Hereafter, the text files generated
after these replacements are referred to as the “John & Jill version of the Spoken
BNC2014 corpus” (see ‘Replacement’ column in Table 7 as to why this name was
chosen).
<table><caption>Table 7: Summary of the regular expressions (regex) used to process the Spoken BNC2014 (see Online Appendix 3.3 for full script)</caption><thead><tr><th>Description of tag</th><th>Search regex</th><th>Replacement</th></tr></thead><tbody><tr><td>Header with full metadata</td><td>&lt;header&gt;.*&lt;/header&gt;</td><td>[nothing]</td></tr><tr><td>Anonymised male name</td><td>&lt;anon type=\"name\" nameType=\"m\"&gt;</td><td>John</td></tr><tr><td>Anonymised female name</td><td>&lt;anon type=\"name\" nameType=\"f\"&gt;</td><td>Jill</td></tr><tr><td>Anonymised neutral name</td><td>&lt;anon type=\"name\" nameType=\"n\"&gt;</td><td>Sam</td></tr><tr><td>Anonymised place</td><td>&lt;anon type=\"place\"&gt;</td><td>IVYBRIDGE</td></tr><tr><td>Anonymised telephone number</td><td>&lt;anon type=\"telephoneNumber\"&gt;</td><td>0123456789</td></tr><tr><td>Anonymised address</td><td>&lt;anon type=\"address\"&gt;</td><td>ADDRESS</td></tr><tr><td>Anonymised e-mail address</td><td>&lt;anon type=\"email\"&gt;</td><td>anonemail@email.com</td></tr><tr><td>End of utterance not immediately preceded by a question mark</td><td>(?<!\\?)&lt;/u&gt;</td><td>.</td></tr><tr><td>Truncated word</td><td>&lt;trunc&gt;.{0,12}&lt;/trunc&gt;</td><td>[nothing]</td></tr><tr><td>Anonymised financial details</td><td>&lt;anon type=\"financialDetails\"&gt;</td><td>FINANCIAL DETAILS</td></tr><tr><td>Anonymised social media name</td><td>&lt;anon type=\"socialMediaName\"&gt;</td><td>@SAM</td></tr><tr><td>Anonymised data of birth</td><td>anon type=\"dateOfBirth\"&gt;</td><td>DOB</td></tr><tr><td>Other anonymised personal information</td><td>&lt;anon type=\"miscPersonalInfo\"&gt;</td><td>PERSONAL INFORMATION</td></tr><tr><td>All other remaining tags</td><td>&lt;.*?&gt;</td><td>[nothing]</td></tr></tbody></table>

### 3.3.2.2 The Youth Fiction corpus

Off-the-shelf corpora of English fiction exist, e.g., the English subcorpus of the PhraseoRom Corpus (Novakova & Siepmann 2020) and the US Novel Corpus (Chicago Text Lab 2020). In addition, most general English corpora include a literature subcorpus of extracts of novels, e.g., the British National Corpora (1994 and 2014). However, for a meaningful comparison of Textbook Fiction with authentic fiction texts to be possible, it was decided that both the communication purposes and the intended target audiences of the texts ought to be matched. Thompson & Sealey (2007) report a number of significant differences between adult and children fiction in the frequencies and contextual uses of the most frequent types, parts-of-speech, lexical verbs, 4-grams and POS-grams. Though less surprising, their additional conclusion that the most frequently occurring semantic categories found in adult and children fiction differ considerably is no less relevant to the present study. Children's literature appears to have a much stronger focus on animals and other living creatures, food, plants and communication (due to the prevalence of direct speech and speech acts). Thus, Thompson & Sealey (2007) highlight the strikingly different representations of world and self in children and adult fiction. Their results confirmed
the need for creating a dedicated Youth Fiction Corpus to be used as a comparison
corpus for the Fiction subcorpus of the TEC: Textbook Fiction.<fnref n="16" />

Since the TEC consists of textbooks intended for ca. 11 to 16 years-olds, the aim was
to compile a balanced and representative corpus of English-language fiction books
suitable for children, teenagers and young adults. Unlike films, books are not usually
explicitly labelled as being suitable or targeted at particular age groups; it was
therefore necessary to find alternative selection criteria. In an attempt to achieve
sample representativeness and balance, the books to be included in the reference
Youth Fiction corpus were selected from the following seven lists. The lists were
chosen to represent the choices made by respected British and US-American media,
as well as those made by the wider internet community, as represented in the two
dynamic user-contributed lists from goodreads.com and NPR.org.

* The List: 100 Best Children's Books of All Time (published on 8 January 2015)
<http://time.com/100-best-childrens-books/>

* LIST: The 100 Best Young Adult Books of All Time (published on 8 January 2015)
<http://time.com/100-best-young-adult-books/>

* The 100 best children's books of all time (published on 19 July 2018, accessed on 13 September 2018) <https://www.telegraph.co.uk/books/childrens-books/100-best-childrens-books-time>

* The Guardian Children's Fiction Prize Winners (from 2000–2016, accessed on 20 January 2019)
<https://en.wikipedia.org/wiki/Guardian_Children%27s_Fiction_Prize>

* The School Reading List - Suggested reading books for primary and secondary aged children in the UK (Years 7 and 8) (by Jan Tolkien, last updated on 20 January 2019 when accessed on 31 January 2019)
<https://schoolreadinglist.co.uk/category/reading-lists-for-ks3-pupils/>

* What Book Got You Hooked? (user-contributed list, accessed on 30 January 2018 with 9,003 contributors at the time)
<https://www.goodreads.com/list/show/651.What_Book_Got_You_Hooked>

* Your Favorites: 100 Best-Ever Teen Novels (user-contributed list with 75,220 contributors, published on 7 August 2012)
<https://www.npr.org/2012/08/07/157795366/your-favorites-100-best-ever-teen-novels?t=1539242729260>

One major drawback of this approach is undoubtedly the highly subjective nature of
such best-of lists; yet, in the absence of book sale numbers, they provided a useful
starting point. Picture books clearly aimed at children younger than 10 years were
excluded, as were translations of books originally in languages other than English.

<footnote n="16">¹⁶ The use of the relevant subcorpora of the Oxford Children's Corpus (Banerji et al. 2013; Wild, Kilgarriff & Tugwell 2013) was also considered. However, the compilers of the corpus did not respond to the author's request for access for the purposes of this project.</footnote>
The final selection from the lists was opportunist and entirely based on the immediate availability of the books<fnref n="17" /> in digital format (Epub or PDF). In part, however, it can be assumed that the availability of the books in digital format is generally a testimony to their popularity. The digital books were subsequently converted to UTF-8 text using the same OCR software as for the TEC and automatically cleaned of unwanted characters and systematic OCR errors using regular expressions, as described in 3.3.1.2.

In total, 300 books were collected, amounting to over 20 million words. The majority of the novels in the corpus were written by British authors (166 books), a large proportion by US-American authors (92 books) and the rest from eleven other countries including Australia, India, and Ireland. The full list of works included in the corpus may be found in the *Online Appendix 3.4*. Due to the nature of such best-of lists, the median initial publication date is 1994 and just 26% of the books in the corpus were first published between 2007 and 2017.

The Youth Fiction corpus consists of four random samples of approximately 5,000 words (splitting was performed at sentence boundaries, hence the slightly varying word counts) extracted from each of the 300 books collected and processed for the corpus, except for three very short books, which were only sampled once in full. With a total of 1,191 Youth Fiction texts, this procedure resulted in a number of texts comparable to that of the Spoken BNC2014.

### 3.3.2.3 The Informative Texts for Teens Corpus (Info Teens)

Whilst corpora of children's or young adults' fiction do exist, it was clear from the outset that the reference corpus for the Informative subcorpus of the TEC, Textbook Informative, would have to be compiled specifically for this project. The aim was to find informative texts that are targeted at English-speaking teenagers of the kinds of topics typically featured in school EFL textbooks. To this end, a list of 20 quality informative websites for teenagers from various English-speaking countries was compiled.

The web scraping process was facilitated by Sketch Engine's (Kilgarriff et al. 2014) corpus-building tool, which relies on the WebBootCaT (Baroni et al. 2006), to create a text corpus from the list of selected websites. Sketch Engine automatically downloads the text materials from all the relevant webpages, removes non-text elements, boilerplates, and duplicates to produce (relatively) clean text files. Of the 20 websites, four were later discarded because they did not permit text scraping.

<footnote n="17">¹⁷ Many thanks to the PhraseoRom team (in particular Johan Didier and Susanne Dyka) who kindly provided text versions of the titles already included in the PhraseoRom corpus.</footnote>
Post-duplication removal, Sketch Engine retrieved and cleaned 17,014 files from the remaining 16 web domains. As shown on Fig. 3, these texts were not distributed evenly among the websites. The corpus as originally compiled by Sketch Engine was then downloaded as a single XML file without part-of-speech tagging and lemmas for further off-line processing.

<figure><img src="image_5.png" /><figcaption>Fig. 3: Composition of the web corpus originally retrieved by Sketch Engine</figcaption></figure>

Off-line processing was performed in python 3 (Van Rossum & Drake 2009). Regular expressions were employed to remove erroneous XML tags, non-UTF-8 characters, indices and tables of contents, any remaining boilerplates and adverts, texts containing language puzzles (e.g., crosswords), marking schemes and past exam papers (only in the texts from revisionworld.com), and user comments (especially in the texts extracted from dogonews.com, teenvogue.com and teen.wng.org). Then, the large XML file (totalling nearly one million lines!) was split into the individual texts of the corpus and saved as separate plain text files with filenames that incorporate relevant metadata on the web domain and title of the webpage, as available in the corresponding XML tags. This was achieved with the {beautifysoup} library (Richardson 2015), which was found to cope relatively well with large, malformed XML files such as those compiled by Sketch Engine.

This procedure led to the creation of 10,104 individual text files, of which 4,895 were under 400 words and were therefore discarded since they were considered too short for any meaningful text-based analyses (see also 6.2.2). To achieve a more balanced corpus, a stratified sampling approach was followed: 100 texts from each web domain were randomly selected for inclusion in the corpus. For two domains, fewer than 100 texts longer than 400 words had been retrieved; for these, the full domain datasets were retained. The final selection thus consists of 1,414 text files (see Table 8) – a number comparable to both the total number of conversation files in the Spoken BNC2014 (see 3.3.2.1) and text samples in the Youth Fiction Corpus (see 3.3.2.2).
<table><caption>Table 8: Composition of the Informative Texts for Teen Corpus</caption><thead><tr><td>Domain name</td><td>Nb. texts</td><td>Nb. words</td></tr></thead><tbody><tr><td>bbc.co.uk/history</td><td>100</td><td>74,722</td></tr><tr><td>dogonews.com</td><td>100</td><td>60,762</td></tr><tr><td>ducksters.com</td><td>100</td><td>67,894</td></tr><tr><td>encyclopedia.kids.net.au</td><td>100</td><td>74,566</td></tr><tr><td>factmonster.com</td><td>100</td><td>60,395</td></tr><tr><td>historyforkids.net</td><td>100</td><td>71,955</td></tr><tr><td>quatr.us</td><td>100</td><td>62,254</td></tr><tr><td>revisionworld.com (GCSE only)</td><td>100</td><td>74,301</td></tr><tr><td>sciencekids.co.nz</td><td>100</td><td>57,097</td></tr><tr><td>sciencenewsforstudents.org</td><td>100</td><td>82,258</td></tr><tr><td>teen.wng.org</td><td>85</td><td>45,515</td></tr><tr><td>teenkidsnews.com</td><td>100</td><td>81,765</td></tr><tr><td>teenvogue.com</td><td>100</td><td>82,117</td></tr><tr><td>tweentribune.com</td><td>29</td><td>26,166</td></tr><tr><td>whyfiles.org</td><td>100</td><td>85,492</td></tr><tr><td>Total</td><td>1,414</td><td>1,007,259</td></tr></tbody></table>
<figure><img src="image_6.png" /></figure>
# 4 Exploring the progressive in Textbook English

Are you swimming in the sea?
Yes, I am.
Are you playing golf in the jungle?
No, I'm not.
<TEC: Achievers Pre-intermediate>

## 4.1 Rationale and aims

This chapter<fnref n="18" /> aims to investigate the representation of the progressive in two textbook registers: Conversation and Fiction. To this end, 3,940 concordance lines featuring verbs in the progressive from the Conversation and Fiction subcorpora of the Textbook English Corpus (TEC) are compared to the same number of progressive concordance lines from the two reference corpora of the same registers: the Spoken BNC2014 (see 3.3.2.1) and the Youth Fiction Corpus (see 3.3.2.2). First, a method is developed to approximate the frequency of the progressive construction in the two Textbook English subcorpora as compared to the two reference corpora. The second section focuses on contextual, morphosyntactic aspects of progressive constructions, such as contraction and negation, and their functions, including time reference and repeatedness. Finally, a third section explores the lexical and semantic associations of the progressive using collostructional methods and correspondence analysis.

### 4.1.1 EFL learners’ use of the progressive

Despite its simple form, BE + V<sub>ing</sub>, the English progressive aspect is fraught with pitfalls which EFL teachers will no doubt be familiar with. It has been claimed that it “constitutes one of the most basic and ubiquitous problems facing language teachers” (Williams 2002: 18, cited in Römer 2005: 172). Indeed, although the progressive is usually introduced in the first term of EFL instruction, the progressive vs. non-progressive alternation remains a stumbling block for many learners of English, even at the higher levels of proficiency (e.g., Hahn, Reich & Schmied 2000; Mindt 1997b; Rautionaho, Deshors & Meriläinen 2018; Westergren Axelsson & Hahn 2001; Wulff & Römer 2009). For many EFL learners, the progressive is tricky in that

<footnote n="18">¹⁸ Preliminary results of the analyses reported in this chapter were presented at the ICAME40 in Neuchâtel, Switzerland. The author is grateful for the audience’s valuable comments and suggestions which contributed to improving this chapter. The conference presentation also led to the publication of selected results from this chapter in: Le Foll, Elen. forthcoming. “I’m putting some salt in my sandwich.” The use of the progressive in EFL textbook conversation. In Susanne Flach & Martin Hilpert (eds.), *Broadening the spectrum of corpus linguistics: New approaches to variability and change*. Amsterdam: John Benjamins.</footnote>
it may not have a direct equivalent construction in their L1 (Römer 2005: 2). An added difficulty for EFL learners is undoubtedly that the construction fulfils a broad range of functions (see, e.g., Binnick 2020; Gut & Fuchs 2013: 245–246; Mair 2012). Moreover, numerous studies have shown that its diverse range of functions has seen considerable changes over the past decades (e.g., Mair & Hundt 1995; Smith 2005; Smitterberg 2005).

The progressive aspect has repeatedly been shown to be frequently over- and misused by EFL learners, especially those acquiring English in instructional settings. For instance, learner corpus studies have concluded that L1 Finnish, Swedish and German learners generally overuse the progressive as compared to English native (ENL) speakers (Virtanen 1997; Hundt & Vogel 2011; Westergren Axelsson & Hahn 2001; Römer 2005). More recent, fine-grained learner English studies have revealed how EFL learners’ use of the progressive differs from ESL and ENL use in terms of frequency, range of semantic functions, choice of verb lemmas and genre-awareness, among other aspects (e.g., Fuchs & Werner 2018; Meriläinen 2018; Rautionaho & Deshors 2018; for comprehensive overviews of earlier studies, see also Römer 2005: chap. 3; Rautionaho 2014: chaps. 1–2).

We may ask whether these differences in the ENL, ESL and EFL speakers’ use of the progressive can, at least partly, be attributed to EFL teaching practices. Concerning EFL learners’ reported overuse of the progressive, Biber & Reppen (2002b: 203) claim that “[o]ne of the most widely held intuitions about language use among TESL professionals is the belief that progressive aspect is the unmarked choice in conversation.” Whilst progressive constructions have been shown to be comparatively more frequent in spoken than in written English, corpus-informed grammars report that progressive are, in fact, rare across all varieties and registers, accounting for between some three and ten percent of all verb phrases (Biber et al. 1999: 461–475; Leech, Rayson & Wilson 2001: 124–125). At the same time, the use of the progressive in English appears to be generally on the rise (e.g., Hundt 2004; Mair & Hundt 1995; Smith 2005; Smith & Rayson 2007; see also Rautionaho 2014: 62–64 for a more recent overview of studies on the evolution of the frequency of the progressive). Hundt & Vogel (2011) concluded that both language contact and exo-normative influences are likely to contribute to the attested differences in progressive usage in ENL, ESL and EFL varieties. They suggest that instructed EFL learners are foremost confronted with the “grammaticality issues and semantic restrictions of the progressive, making them more likely to overuse the prototype of the construction and less likely to ‘stretch’ the progressive to new contexts, such as combinations with certain stative verbs or new aspectual uses” (Hundt & Vogel 2011: 160). All of this raises the question as to whether EFL grammars and textbooks have evolved to reflect current usage.
4.1.2 The progressive in EFL textbooks

It is beyond the scope of this chapter to provide even a brief summary of the extensive
research that has hitherto been carried out on the patterns of use and functions of
the progressive in various ENL, ESL and EFL varieties (see Römer [2005: Ch. 3] and
Rautionaho [2014: Ch. 1 and 2] for comprehensive overviews). Instead, this section
will focus on previous research on the use of the progressive in EFL textbooks for
secondary education.

Compared to the many studies on English learners’ use of the progressive, ELT
materials have more rarely been the subject of investigation (see 2.2.2.3). Bland
(1988) and Belli (2017) examined the treatment of stative verbs in the progressive in
ESL/EFL pedagogical materials and concluded that it did not match authentic usage,
whilst Biber & Reppen (2002b: 203) reported that EFL/ESL conversation textbooks
overuse the progressive in their representations of spontaneous spoken language.
However, to date, only Römer (2005) has provided a comprehensive, corpus-based
analysis of the progressive in EFL textbooks.

As mentioned in Chapter 2, Römer (2005) examined the lexico-grammatical usage
patterns of the progressive in a corpus of spoken-like language from two series of EFL
textbooks used at lower secondary school level in Germany. She compared those to a
reference corpus of spoken British English consisting of the spoken component of the
British National Corpus (BNC1994) and the Bank of English. Some of the key
findings were presented in 2.2.2.3. They included an under-representation of negated
progressives and contracted forms of the auxiliary BE in textbooks as compared to
naturally occurring speech, but an over-representation of progressives in question
forms. Functionally, Römer (2005) observed a distinct underuse of progressives
conveying repeated actions. Indeed, 90% textbook progressives refer to a single
continuous event (Römer 2005: 6.2.2 and Römer 2010: 22–24) – a prototypical
example being:

(10) *Robert: Well, no. I'm not listening to Radio 1. It's Radio Nottingham.*
*I'm listening to my mum.* (extract from *Green Line New*, as cited in Römer
2010: 23).

Römer (2005) did not directly investigate which verb lemmas are frequently featured
in the progressive, or the semantic domains of these verbs. However, her study
frequently subdivides the analysis of specific morphosyntactic and functional features
according to individual high-frequency verbs. Thus, for instance, she reports that
present progressive tense forms with the verb lemmas ASK, GO, PLAY and STAY are
over-represented in textbook dialogues (Römer 2005: 244–246). In contrast, she shows
that these same verb lemmas are under-represented in the past progressive and
present perfect progressive forms. Similarly, Römer (2005: 250–252) notes a general
under-representation of negated progressives, in particular those featuring DO, GO,
HAVE and SAY, yet observes an overuse with ASK, GET, LISTEN and MAKE in textbook dialogues, as compared to her reference corpus of spoken British English. However, Römer's (2005) work does not attempt to tease apart the associations of individual verbs with the progressive, nor to compare these associations across textbook dialogues and real-life L1 conversations. An exception is made for phrasal/prepositional verbs in a section devoted to verb-preposition co-occurrences. Thus, in the progressive, the phrasemes AGREE WITH, DEAL WITH, RING UP and SET UP are found to be less frequent in textbook dialogues than in naturally occurring British English conversation, whereas COME BACK, GET AT, LOOK AT and WORK ON are reported to be more frequently observed in the progressive (Römer 2005: 249–250).

In conclusion to her Textbook English vs. authentic ENL English comparison, Römer (2005: 271–273) draws up a long list of the most striking disparities in the patterns of co-occurrences and function of the progressive in textbook and authentic dialogues. She concludes that school EFL “coursebooks [dialogues] provide a rather monolithic view of what progressives can express” (Römer 2005: 284). However, these results should be approached with caution since they are based on a relatively small corpus of pedagogical dialogues (108,000 words) stemming from just two series of textbooks used in German secondary schools. Effect sizes or confidence intervals are not reported for the many quantitative differences observed between textbook dialogues and natural conversation. In common with all Textbook English studies to date (see 3.1), the analyses do not enable us to tease apart which linguistic variable(s) contributed to any one reported difference in frequency of use (e.g., polarity or verb lemma, tense or time reference). Crucially, whilst the frequencies and distributions of progressive forms and functions in Römer's (2005) textbook dialogue corpus are frequently reported separately for the two textbook series that span five years of English tuition, no attempt is made to distinguish how representations of the progressive in school EFL textbooks may change as the target readers' language proficiency increases.

### 4.1.3 Aims and research questions

This chapter aims to investigate how the progressive construction is portrayed in two EFL textbook registers: Conversation and Fiction, as compared to the Spoken BNC2014 and the Youth Fiction Corpus, respectively. This differentiated look at the progressive across two different registers is crucial since the construction has been shown to be highly susceptible to mode, genre and register effects (e.g., Biber et al. 1999; Rautionaho & Deshors 2018; Smith 2005).

By investigating many of the linguistic parameters already studied by Römer (2005), it will be possible to compare the language of EFL textbooks currently in use in secondary schools in France, Germany and Spain to that of older German EFL textbooks. Following Römer (2005), the present investigation comprises both
contextual morphosyntactic features, such as tense forms, contractions and negation, and functional phenomena, such as time reference and repeatedness.

Since Römer's study did not attempt to assess the frequency of progressives, there is, to the best of the author's knowledge, currently no empirical data to back or refute the claim made by Biber & Reppen (2002b: 203) that progressives are over-represented in textbooks' representations of spoken English. This chapter therefore also investigates whether the progressive is over-represented in both the conversation and fiction texts of contemporary secondary school EFL textbooks.

In her concluding chapter on the pedagogical implications of her research, Römer (2005: 285) advocates for a shift towards a more lexical grammar of the progressive. Her analyses of verb-specific patterns reveal strong associations between progressive forms of individual verb lemmas and morphosyntactic features and functional phenomena with which they typically co-occur (Römer 2005: 285). Following a usage-based approach (see 1.4), we assume that lexical restrictions apply to grammar in general, and specifically here, to the progressive aspect. This chapter will therefore also apply a collostructional approach to investigating the lexical associations of the progressive in Textbook English. This will be achieved by comparing the results of pairs of Distinctive Collexeme Analyses (DCAs) of the lemmas associated with both progressive and non-progressive constructions in textbook dialogues and real-life conversation, on the one hand, and textbook fiction and non-textbook fiction, on the other. Although there have been successful multifactorial approaches to the study of progressives in different genres and ENL, ESL and EFL varieties (e.g., Deshors 2017; Fuchs & Gut 2015; Rautionaho, Deshors & Meriläinen 2018; Rautionaho & Deshors 2018; van Rooy 2006), so far, such models have not included verb lemma as an independent variable.<fnref n="19" /> Excluding the variable lemma from such analyses, however, is potentially problematic both from a theoretical, usage-based approach, and a pedagogical perspective. Indeed, the aspect hypothesis, which is in turn motivated by the distributional bias hypothesis, postulates that:

> First and second language learners will initially be influenced by the inherent semantic aspect of verbs or predicates in the acquisition of tense and aspect markers associated with or affixed to these verbs (Andersen & Shirai 1994: 133).

In line with the predictions of the aspect hypothesis, Bardovi-Harlig (2012) advances that EFL learners first acquire the progressive aspect in the context of activity verbs and accomplishment. They later extend the construction to repetitive (iterative and habitual) actions, before beginning to use it in futurate and, finally, stative functions. Supporting the aspect hypothesis, Rautionaho & Deshors (2018) report that, in their final regression model of progressive use by ENL, ESL and EFL speakers, only the

<footnote n="19">¹⁹ Rautionaho & Deshors (2018: 236) report that they did attempt to model lemma as a random effect variable but that such models failed to converge.</footnote>
semantic domain variable does not contribute to any higher-level interactions. They thus conclude that “regardless of varieties, written genres, tense, Aktionsart [accomplishment, achievement, process, or stative], voice, etc., the semantics of a given lexical verb systematically influences writer’s constructional choices” (Rautionaho & Deshors 2018: 238). Since lemma and semantic category are usually highly correlated, we might speculate that, had it been feasible to include them in such a model, individual verb lemmas may have played an even more important role than semantic domain in mapping the use of the progressive. In the context of this pedagogically-motivated study, it therefore seems imperative to also consider which specific verb lemmas are more strongly associated to the progressive (as opposed to the non-progressive) in EFL textbooks and real-life conversation.

Consequently, 4.3.4 zooms out from the broad contextual and functional aspects of the progressive to zoom in on the specific verb lemmas. Following the methodology proposed and tested by Deshors (2017) in an investigation of the use of the progressive in five varieties of World Englishes, it also explores the relationship between verb lemmas, the semantic domains of these verbs and the progressive construction. To this end, and following Deshors (2017), a series of Co-varying Collostructional Analyses (CovCAs) and a Correspondence Analysis (CA) are conducted. The final section takes an alternation stance on the lexical verbs featured in the progressives by comparing the results of two pairs of Distinctive Collexeme Analyses (DCAs) of the lemmas associated with both progressive and non-progressive constructions in the four (sub)corpora under study.

In sum, the present chapter seeks to address the following research questions:

1. Is the progressive over-represented in conversation and fiction texts of secondary school EFL textbooks?

2. What are the morphosyntactic contexts in which EFL learners encounter progressives in their textbooks? Are these contexts representative of those found in authentic ENL target-learner-language corpora of the same registers?

3. Which functions do progressives in Textbook English typically fulfil? Are these functions representative of the most common functions observed in the reference corpora?

4. Have the progressives featured in the dialogues of contemporary lower secondary school EFL textbooks become more representative of those typical of present-day authentic ENL conversation than was the case at the time of Römer's (2005) study?

5. Which lexical verbs and semantic domains are significantly associated with or repelled by the progressive in the conversation and fiction texts of EFL textbooks? How does this compare to naturally occurring ENL registers?

6. To what extent do the distinctive semantic domains of verbs in the progressive correlate with the registers Conversation and Fiction and with the varieties Textbook English vs. naturally occurring ENL?

7. Which of the differences noted in the use of progressives across Textbook English and the ENL reference corpora may be pedagogically well-founded? Which might
deprive learners of valuable exposure to frequent forms with particularly useful communicative functions?

## 4.2 Methodology

The corpus data analysed in this chapter were presented in Chapter 3. In the present chapter, two subcorpora of the TEC are examined: Textbook Conversation and Textbook Fiction (see 3.3.1.4). These are compared to the Spoken BNC2014 (see 3.3.2.1) and the Youth Fiction Corpus (see 3.3.2.2) respectively. The following sections focus on the methods applied in the analyses presented in this chapter. First, 4.2.1 explains how occurrences of the progressive were automatically extracted from the corpora and pre-processed. Next, 4.2.2 outlines how the extracted concordance lines were subsequently manually annotated for a number of morphosyntactic and functional variables. The statistical methods used to compare frequencies across the two corpora are discussed in 4.2.2.7, before the procedure followed to conduct the collostructional analyses of the lexical verbs attracted to or repelled by the progressive in textbook and naturally occurring English is explained in 4.2.3.

### 4.2.1 Extraction of progressive concordance lines

The aim of this extraction process was to retrieve all instances of the progressive from the two TEC subcorpora, Conversation and Fiction, and to subsequently extract, at random, the same number of progressives from each of the two reference corpora for comparison.

Many earlier corpus-based studies on the progressive did not make use of part-of-speech (POS) tagging to extract progressives, relying instead on a regular expression approach, i.e., combining a query for any token with the form <.*ing> with a labour-intensive manual disambiguation phase (Deshors 2017; Edwards 2014: 167; Rautionaho 2014: 73–74; Römer 2005: 50). Since English POS taggers are known to struggle with participle forms (see, e.g., Manning 2011), a non-POS-based query ([word=“[a-z]*ing”]) was first run on the Textbook Conversation subcorpus and the Spoken BNC2014 to check for potential issues in using Sketch Engine’s (Kilgarriff et al. 2014) POS tagging<fnref n="20" /> as a basis for a query on progressives: for each corpus, a random sample of 200 POS-tagged concordance lines with tokens ending in -ing were thus manually checked. Of these, one concordance line (11) in the Textbook Conversation sample included a present participle form inaccurately tagged as a verb base form. Because of an end-of-line hyphen in the original textbook layout, *ting* was tokenised as a separate word. This is unfortunate but a subsequent corpus query confirmed that it was a one-off case across the entire TEC.

<footnote n="20">²⁰ English TreeTagger PoS tagset with Sketch Engine modifications, see https://www.sketchengine.eu/english-treetagger-pipeline-2)</footnote>
(11) Thank you for helping me out, I'm get-**ting** a little tired of royal schedules<sup>21</sup> <TEC: Piece of Cake 3e>

Due to the unplanned and interactional nature of spontaneous conversation, the Spoken BNC2014 presents more of a challenge for POS taggers. Nonetheless, only two instances of poorly tagged present participle forms were found among the 200-concordance line sample. In (12) *being* is identified as an adjective because the expletive ending in -*ing* immediately before it is inaccurately identified as a present participle. Therefore, despite the erroneous automatic tagging, this occurrence of the progressive would still be picked up by a query relying on POS tags (albeit for the wrong lemma).

(12) I've never fucking met this girl and she was fucking **being** a dick was she actually shouting as well cos she was very quiet? <BNC2014: SAR5>

No plausible explanation for the second inaccurately tagged present participle (13) (tagged as a noun) could be found. In fact, the trigram *were you singing* occurs three times in the Spoken BNC2014 and is tagged correctly in the other two occurrences.

(13) what were you doing? were you **singing** or something? <BNC2014: SY8B>

At the same time, around a third of the concordance lines retrieved using this non-POS-based query were not present participles and were not tagged as such either. It thus becomes evident that, for a minimal drop in the recall rate, a POS-based query is likely to yield a 30% boost in precision (assuming that the concordance line samples were representative and that the POS tagging algorithm only assigns tokens ending in -ing to present participle tags). In the context of this study, it therefore seems reasonable to rely on Sketch Engine's POS tagging in order to considerably reduce the workload in the manual disambiguation phase. After some experimenting, the following Corpus Query Language (CQL) query (14) was chosen to extract progressives from the corpora stored in Sketch Engine.

(14) [tag="VB.*"] [tag! = "SENT|,"] {0,3} [tag="V.G" & word! = "gonna"]

The query first calls for all forms of the verb to BE. This includes the base form for modals with progressives, e.g., forms such as *will* + *be* + -*ing* (15), and the past participle form *been* for perfect progressives (16). Note that this CQL-based approach means that progressive constructions in which the auxiliary BE is omitted were not included in this investigation (unlike previous studies using an entirely manual filtering approach, e.g., Römer 2005, Rautionaho 2014). This is not deemed problematic since these make up less than 1% of progressive occurrences in spoken English (Römer 2005: 61).

(15) I probably can't get there till after five. Mum will be working.<TEC: Green Line New 5>

<footnote n="21"><sup>21</sup> Unless stated otherwise, all emphasis in example sentences added.</footnote>
(16) I've never **been skiing**, but I'd love to go. <TEC: Solutions Pre-intermediate>

The CQL query (14) allows for the verb to be optionally followed by one to three
tokens of any POS except for end-of-sentence punctuation marks (SENT) or commas
before the -ing verb form. The span of zero to three was chosen to allow for negation
and up to two adverbs in declarative sentences, as well as for nominal phrases as
subjects in questions, e.g., (17)–(18), and, particularly with the Spoken BNC2014 in
mind, adverbs, discourse markers, and repeated words and/or hesitations placed
between the auxiliary and the present participle, e.g., (19)–(20). Spot checks
suggested that a larger span would not have led to a substantially improved recall
rate, but clearly to a considerably lower precision rate.

(17) Are those kids really sleeping? < TEC: Access 1>

(18) yeah how's your university application going? <BNC2014: SQPN>

(19) I'm you know rapidly losing patience with him <BNC2014: S2T6>

(20) and then as far as I could tell she was **essentially like** crashing at some guy's place who like she'd just met <BNC2014: S3LE>

The word gonna was also excluded because it is mostly associated with going to +
infinitive constructions expressing future actions and events, and this construction
has been excluded from the present investigation (see 4.2.2.3).

4.2.2 Annotation

In order to reduce the manual annotation workload and thus decrease the rate of
inattention errors, the concordance lines were automatically pre-annotated for aspects
related to the forms of the progressives. This process is described in 4.2.2.1, before
turning to the development, implementation and validation of the manual annotation
scheme in the following subsections (4.2.2.2–4.2.2.6).

4.2.2.1 Automatic pre-annotation

When retrieving concordance lines using the CQL query described above (14), Sketch
Engine gives the user the option of downloading the lines in a tabular format whereby
the strings corresponding to the query results themselves are stored in one column,
whilst their immediate left and right contexts are stored in two additional separate
columns. Thus, it was possible to process only the strings retrieved by the CQL query,
i.e., the middle of each KWIC concordance line, in order to pre-code each instance of
a progressive for the following variables: a) the lemma of the verb in the progressive,
b) whether or not the progressive includes a contracted form of the auxiliary BE, c)
whether the verb phrase is negated and d) the tense form of the progressive. In the
following, the R script written to this effect is briefly described. The script itself can
be consulted in the <u>Online Appendix 4.2</u>).
The main verb lemma from each concordance line was determined by extracting the tokens with an -ing ending from the table column containing only the query results. This is easily achieved using a regular expression of the form `[:alpha:]*ing`. These tokens were then copied to a new variable and lemmatised using the R package `{textstem}` (Rinker 2018). Since the lemmatiser can then only lemmatise the verbs on the basis of these isolated present participle forms, this method is evidently not particularly accurate (e.g., *annoying* and *dying* were always lemmatised as the adjectives with the same form) but, in practice, this was not a problem because all the lines were, in any case, to be subsequently checked as part of the manual filtering and annotation phase (see 4.2.2.3).

A simple regular expression was also used to detect apostrophes since they likely point to contracted auxiliary verb forms. Similarly, the regular expression `grepl("n't/||bnot||b")` was used to capture negation in the captured progressive forms. This regular expression can detect both contracted and non-contracted negated forms and the word boundary metacharacter `||b` was used to ensure that *not* was always an individual token, rather than detected as part of a longer word such as *noting* or *another*.

Automating the detection of tense was a little more complex, but since high accuracy was not the aim here, a similar approach with a series of regular expressions could be developed. The default value for the tense variable was set to 'present progressive' and occurrences of markers of past progressives (`||bwas||b/||bwere||b/||bwasn't||b/||bweren't||b`) resulted in a past progressive coding, whereas the detection of `||bbeen||b` changed the default setting of the tense variable to 'perfect' progressives and `||bbe||b` to 'modal/infinitive' progressives. This last category, in particular, subsequently required considerable manual disambiguation since it included many concordance hits that were later excluded from this analysis of progressive verb phrases (see 4.2.2.3).

Evidently, such an approach does not pretend to yield highly reliable results. It should be stressed that the purpose of this automated pre-annotation phase was simply to reduce the total manual workload and, for this purpose, the script described above proved to be highly effective.

### 4.2.2.2 Semantic domains of progressive verbs

Following Deshors (2017), each verb occurring in the progressive was also coded according to the seven-class taxonomy of semantic domains developed by Biber et al. (1999: 360, emphasis in examples added):

* **Activity** verbs are used for events controlled by a volitional agent, e.g., BRING, GO, OPEN, RUN, TAKE.
* **Aspectual** verbs characterize the stage of progress of an event or activity, e.g., BEGIN, CONTINUE, KEEP, START, STOP.

* **Causative** verbs “indicate that some person or inanimate entity brings about a new state of affairs” e.g., CAUSE, HAVE, HELP, LET, MAKE.

* **Communication** verbs are a subcategory of activity verbs involving spoken and written communication, e.g., ASK, DESCRIBE, PROPOSE, SAY, WRITE.

* **Existence** verbs can be divided into verbs of existence or stance (e.g., BE, EXIST, LIVE, STAY, STAND) and relational verbs (e.g., APPEAR, BELONG, DEPEND, HOLD, RESEMBLE).

* **Mental** verbs and states verbs signal human experiences and fall into the categories of perception (SEE), cognition (THINK), decision (ACCEPT), mental effort or intent (AIM), and receipt of communication (READ).

* **Occurrence** verbs denote physical events that occur independently of volitional activity, e.g., BECOME, DIE, EMERGE, HAPPEN, INCREASE.

While chosen for reasons of comparability with previous research on the progressive (e.g., with Collins 2008; Deshors 2017; Edwards 2014; Rautionaho 2014; Smith 2005), this classification poses a few theoretical and practical problems. Conceptually, the main issue is that this framework does not include a distinct category for stative verbs, although this category evidently has the potential to be quite informative when investigating variation in the progressive construction (see also Deshors 2017: 266). In practice, many verbs are tricky to classify and thus the annotation process is subjective, time-consuming and, inevitably, prone to error. Deshors (2017) relied on data manually coded by Edwards (2014) as part of a larger project. Regrettably, it was not feasible to manually code over 7,000 progressive concordance lines for semantic domain in the context of this case study. Instead, a list of 345 verbs and their semantic domains mentioned in Biber (2006: 249) was merged with a list of 550 verbs belonging “fairly unambiguously to each of the seven categories” compiled by Alison Edwards (mentioned in Edwards 2014: 168fn, obtained via personal communication) which she used to pre-annotate concordance lines before manual disambiguation. Since the concordance lines were to be automatically automated for semantic domain, the category ‘other/unclear’ was added for all verbs which frequently belong to more than one semantic category. The few disagreements between Biber’s and Edwards’ lists usually concerned such polysemous verbs and, given that they cannot be reliably automatically annotated for semantic category, these were all re-classified as ‘other/unclear’. The resulting agglomerated list was then manually expanded to include all remaining verbs found in the 7,880 progressive concordance lines under study (see Online Appendix 4.3). It was then possible to automatically match every concordance line to a semantic category for its verb, though it should be noted that many high-frequency verbs being highly polysemous, 26% of concordance lines were assigned the semantic category ‘other/unclear’.
### 4.2.2.3 Manual identification of non-progressives

The manual annotation process began with the filtering of false positives, i.e., non-progressives that were unintentionally retrieved by the corpus query. Thus, a number of adjectives (e.g., *boring*, *interesting*), as well as fewer nouns (e.g., *living room*) that were erroneously tagged as present participles and occurred shortly after the verb BE were screened out. Furthermore, *BE going to* + *infinitive* constructions (21), non-finite verb phrases (22)–(24), *BE* + *present participle* (25) and unclear and/or incomplete constructions were excluded (see Smitterberg 2005; Rautionaho 2014 for similar procedures).

(21) Well, I'm **going to** prepare dinner. <TEC: Achievers A1>

(22) Let's **try shouting** 'Lady Time' <TEC: New Green Line 1>

(23) He was never **afraid of making** a fool of himself, either. <TEC: Join the Team 3>

(24) On the cover, there is a **big woman wearing** goggles, a type of protection glasses, and a shield <TEC: Piece of Cake 3>

(25) And another great thing about my job is **travelling**. <TEC: Green Line 2>

The remaining concordance lines were manually annotated for morphosyntactic features which could not easily be automated (see 4.2.2.4) as well as for the functional aspects detailed in 4.2.2.5.

### 4.2.2.4 Morphosyntactic features

Concordance lines were only annotated as questions if the progressive verb itself was the focus of the question, regardless of the syntax (26); the presence of a tag question following the progressive verb phrase was not coded as a question (27).

(26) You're **working tomorrow?** yes yes <BNC2014: S33B>

(27) you're not going swimming yet **are you?** <BNC2014: SQG4>

GET and HAVE passives were annotated as passives.

### 4.2.2.5 Functions of the progressive

Time reference occasionally proved more difficult to annotate than anticipated. For instance, the Youth Fiction corpus yielded many examples of free indirect speech, whereby the authors used time adjuncts associated with the present time reference to create a sense of immediacy in their narration of past events (28). Despite the contradicting adjuncts, such instances were nevertheless coded as 'past' time references.

(28) The smile **was getting** on his nerves, and **now** he was hearing things.
<Youth Fiction: Pratchett 1999: The Fifth Elephant>
A further categorising decision had to be made concerning the interpretation of the time reference of progressives in reported thought and speech. Unfortunately, cases of tense shifting are not discussed in Römer's study (2005). It was decided that, in the present study, the variable time reference refers to the (presumed) intended time reference at the time the progressive verb phrase was first uttered or thought. Thus, excerpt (29) was coded for time reference 'present' and (30) for 'future'.

(29) I called out to them, saying **I was going** to bed. I didn't want them asking any awkward questions. <TEC: Access G 5>

(30) Finally, I asked if Benny had plans to visit more festivals. He told me that in May **he was attending** the Calaveras County Fair in California for the world's biggest frog jumping competition, with thousands of frogs. <TEC: Achievers B1>

Further, the core functions and additional functional features of the progressive that emerged from Römer's (2005: 80-110) corpus-driven investigation of the progressive in spoken British English were also coded for manually. Over the course of the first annotation cycle, a number of changes to Römer's (2005) categorisation were deemed necessary. First, the category 'general validity', which in Römer's (2005) annotation scheme was treated as an 'additional function' category, was transferred to the 'time reference' variable because it proved very difficult to logically assign a verb referring to a generally valid state or action to past, present or future time reference, e.g.:

(31) I am Plymouth's official town crier. Ah, you call out the news so that everybody in the city knows **what's happening**. That's correct. <TEC: Access G 2>

Second, due to low inter-rater agreement disambiguating between the two categories 'emphasis or attitude' and 'shock or disbelief', these were merged into one broader 'emphasis/shock' category. Finally, the additional function category 'old and new habits' was deleted because it frequently overlapped with the 'repeatedness' function and the two were thus difficult to disambiguate. Table 9 lists the additional function categories used for the present study.

<table><caption>Table 9: Core and additional function categories for the progressive (all examples from Römer 2006b: Appendix)</caption><thead><tr><td>Function</td><td>Example sentence</td></tr></thead><tbody><tr><td>Continuousness</td><td>the strange black things that some of <strong>you are holding</strong> in your hands are called riders, and these are end leaves for the storage binder</td></tr><tr><td>Non-continuousness</td><td>I say real progress has been made but today I <strong>am asking</strong> you to think about the next step</td></tr><tr><td>Repeatedness</td><td>she doesn't eat that much, but what <strong>she is eating</strong> i everything's sweet. Mm. Chocolates and Is she eating them? Pardon?</td></tr><tr><td>Non-repeatedness</td><td>Oh I see. Pork's very nice. <strong>What's Geoff eating?</strong> Sausage roll. Oh Geoff, you've only just had your tea!</td></tr></tbody></table>
<table><tr><td>Politeness or softening</td><td>I'm sorry I'm not clear as to whether <strong>you are suggesting</strong> that there should be policy upper case criteria and some non policy lower case criteria.</td></tr><tr><td>Emphasis/shock</td><td><strong>You're not suggesting</strong> pregnancy's a disease there are you?</td></tr><tr><td>Gradual change and development</td><td>Er, it is a very difficult climate, <strong>it's becoming</strong> increasingly difficult, and indeed, it's affecting the work that we do</td></tr><tr><td>Framing</td><td>So anyway yesterday afternoon <strong>I was checking</strong> through it when the phone went again.</td></tr></table>

### 4.2.2.6 Validation of the annotation scheme

Having finalised the coding scheme, the present author and a student research assistant<fnref n="22" /> each annotated a set of 100 random concordance lines from each of the four corpora under study. Inter-rater agreement was found to be 96.22% (across the nine manually coded variables). The majority of disagreements concerned the ‘continuous/non-continuous’ variable, which is particularly sensitive to subjective interpretation. Following discussions, it was agreed to follow Römer’s (2005: 87–88) criteria (“procedural”, “extension over a certain time span” vs. “short-term actions”, “punctiform”) and examples as closely as possible. As a result, (32) was coded as ‘continuous’ whilst (33)–(34) were not. Half the concordance lines were then annotated by the present author and the other half by the research assistant. Difficult cases were discussed to reach a common agreement and the label ‘unclear’ was applied for cases which lacked the necessary context to make a reliable decision.

(32) He didn't know what **was happening**, he couldn't explain anything.
<TEC: Green Line 4>

(33) I'm not saying that **I'm never coming back**. <TEC: New Missions 2<sup>de</sup>>

(34) **it's you're putting** a T sound in there <BNC2014: SMHY>

Intra-rater agreement rates over a period of four weeks were 98.66% and 96.22% and, aside from a few inattention errors in either the first or second annotation phase, the differences were mostly due to the second coder seemingly gaining in confidence with fewer concordance lines being coded as ‘unclear’ in the ‘continuous/non-continuous’ and ‘repeated/non-repeated’ variables in the later sample.

### 4.2.2.7 Frequency of the progressive

This analysis aims to compare the frequency of progressives in the two TEC registers with that of their corresponding reference corpora. To this end, the simplest method is undoubtedly a comparison of relative frequencies per number of words or tokens (see (35), termed ‘M-coefficient’ in Smitterberg 2005). However, a meaningful comparison of the frequencies of progressives is difficult to achieve using such a

<footnote n="22">²² Many thanks to Tatjana Winter for her meticulous work and insightful comments.</footnote>
coefficient because it necessarily confounds the frequency of progressives with that of finite verb phrases.

$$ (35) \qquad M = N_{\text{PROG}} / N_{\text{WORD}} \times 100,000 $$

Preliminary analyses showed Textbook Conversation to be considerably more nominal than naturally occurring conversation (Le Foll 2017) so that M-coefficients would likely not suffice to conclusively answer the first research question (see 4.1.3). Moreover, attempting to conduct statistical tests to compare M-coefficients would make little sense because, in practice, language users do not choose between a progressive form and any other word. Instead, they are more likely to choose between a progressive or a non-progressive verb form (see also Mair & Hundt 1995: 114f). Furthermore, the test typically used for such comparisons, the chi-square test, requires the categories to be compared to be mutually exclusive. This is difficult to achieve in the case of progressives because a progressive verb phrase can span more than two words (e.g., *have been eating*, *are being wasted*) and thus the word/progressive categories are not necessarily mutually exclusive (see Smitterberg 2005: 39-42).

The K-coefficient (36) (devised by Nickel [1996] and discussed in Smitterberg 2005: 42-45 in detail) addresses these drawbacks by dividing the number of progressives by the number of verb phrases, excluding those that the progressive construction cannot fill.

$$ (36) \qquad K = N_{\text{PROG}} / (N_{\text{VERB}} - N_{\text{NOPROG}}) \times 10,000 $$

Whilst this approach satisfies the statistical requirement for a progressive vs. non-progressive alternation ratio measure, it is both difficult to operationalise without considerable time-consuming manual annotation and highly subjective. Indeed, decisions as to whether a progressive would theoretically have been possible in any given verb phrase are inherently theory-dependent. Consequently, such a measure would yield results which would not be comparable across studies. An arguably more objective measure is the V-coefficient (37) (Smitterberg 2005: 44-45) which relates the number of progressives to the number of verb phrases:

$$ (37) \qquad V = N_{\text{PROG}} / N_{\text{VERB}} \times 10,000 $$

Whilst the V-coefficient is fairly easy to approximate using a POS-tagged corpus, it retains a number of the disadvantages encountered with the M-coefficient. If non-finite verb phrases are not included in progressive count (as in this study and several others, e.g., Rautionaho 2014; Rautionaho & Deshors 2018; Rautionaho, Deshors & Meriläinen 2018), these also need to be excluded from the verb phrase count. Furthermore, if a variationist stance on the progressive/non-progressive construction is to be adopted, the number of progressives presumably ought to be subtracted from the total number of finite verb phrases. Smitterberg (2005: 45-48) also advocates for the exclusion of certain constructions in which the progressive is never featured, i.e.,