generosity, hope... great values. You see, he's a role model! He will be remembered forever. <TEC: New Mission 2>

For example, excerpt (213) scored -6.10 on Biber's (1988) Dimension 1, which is the result of its considerably higher type/token ratio and longer average word length than most natural conversations, as well as the fact it features many complex nominal phrases, which lead to high relative frequencies of prepositions and attributive adjectives – all of which contribute negatively to Dimension 1 scores. Thus, textbook dialogues that score particularly low could be flagged as potentially worth re-examining or revising (see Le Foll 2021b). Inversely, when textbook informative texts score particularly high on Biber's (1988) Dimension 1, this is a sign that they are unlikely to be of use as models for students to acquire the skills necessary to write their own informative texts or read for information independently outside the classroom; hence, here too, corpus-informed revisions should be considered (see Chapter 8).

## 6.5 Limitations

The present chapter also highlighted several issues in applying Biber's (1988) model as the baseline in an additive MDA aimed at exploring linguistic variation in school EFL textbooks and comparable target language registers. Solutions to overcome issues related to text length (see 6.2.2), the non-independence of texts from the same textbook series, web domain or novel (see 6.2.6), and the punctuation-dependent operationalisation of some of the features entered in Biber's (1988) MDA (see 6.3.2.1) were discussed and implemented. Mixed-effects models were used to identify and quantify additional factors of linguistic variation, such as the proficiency levels, series, and countries of publication/use of the textbooks (see 6.2.6). In comparison with text register, these were shown to only play a marginal role in mediating textbook language variation.

For a more precise and robust analysis of Textbook English registers, however, a full MDA is conducted in Chapter 7. This new multi-feature/multi-dimensional model aims to overcome some of the limitations encountered in the application of additive MDA to the study of Textbook English. First, this chapter has shown that there is clearly a need to carefully consider which lexical, grammatical and/or semantic features are most relevant for the linguistic analysis of school EFL textbooks. Biber's (1988) Dimensions 5 and 6, in particular, were shown to be of very limited use because they capture features which are rare in general English and would thus require considerably longer texts to be measured with any degree of accuracy.

Crucially, this chapter has also shown that the operationalisation of a number of Biber's (1988) features is not well suited to the present data. Thus, in addition to choosing salient features, it must also be ensured that the selected features can be reliably identified using automatic means because, given the large number of texts
and features to be tagged and counted, even partial manual annotation is clearly not
feasible. This chapter has pinpointed a number of issues with the feature
operationalisations applied in Biber's (1988) MDA. The exclusive reliance on
punctuation marks to identify linguistic features such as discourse markers and
stranded prepositions, which are highly frequent in transcripts of spoken language
that may not include any punctuation, is a case in point. In addition, the discussion
of register variation along Biber's (1988) Dimension 3 (see 6.3.1.3) revealed that,
applying the original operationalisation, the nominalisation variable captures typical
and highly frequent classroom nouns, such as *activity* and *document*, which inevitably
skew the results. The nature of ELT tasks may also call for a re-consideration of the
semantic verb categories used (which Biber borrowed from Quirk et al. 1985: 1180-
1883). Indeed, the private verb class includes *BELIEVE*, *FEEL* and *PRETEND*, but also
*KNOW*, *LEARN* and *SHOW*, and the public verb class includes many reporting verbs
such as *CLAIM*, *SAY*, *REPORT* and *WARN*, but also *EXPLAIN*, *PRONOUNCE*, *REPEAT* and
*WRITE*, which are likely to fulfil rather different functions in textbook language.
Similarly, there is a risk that the results concerning the comparison of Textbook
Conversation vs. the Spoken BNC2014 representing naturally occurring conversation
were skewed because Biber's (1988) noun variable aggregates common and proper
nouns and many textbook dialogues include the name of the person speaking at the
start of every turn, e.g., (204)–(205). This will undoubtedly have inflated the relative
frequencies of nouns in these textbook dialogues and an improved method will need
to remedy this.

As a result of these aforementioned limitations that arose from following the
traditional MDA framework and Biber's (1988) model as a baseline, the following
chapter lays out an improved method with the aim of developing a more accurate
and robust model of linguistic variation in Textbook English.
<figure><img src="image_54.png" /></figure>
# 7 Towards a new multi-dimensional understanding of Textbook English

> I hope to travel and learn some languages. Maybe I'll become a journalist. Or an interpreter. Or even an English teacher. That will be tough - I don't know enough about English grammar!
>
> <TEC: Achievers B1+>

## 7.1 Introduction

The present chapter addresses the same three sets of research questions pursued in Chapter 6:

1. What is the extent of the linguistic variation across the major registers of Textbook English? How are the different textbook registers characterised linguistically? To what extent do the proficiency levels of textbooks interact with register-based variation? Do some textbook series show significantly more or less register-based variation?

2. To what extent do Textbook English registers differ from situationally similar, naturally occurring registers? To what extent are (some of) the observed patterns moderated by textbook series, their country of use, and/or the proficiency level of individual textbook volumes?

3. What are the defining linguistic features that characterise Textbook English registers as compared to these target language registers?

Chapter 6 already provided initial answers to these questions and concluded with a summary of the strengths and limitations observed in applying Biber's (1988) model of general spoken and written English to the study of Textbook English. Whilst Chapter 6 followed an 'additive MDA' approach (Berber Sardinha et al. 2019) to the study of Textbook English as a variety of English, the present chapter tackles the aforementioned research questions by conducting a 'full' or 'novel' MDA (see 6.1.2.2 for an explanation of the distinction) with the aim of providing more comprehensive, detailed and robust answers to these questions.

## 7.2 Methodology: the MDA framework revised

Whilst the MDA framework (Biber 1984; 1988; 1995) remains a highly influential and popular method of multi-feature linguistic variation analysis (see, e.g., Berber Sardinha & Biber 2014; Berber Sardinha & Veirano Pinto 2019; Goulart & Wood 2021), it is not without its problems. Some of the most comprehensive criticism has been expressed by Evert (e.g., 2018) and Lee (2000). Some of these issues have already been touched upon in 6.5.
Potential methodological pitfalls associated with the MDA paradigm (as it is most frequently applied) can be summarised as:

* Design bias in the selection of text samples

* Design bias in the choice and operationalisation of features

* Uncertainty over the reliability of the feature counts

* Confusion between covariation of features due to situational variation and covariation due to grammatical structure

* Lack of transparency in the quantitative patterns captured by factor analysis

* Degradation of correlations due to poorly distributed variables

* Arbitrary thresholds in the computation of dimension scores

* Misleading visualisation of the results

* Difficulties in establishing the statistical significance and robustness of the results

* Issues with the reproducibility and replicability of the results

The following section addresses these ten areas of potential issues, outlining how they have guided the various linguistic, computational, and statistical decisions made as part of the design of the multi-feature/multi-dimensional method applied in the present chapter. In sum, this methodology is strongly inspired by Biber's MDA framework (see 6.1.1) but departs from the way it is traditionally applied in a number of significant ways. Modifications to the framework have been implemented both as a result of general, methodological issues associated with MDA (as outlined by Evert 2018; Lee 2000 and others) and of specific problems arising from the nature of Textbook English and the research questions outlined in 7.1.

Section 6.1.1 explained how, in unsupervised approaches to situational variation analysis such as MDA, the characteristics of each text in a corpus are first summarised in a numeric vector, which stores the relative frequencies of a large number of linguistic features in any one text. With this in mind, two potential design biases are evident: the selection of text (samples) to be analysed and the choice and operationalisation of linguistic features to be counted in these texts. The first issue is discussed in 7.2.1, whilst the latter is covered in 7.2.2. This is followed by a section discussing issues related to the (often unquantified or, even, unquantifiable) reliability of these feature frequencies and how it has been dealt with in the present chapter (7.2.3). Issues arising from the blanket use of per-word normalisation baselines are addressed in 7.2.4. Sections 7.2.5 and 7.2.6 shed light on some of the potential pitfalls associated with different factor analysis methods and, in particular, exploratory factor analysis (EFA) with oblique rotation of the factor matrix and explain why principal component analysis (PCA) was chosen instead. The next two sections deal with potential pitfalls in the interpretation (7.2.7) and presentation (7.2.8) of the results of such analyses, whilst the remaining sections of this methodology section outline potential issues in assessing the significance (7.2.9) and reproducibility (7.2.10) of the results of MDA studies. Each of these sections also explains how the methodology applied in the present chapter intends to overcome, or at least mitigate, these issues.
7.2.1 Design bias in the selection of text samples

One of the first question to be addressed in the text selection process is: What is the minimum number of texts that needs to be entered in the analysis to obtain robust results? In text-linguistic research designs, such as those applying MDA, the sample unit is the text, hence the sample size represents the total number of texts. As a general rule of thumb, factor analysis is said to require a dataset of at least five times as many observations (i.e., here, texts) as independent variables (i.e., linguistic features) to be included in the analysis (Hair et al. 2019: 133). Thus, when using Biber's original 67 lexico-grammatical features, a minimum of 335 texts are needed to conduct a full MDA. That said, a high ratio of number of texts to independent variables is desirable (Hair et al. 2019: 133; in the context of MDA specifically, see Friginal & Hardy 2014: 304). As will become evident in the following, the text to linguistic feature ratios in the analyses carried out in the present chapter are, indeed, considerably higher than five.

Section 6.2.2 described how the text units of the Textbook English Corpus (TEC) were defined for the purposes of the additive MDAs carried out in Chapter 6, as well as the steps undertaken to deal with texts that did not reach the chosen minimum text length threshold of 400 words. As compared to Chapter 6, minor changes were made to the text unit subdivisions of the Conversation subcorpus of the TEC (Textbook Conversation). First, as noted in 6.5, the names or character denominations of the interlocutors in textbook dialogues (e.g., *teacher*, *pupil 1*, *journalist*) were removed from all the conversation texts of the TEC when they are printed at the start of every line. As part of this semi-automatic process, the subdivisions of the text units of Textbook Conversation were revised to account for the fact that sometimes more than one text is featured on a single CD track or DVD unit. The first process decreased the total number of words in the Textbook Conversation subcorpus from 512,587 to 505,147 whilst the second increased the total number of texts to 593.

TEC texts categorised as Poetry & Rhyme were too few to be meaningfully included in the analyses presented in this chapter; hence, the following multi-feature/multi-dimensional analyses of Textbook English compare the linguistic characteristics of 1,977 TEC texts categorised as belonging to the following five registers: Conversation, Informative texts, Instructional texts, Fiction, and Personal correspondence (see Table 62 for an overview of the TEC texts entered in the present analyses).
<table><caption>Table 62: Textbook English Corpus (TEC) texts processed in this chapter</caption><thead><tr><th>Textbook Registers</th><th>Number of texts</th><th>Number of words<sup>46</sup></th></tr></thead><tbody><tr><td>Conversation</td><td>593</td><td>505,147</td></tr><tr><td>Fiction</td><td>285</td><td>241,512</td></tr><tr><td>Informative texts</td><td>364</td><td>304,695</td></tr><tr><td>Instructional texts</td><td>647</td><td>585,049</td></tr><tr><td>Personal correspondence</td><td>88</td><td>69,570</td></tr><tr><td><strong>Total</strong></td><td><strong>1,977</strong></td><td><strong>1,705,973</strong></td></tr></tbody></table>

Issues pertaining to the selection of the texts or text samples, in particular concerning the comparability of the textbook texts and those of the target language reference corpora, have already been addressed in 3.3. This chapter relies on the same versions of the Youth Fiction and Info Teens corpora used in the previous chapters (see 3.3.2 for details). The only difference between the reference texts tagged for the additive MDA in Chapter 6 with the MAT (Nini 2014; 2019) and those tagged with the Multi-Feature Tagger of English (hereafter MFTE; Le Foll 2021a; 2021e) in the present chapter is that, for the present analyses, the texts of the Spoken BNC2014 included full stops at utterance boundaries to boost the reliability of the feature counts of the tagger (see *Online Appendix 3.3* for details of the procedure and Le Foll 2021b for an evaluation of the accuracy of the MFTE on the Spoken BNC2014 with punctuation marks at utterance boundaries).

### 7.2.2 Design bias in the choice and operationalisation of features

Of all the points of criticism listed at the beginning of 7.2, this one is perhaps the one with the most far-reaching consequences. It was already explained in an early and particularly insightful review of Biber’s 1988 publication:

> It is obvious that a method that hinges on statistically determined patterns of co-occurring features will be very sensitive to the selection and identification of these features. If the features are ill-defined, functionally heterogeneous, stylistically skewed, etc., this is likely to have an immediate effect on the results (Altenberg 1989: 171).

In Biber’s original MDA design, the linguistic features were chosen on the basis of previous literature (Biber 1988: 71–72). Although Biber and others have always made clear that the range of features entered in MDAs ought to be as broad as possible so as to have the potential to unearth hitherto unseen patterns of variation (Conrad & Biber 2013: 15; Egbert & Staples 2019: 127), such an approach nevertheless risks introducing biases (Diwersy, Evert & Neumann 2014: 174). Altenberg (1989: 173)

<footnote n="46">⁴⁶ As counted by the MFTE.</footnote>
best illustrates this risk with what he calls the “stylistic ‘predisposition’” of some of Biber’s (1988) categories. For instance, since they both have connective functions in discourse, Altenberg argues that Biber’s (1988) ‘conjuncts’ category (which includes *alternatively, consequently, further, hence, however*, etc.) is functionally equivalent to the category of ‘discourse markers’ (which includes *anyway(s), anyhow, now, well*, etc.) and that the distinction between the two is merely situational – the first being specific to literate genres, whilst the second is typical of spoken interactions (see also Siepmann 2004).

To counter the risk of circularity that arises from using top-down feature selection methods, some (foremost computational linguistic) studies have opted for bottom-up approaches to generate features. These approaches correspond to what has also been referred to by some as “corpus-driven research” (see, e.g., Meunier & Reppen 2015: 499; Xiao 2009: 993–996). Such research rejects “*prima facie* those theories, axioms and precepts that were formulated before corpus data became available” (Tognini-Bonelli 2001: 179), hereby avoiding all types of linguistic preconceptions, including those concerning lexico-grammatical categories, e.g., parts-of-speech. Arguably the most data- or corpus-driven approaches to the selection of features involve character n-grams<fnref n="47" />. However, studies based on character n-grams frequently reveal relatively trivial topic-related patterns rather than more generalisable linguistic ones (Baroni & Bernardini 2005: 264; Popescu 2011: 638; Volansky, Ordan & Wintner 2015: 111). Additionally, character n-grams can be argued to lack “direct linguistic motivation or interpretation” (Argamon 2019: 111) and, like token/word n-gram-based methods, they largely fail to account for polysemy. All other data-driven, bottom-up approaches inevitably involve some form of theory-dependent pre-processing steps such as tokenisation, lemmatisation, part-of-speech tagging, (shallow) syntactic parsing, or combinations thereof. It can be argued that such tools add layers of biases in that they rely on specific, pre-established theoretical models of language analysis (cf. Sinclair 1992: 385–390).

In sum, seeking to entirely eliminate bias in the feature selection and operationalisation process whilst nevertheless arriving at a linguistically meaningful and generalisable set of linguistic features may be an unattainable objective. Taking a more optimistic stance, McEnery & Hardie (2011: 114) suggest that bias in the selection of features for MDA can be reduced by ensuring that the selection is “both *principled* and *exhaustive*”. The present multi-feature analyses rely on the feature portfolio of a new lexico-grammatical tagger specifically designed for the analysis of situational variation in general spoken and written English: the Multi-Feature Tagger

<footnote n="47">⁴⁷ The first three character tri-grams of this footnote are: *the he_* and *e_f*. Note that in many computational linguistic studies relying on character n-grams, however, whitespaces and/or cross-token n-grams are ignored. When adopting the latter option, the method is arguably no longer truly data-driven since it requires an additional layer of tokenisation.</footnote>
of English (hereafter: MFTE; Le Foll 2021e). Whilst the MFTE makes no claim to
have an entirely “principled and exhaustive” feature portfolio, numerous steps were
undertaken to reduce bias in both the selection and operationalisation of the features.
To this end, simplified Hallidayian system networks were examined to ensure that no
major aspect of English lexicogrammar would be overlooked (for details, see Le Foll
2021e, cf. Matthiessen 2019; and Whitelaw & Argamon 2004). Ultimately, however,
the final choice of features was necessarily restricted by both practical and
computational constraints. In particular, the large number of texts usually entered in
MDAs means that only features that can relatively reliably be retrieved using
automated queries were ultimately included in the feature portfolio of the MFTE. To
cite but one example, these constraints resulted in a tagger that makes no distinction
between *that*-relative clauses and other *that*-subordinate clauses (unlike the Biber
Tagger, whose output, however, is often manual “fix-tagged” for such problematic
features; see, e.g., Gray 2019).

Crucially for the interpretability of the dimensions that emerge from the present
multi-feature analyses, the MFTE was developed with the aim of arriving at a set of
features that not only can be identified to a high degree of accuracy in a variety of
written and spoken registers of English, but which can also be meaningfully
interpreted in terms of their function. In other words, the developer's aim was that
each feature's "scale and values represent[s] a real-world language phenomenon that
can be understood and explained" (Egbert, Larsson & Biber 2020: 24). Whilst no
automatic tool can ever pretend to be able to achieve this perfectly, countless tagger
development-evaluation cycles were completed to arrive at a set of algorithms that
best fulfils this criterion. The manifold decisions involved in the selection and
operationalisations of these features are detailed in the MFTE's user documentation
(Le Foll 2021e; see also *Online Appendix 7.1*).

Appendix I (see also *Online Appendix 7.1*) provides a comprehensive list of the final
set of over 80 features of the MFTE feature portfolio for which the texts of the TEC
and the three reference corpora were tagged for the present analyses (see also Table
63 for an extract). Note that, although the table in Appendix I is subdivided into
broad linguistic categories (see also the first column of Table 63 for illustration
purposes), these merely serve organisational purposes and do not seek to represent
any specific theoretical or functional categorisation. Indeed, many features could
equally well be subsumed under a different category. The second column (see also
Table 63) provides a very brief description of each linguistic feature. The third
corresponds to the tags assigned by the MFTE. Note that these same abbreviations
are also used in the tables and figures presented in the results section (7.3). Examples
of different language patterns exemplifying these features are found in the fourth
column. Finally, the operationalisation column contains simplified, written-out
explanations of the combinations of regular expressions used to identify each feature.
For more details, the interested reader is invited to examine the tagger source code available on GitHub (https://github.com/elenlefoll/MultiFeatureTaggerEnglish).

<table><caption>Table 63: Excerpt of Appendix I: Operationalisation of the 'DO as an auxiliary' feature</caption><thead><tr><th>Category</th><th>Feature</th><th>Code</th><th>Examples</th><th>Operationalisation</th></tr></thead><tbody><tr><td>Verb semantics</td><td>DO auxiliary</td><td>DOAUX</td><td>Should take longer than it does. Ah you did. She needed that house, didn't she? You don't really pay much attention, do you? Who did not already love him.</td><td>Assigned to do, does and did as verbs in the following patterns: (a) when the next but one token is a base form verb (VB) (e.g., <u>did it work?</u>, <u>didn't hurt?</u>); (b) when the next but two token (+3) is a base form verb (VB) (e.g., <u>didn't it work</u>); (c) when it is immediately followed by an end-of-sentence punctuation mark (e.g., you <u>did</u>?); (d) when it is followed by a personal pronoun (PRP) or not or n't (XX0) and an end-of-sentence punctuation mark (e.g., do you? He didn't!); (e) when it is followed by not or n't (XX0) and a personal pronoun (PRP) (e.g., didn't you?); (f) when it is followed by a personal pronoun followed by any token and then a question mark (e.g., <u>did you really?</u> <u>did you not</u>?); (g) when it is preceded by a WH question word. Additionally, all instances of DO immediately preceded by to as an infinitive marker (TO) are excluded from this tag.</td></tr></tbody></table>

The MFTE performs feature extraction over several iterations over the texts of a corpus. First, each text is tagged for part-of-speech with the Stanford Tagger (bidirectional version 3.9.2; Toutanova & Manning 2000; Toutanova et al. 2003). Next, rule-based algorithms are run to identify linguistic features necessary for the identification of other features; e.g. DO auxiliaries are first identified on the basis of various combinations of POS tags and forms of the verb DO, before imperatives can be tagged. This ensures that imperative forms of the verb DO can be disambiguated from auxiliary forms, in particular those included in *yes/no* questions where the *do/does/did* frequently occur after an end-of-sentence punctuation mark (see *Online Appendix 7.1* for details). Since the Stanford Tagger provides the first layer of linguistic annotation (tokenisation and POS tagging), the accuracy of the feature extraction is heavily dependent on the accuracy of the Stanford Tagger. Whilst it is a well-tested and robust model, it is by no means perfect (Toutanova et al. 2003; Spoustová et al. 2009; Manning 2011). As a result, some of the feature operationalisations outlined in Appendix I include more tags and/or loops then would be necessary if the POS-tagging process were failproof. For instance, since the Stanford Tagger was found to frequently fail to differentiate between past tense (VBD) and past participle forms (VBN), the algorithms designed to capture passives (PASS and PGET) and the perfect aspect (PEAS) include syntactic patterns with
either the VBN or the VBD tag in order to improve recall rates whenever past participles have been erroneously tagged as VBD. Whilst using a POS-tagger as the basis for the feature extraction process reduces the reproducibility of the method as different tagging software (and versions) will inevitably produce different results (Bohmann 2017: 165), the gain in recall and precision is huge and many of the linguistic features of the MFTE’s feature portfolio simply cannot be extracted without this initial annotation layer.

Although the MFTE was designed as an all-purpose tagger of general English, its first intended use was for the present project. As a result, some of the feature operationalisations could be adapted to the specificities of the corpora under study. An example of such tailoring concerns the operationalisation of the imperative verb feature. To begin with, the MFTE assigns the imperative tag (VIMP) to tokens identified by the Stanford Tagger as base form verbs (VB), which have not previously been tagged as DO auxiliaries (DOAUX) and are immediately preceded by a punctuation mark other than a comma or such a punctuation mark and an adverb. Textbook instructions often begin with a verb in the imperative; however, these are not always proceeded by an end-of-sentence punctuation mark. Instead, tasks are frequently delimited by a symbol or icon of some kind. These frequently cause OCR issues and produce tokens which are inconsistently identified by the Stanford Tagger as symbols (SYM), list markers (LS) or foreign words (FW). Consequently, the MFTE was designed to also assign to the imperative variable base form verbs which occur after such tokens. It was also noticed that the Stanford Tagger often considers sentence-initial *please* to be a base verb form, hence exceptions were added for the tokens *please* and *thank*. Having identified these sentence-initial imperatives, a second loop then searches for a potential second imperative verb which may occur after *and* or *or* with up to two optional intervening tokens, e.g., (214). Finally, it was noticed that *work* in the phrase *work in pairs*, which occurs more than 700 times in the TEC, is almost invariably identified by the Stanford Tagger as a noun (NN). As a result, this phrase, together with several other frequently occurring phrases which also proved problematic for the POS-tagger were hard-coded as additional exceptions in the version of the MFTE used for the present analyses (version 3.1 ran on perl v.5.22.1 built for x86_64-linux-gnu-thread-multi).

(214)

**Describe or draw**
**Listen carefully and repeat**
**Read the text and answer the questions**

### 7.2.3 Uncertainty over the reliability of the feature counts

It is obvious that the robustness of a statistical method that relies on counts of features depends on the high accuracy of these counts; however, very few MDA studies include thorough evaluations of tagger accuracy (two major exceptions deserving of a mention are Biber & Gray 2013; Gray 2015). Crucially, when reporting
the accuracy of a tagger to be used in MDA, it is important to consider not just the overall accuracy of a tagger, since this will be heavily skewed towards very frequent tags – many of which are particularly easy to tag, e.g., punctuation marks and determiners, but also the tagger’s accuracy per feature. To complicate matters, accuracy can be measured in various ways (see Table 64). Commonly, only ‘precision’, i.e., the percentage of correctly assigned tags within a category, is reported. This is for practical reasons since precision is much easier to spot-check than ‘recall’, i.e., the percentage of a particular feature that is correctly identified as such by the tagger. In practice, however, both precision and recall are important for the results of MDAs to be reliable.

<table><caption>Table 64: Summary of the terminology used in tagger performance evaluation</caption><thead><tr><td>Term</td><td>Definition</td></tr></thead><tbody><tr><td>True positive</td><td>Feature correctly tagged by the MFTE as X</td></tr><tr><td>False positive</td><td>Feature incorrectly tagged by the MFTE as X</td></tr><tr><td>False Negative</td><td>Feature incorrectly not tagged by the MFTE as X</td></tr><tr><td>Precision</td><td>True positive count / (true positive count + false positive count)</td></tr><tr><td>Recall</td><td>True positive count / (true positive count + false negative count)</td></tr><tr><td>F1 score</td><td>2 * (precision * recall) / (precision + recall)</td></tr></tbody></table>

The present MDA relies on the MFTE, whose documentation (Le Foll 2021e) details a full evaluation of the tagger’s accuracy, including breakdowns per subcorpus of the evaluation corpus (the BNC2014 Baby+) and a table listing recall, precision and the combined F1 measure (see Table 64) for each feature. Whilst the overall accuracy of the MFTE is high and the vast majority of features score above 90% on all three accuracy metrics, the results of the tagger evaluation remain relevant to the interpretation of the results presented in this chapter. Indeed, any potential systematic mismatches between the linguistic constructs under study and the features as they are counted in practice (here, by the MFTE) have the potential to skew any model that emerges from an MDA conducted on the basis of these counts. Picoral et al. (2021) demonstrated that differences of more than 10% in the tagging accuracy of two automatic parsers (the Malt Parser and the Stanford Dependency Parser) and one tagger (the Biber Tagger) are not unusual, yet even such relatively small differences can be of relevance to our understanding and modelling of Textbook English as a variety of English.

Consequently, it was important to ensure that the MFTE performs at least as well on the data used in the present study as on the BNC2014 Baby+. In order to investigate any such potentially problematic mismatches, the results of the tagging procedure were manually checked in six texts from the TEC, originating from six different textbook series (two per country of use). These texts were randomly sampled so as to have one text representing each of the six textbook registers of the TEC (see 3.3.1.4). In total, 4,515 tags were manually checked across these six texts, of
which 114 were found to be incorrect. Excluding punctuation tokens and symbols which were always accurately tagged but whose counts are not entered in the analysis, this means that the MFTE has a satisfactorily high rate of accuracy 97.11% [95% CI: 96.53–97.61%] on the TEC data.

As for the three reference corpora used in the present study, the accuracy of the MFTE on the Spoken BNC2014 data was already evaluated in Le Foll (2021e). Excluding unclear tokens which human annotators could not reasonably interpret from the transcripts (n = 7) and punctuation marks, 5,388 tags were found to be correctly assigned by the MFTE whilst 224 were flagged as incorrect, corresponding to an overall accuracy of 96.01% [95% CI: 95.46–96.51%]. No formal evaluation of the MFTE output was carried out for the remaining two reference corpora because they were deemed to be very similar to subcorpora of the BNC2014 Baby+, for which a thorough investigation has already been conducted. The Youth Fiction corpus is highly comparable to the fiction subcorpus of the BNC2014 Baby+, for which the MFTE accuracy was found to be very high (96.93% [95% CI: 96.46–97.39%], excluding punctuation and symbols), whilst the Info Teens corpus shares similarities with the news subcorpus and some of the e-language subregisters, in particular: blogs and forum posts (see Brezina, Hawtin & McEnery 2021). The evaluation files corresponding to these (sub)registers of the BNC2014 Baby+ from the original tagger evaluation (Le Foll 2021e) were thus reanalysed for the purposes of the present study. On these eight news articles, blog, and forum posts, the MFTE reached a slightly lower, but nevertheless satisfactory, accuracy rate of 95.84% [95% CI: 95.36–96.28%] excluding punctuation and symbols.

Taken together, the overall MFTE accuracy for the TEC and the three reference corpora, excluding unclear tokens, punctuation, and symbols, can thus be estimated to be around 96.38% [95% CI: 96.13–96.62%]. The per-feature accuracy measures are visualised in Fig. 49. The full breakdown of the evaluation results and the corresponding code can be found in the *Online Appendix 7.3*.
<figure><img src="image_55.png" /><figcaption>Fig. 49: Per-feature accuracy measures of the MFTE on the TEC, the Spoken BNC2014 and data comparable to the Youth Fiction and Info Teens</figcaption></figure>
In total, 198 different error types (combinations of an erroneous tag assigned by the MFTE and of the corrected tag) were identified as part of the present tagger evaluation. Nearly half of the error types (n = 94) were only identified once. Table 65 lists the most frequent error types which were found 12 or more times across the six evaluation texts of the TEC and the BNC2014 Baby+.

<table><caption>Table 65: Most frequent tagging error types</caption><thead><tr><th>Error type (MFTE tag -> corrected tag)</th><th>N</th><th>Example of error type (incorrectly tagged token and tag in bold)</th></tr></thead><tbody><tr><td>NCOMP -> NULL</td><td>37</td><td>my part-time_NN book_NN NCOMP - stacker</td></tr><tr><td>NN -> JJAT</td><td>35</td><td>a six-game losing_NN run</td></tr><tr><td>JJAT -> NN</td><td>27</td><td>it was the backroom_JJAT staff</td></tr><tr><td>NN -> VB</td><td>27</td><td>YOU MUSTN'T: chat_NN whilst your classmate</td></tr><tr><td>IN -> RP</td><td>25</td><td>He could put on_IN a bit of a show</td></tr><tr><td>NN -> VPRT</td><td>24</td><td>Suzannah the tail-wagger wags_NN the tail of</td></tr><tr><td>VB -> NN</td><td>22</td><td>would expect them to deliver their first win_VB</td></tr><tr><td>THSC -> DEMO</td><td>19</td><td>I'll do that_THSC a bit later cuz</td></tr><tr><td>VB -> VIMP</td><td>19</td><td>List events and place_VB them on a timeline</td></tr><tr><td>NN -> OCR</td><td>16</td><td>Be the most creative! Thin_JJ k_NN: Why?</td></tr><tr><td>VBN -> JJAT</td><td>16</td><td>a barely published_VBN novelist</td></tr><tr><td>ACT -> NULL</td><td>15</td><td>guilty of a foul throw_VB ACT in his own half</td></tr><tr><td>THATD -> NULL</td><td>15</td><td>that's how you remember_VPRT THATD things isn't it?</td></tr><tr><td>CD -> NN</td><td>12</td><td>there's a good one_CD</td></tr><tr><td>MENTAL -> NULL</td><td>12</td><td>No there's loads in here I like_VPRT MENTAL there's a good one</td></tr></tbody></table>

A qualitative analysis of the most frequent tagging errors showed that the first source of errors arose from the texts themselves: the MFTE most frequently mistagged tokens that were either nonsensical as the result of OCR errors or typos/non-standard spellings in the original texts.<fnref n="48" /> In spite of intensive manual corrections, the scanned textbooks of the TEC, in particular, still contain many OCR errors, whilst the e-language subcorpus of the BNC2014 Baby+ features many typos and non-standard spellings (and, indeed, this is also the case in the Info Teens). Unrecognised words are usually assumed by the Stanford Tagger to be nouns, which is why many of the most frequent error types listed in Table 65 involve the MFTE mistagging tokens as nouns (NN) and identifying noun-noun compounds (NCOMP) when there are none. In such instances, the human annotators added the NULL tag in the correction field (for details, see Le Foll 2021e).

<footnote n="48">⁴⁸ Note that, for the purposes of the present evaluation, tokens that were poorly processed but nevertheless correctly tagged were not counted as tagging errors (e.g., when *Helen* was misrecognised as *Llelen* by the OCR software but still correctly identified as a noun by the MFTE).</footnote>
The second major source of tagging errors stems from the underlying layer of POS-
tagging carried out by the Stanford Tagger that the MFTE relies on for nearly all its
algorithms. These errors typically involve confusing the POS of word forms with more
than one possible POS. For example, in one of the evaluation files, the word *build*
occurred four times as a noun in the compound *beta build* but was erroneously
identified as a verb by the Stanford Tagger. As explained in 7.2.2, the MFTE was
designed to counter some of the most systematic frequent errors that the Stanford
tagger was found to make (e.g., identifying the ‘s in *let’s* as a possessive marker, or
better differentiating between past participle and past tense forms). Nevertheless,
when a word form with different possible POS is found in an unusual context, the
most frequent of the possible POS is usually assumed by the Stanford Tagger.

Finally, the third largest source of feature identification errors is due to how the
features are operationalised in the MFTE. For many of the more complex
grammatical features, in particular, the algorithms of the MFTE outlined in
Appendix I and in the *Online Appendix 7.1* are best approximations, which are
designed to capture the majority of these features, but they are by no means expected
to be 100% accurate in all contexts. The tagging issues related to the incorrect
identification of imperatives (VIMP), verbs of action (ACT), and *that*-omissions
(THAD), in particular, belong to this category.

A full list of all the erroneously assigned tags and their corrections can be found in
*Appendix 7.3*. They will not be discussed any further here because they are very
similar to those found in the formal evaluation of the MFTE carried out on the
BNC2014 Baby+ data (Le Foll 2021e). In sum, the results of this brief evaluation
confirm that the corpora used in the present study are suitable for tagging with the
MFTE.

### 7.2.4 Confusion between covariation of features due to situational variation and covariation due to grammatical structure

Most MDAs are carried out on collections of texts that vary in length. However,
counts of linguistic features in texts of different lengths cannot be directly compared.
Let us imagine a short business e-mail of 200 words that features four occurrences of
the word *if* and compare this to an imaginary novel of, say, 20,000 words in which
the word *if* is observed eight times across the entire book. Comparing these raw
counts, we may naively be tempted to conclude that *if* is twice as frequent in fictional
writing as in professional e-mails. Evidently, however, this comparison does not
account for the vastly different number of potential opportunities of use of the word
*if* in the two texts. To remedy this, the *de facto* standard in corpus linguistics has so
far consisted in normalising raw counts to a common word-based denominator. For
example, the count of four *if’s* in our hypothetical e-mail can be divided by the total
number of words in the e-mail (200) and the count of eight in the book by the length
of the novel (20,000) before multiplying both results by a common denominator, e.g., by 1,000 words. In our example, this approach results in normalised *if* frequencies of 20 per 1,000 words in the e-mail and 0.4 per 1,000 words in the novel. In other words, once normalised on a per-word basis, we might conclude that *if* is in fact 50 times more frequent in professional e-mails than in novels! Typical word-based denominators are 100, 1,000, 10,000 and a million words and the resulting numbers are referred to as normalised, normed, or relative frequencies.

Word-based normalisation baselines imply that words are used independently of each other and therefore do not account for, or attempt to model, the actual choices that language users make when producing (or, for that matter, when processing) language. Thus, word-based normalisation baselines can be said to conflate frequency of use and opportunity of use (Wallis 2020: 47–52). In reality, once a language user has chosen a particular word, they have a limited number of choices at their disposal as to which word can logically come next. This phenomenon is illustrated in Fig. 50. It displays the most frequent word forms that immediately follow the word *if* in the Spoken BNC2014. It shows that more than a third of occurrences of *if* are followed by the word *you* and that the seven most frequent word forms after *if* are all personal pronouns. Thus, Fig. 50 makes clear that not all words (or word classes) are equally likely to occur after the word *if*.

<figure><img src="image_56.png" /><figcaption>Fig. 50: The ten most frequent word forms occurring immediately after *if* in the Spoken BNC2014 (as counted and displayed by Sketch Engine, see https://ske.li/nu8 for full results)</figcaption></figure>

Corpus linguistics has a long history of using word-based normalisation rates indiscriminately and the MDA framework is no exception. Whilst per-word normalised frequencies can be argued to represent language users’ rates of exposure (which may well be what some researchers, e.g., lexicographers, are attempting to model), a number of recent publications have pointed out potential issues with the indiscriminate application of this approach. Wallis (2020: 56), for instance, explains
how per-word frequencies undermine the assumptions of many of the statistical
models used in corpus linguistics which assume that linguistic features follow binomial
distributions, i.e., that it is, in principle, possible to observe proportions of 100%. In
practice, however, it is highly improbable that a language user would simply repeat
a single word or use only words of a single word class for the entire duration of a text!
In the Biberian MDA framework, the indiscriminate use of word-based normalisation
for all linguistic features results in a highly problematic confusion between the
covariation of features due to situational variation (which of course is what the
method actually aims to tease out) and covariation due to grammatical structure (see
also Grieve-Smith 2007). Consider the linguistic features that load on Biber's (1988)
Dimension 1: at the positive, 'involved' end, these include the number of verbal
contractions, negated verbs, and present tense verbs per 1,000 words, which all
correlate strongly with each other. Whilst it is true that these features are particularly
frequent in spoken interactions, it is also undeniable that these correlations are
inevitably mediated by the overall frequency of verbs. Similarly, the high positive
correlations that contribute to negative scores on Biber's (1988) Dimension 1, e.g.,
between the per-1,000-word normalised frequencies of nominalisations, determiners
and, though to a slightly lesser extent, prepositions, are all "grammatically mediated"
by the frequency of nouns.

In the context of MDA, the choice of the normalisation units for the linguistic features
to be entered in a model is anything but trivial. For a start, factor analysis (FA) and
principal component analysis (PCA) are both "exquisitely sensitive to the sizes of
correlations" (Tabachnick & Fidell 2014: 665). Consequently, the authors warn that
"it is critical that honest correlations be employed" (Tabachnick & Fidell 2014: 665,
cf. 95-96). However, covariation due to grammatical structure can be argued to
generate linguistically "obvious correlations" (Evert 2018: 24) and, as such,
"dishonest" ones.

That being true, the choice of normalisation unit(s) for the counts to be entered in
MDAs is by no means a simple one. Reflecting on the quantification of linguistic
measures in general terms, Schegloff (1993: 103) speaks of the need to account for
"environments of possible *relevant* occurrence" (emphasis original) and argues that
"quantitative analysis requires an analytically defensible notion of the denominator".
In practice, however, the choice of the denominator of normalised frequencies, the
normalisation baseline, will depend on both the linguistic conceptualisation of a
linguistic feature, as well as the feasibility of reliably counting what is considered to
be the most meaningful unit to capture an opportunity of use (cf. Wallis 2020: 69-
70). For instance, as a denominator for the counts of *if*, the total number of sentences
or clauses in a given text might seem like the most linguistically meaningful or
"analytically defensible" unit. However, whilst identifying sentences is relatively
trivial in written registers, not only can this unit be argued to not make much
linguistic sense in spoken registers<fnref n="49" />, but it is also impossible to reliably implement
with spoken corpora whose transcription scheme do not include any sentence
boundaries (e.g., the Spoken BNC2014). As for automatically identifying clauses, this
would require dependency parsing, which, to date, remains highly unreliable for
transcriptions of spontaneous spoken language, and which would, in any case,
certainly result in units that would be equally difficult to compare across different
modes and registers.

The present analyses rely on the normalisation baselines as implemented in the
“complex normalisation” output of the MFTE (see Le Foll 2021e for details). For this
output, the MFTE normalises counts of the majority of features including conditional
conjunctions (*if* and *unless*; COND), contractions (CONT), negation (XX0), present
tense verbs (VPRT), and WH-questions (WHQU) to 100 finite verb phrases. The
number of finite verb phrases is approximated to a satisfactorily high degree of
accuracy by the MFTE by adding the counts for present tense (VPRT), past tense
(VBD), imperatives (VIMP) and all the modal verbs (MDCAN, MDCOU, MDMM,
MDNE, MDWO, MDWS) together. Five features, attribute adjectives (JJAT),
s-genitives (POS), noun compounds (NCOMP), quantifiers (QUAN), and determiners
(DT), are normalised to 100 nouns, whilst only the remaining 19 features, e.g., emoji
and emoticons (EMO), discourse markers (DMA) and nouns (NN), are normalised to
100 words.

7.2.5 Lack of transparency in the quantitative patterns captured by factor analysis

In the Biberian framework, MDA relies on factor analysis to reduce a large set of
associations of normalised counts of many different linguistic features across a large
number of texts to a more parsimonious set of underlying, or latent, variables. These
summarising variables are first referred to as ‘factors’ and then, once they have been
functionally interpreted, as ‘dimensions’. Thus, MDA makes use of factor analysis to
reduce complexity and “consolidate variables in a principled manner” (Loewen &
Gonulal 2015: 183) in order to more concisely describe, and ultimately hopefully
understand, the relationships among the linguistic features. The underlying belief is
that such parsimonious solutions will have greater external validity and will therefore
be more likely to replicate (Henson & Roberts 2006: 394).

At this stage, it should be noted that the terminology is often used ambiguously and
that statisticians disagree as to what exactly does or does not constitute ‘factor
analysis’ (see, e.g., Henson & Roberts 2006: 398; Jolliffe 2002: 150). In the present
thesis, ‘factor analysis’ will be used as an overarching term that includes ‘common

<footnote n="49">⁴⁹ “A sentence is a constituent of writing, while a clause complex is a constituent of grammar” (Halliday 1993: 216).</footnote>
factor analysis' and 'principal component analysis' (see explanation below). Following
Biber (1988), the factor-extracting method of choice in MDA studies has traditionally
been exploratory factor analysis (EFA), which is a common factor analysis method.
Although central to the MDA methodology, a meta-analysis of MDA studies
published in English or Portuguese between 1984 and April 2020 found that as many
as 69% (n = 65) of the examined studies did not report which factor extraction
method they used (Goulart & Wood 2021: 124). Of those that did, 78% (n = 25)
reported using EFA.

The use of EFA as a means of reducing complexity in MDA studies has been criticised
as “lacking in transparency” (Evert 2018: 12). One of the reasons for this is that the
results of EFAs are contingent on the number of factors retained by the researcher(s).
This means that solutions in EFA are not unique, but rather many different
dimension scores can be computed for each observation (Lee 2000: 171). This potential
for researcher bias is well-documented which is presumably why all best-practice
guidelines on how to conduct and report EFAs (e.g., Tabachnick & Fidell 2014: 696–
699; Loewen & Gonulal 2015: 194–197) devote a section to various (more or less
objective) factor retention criteria. The problem is that, whilst methods to determine
the number of factors to retain abound (e.g., Kaiser-1 rule, Joliffe’s criteria, visual
inspection of the scree plot, parallel analysis...), they each tend to produce vastly
different results. Crucially, factor indeterminacy in common factor analysis means
that the results of EFAs are not computationally stable. In other words, using the
same data, researchers will obtain different results for, say, the first three factors of
an EFA model depending on whether they decide to extract three, four, five, or more
factors.

Within the traditional MDA framework, determining the number of factors to extract
has been described as an “important part of the iterative process between statistical
procedure and subjective researcher interpretation” (Egbert & Staples 2019: 130).
Comparing the interpretability of the various combinations of factors that emerge
from different factor solutions is considered a valid approach. It is argued that:

> If particular factor solutions (greater number or smaller number) are more interpretable
> than others, then it should be considered as a more favorable solution (Egbert & Staples
> 2019: 130).

Given that the number of factors to be extracted has a more profound impact on the
final solution than any of the other decisions related to the statistical method
employed (e.g., threshold levels, EFA vs. PCA, rotation method) (Lee 2000: 99), the
risk of (conscious or unconscious) researcher bias that such an approach entails is
obvious. This is one of the reasons why Diwersy et al. (2014) and Neumann & Evert
(2021) advocate for the use of principal component analysis (PCA) rather than EFA.
Whilst researchers conducting PCAs still need to choose one of the many methods to
decide on how many summarising, latent variables (referred to as ‘principal
components' in PCA) to retain and interpret, the results themselves will remain the same regardless of how many components are deemed to be worthy of further analysis and (linguistic) interpretation (Jolliffe 2002: 159–160).<fnref n="50" /> Several multi-feature and multi-dimensional linguistic studies have already been successfully conducted using PCA. Notably, Sigley (1997) convincingly applied PCA to develop a formality index of English, whilst Neumann & Evert (2021; based on a method originally developed in Diwersy, Evert & Neumann 2014) have proposed a new approach inspired by Biber's MDA framework, Geometric Multivariate Analysis (GMA), that uses the orthogonal projections (i.e., those of mathematically independent, uncorrelated, factor axes) of PCA to explore differences between texts and groups of texts in multi-dimensional feature space.

Like common factor analysis, PCA is also a dimension-reduction statistical method. However, mathematically, EFA and PCA differ in that PCA accounts for all the variance in the data, thus pooling together shared (or ‘common’) variance between variables, specific variance, and error variance, whilst EFA attempts to estimate the variance due to error and the variance that is unique to each variable in order to eliminate these sources of variance to focus exclusively on the shared (or common, hence the term ‘common factor analysis’) variance. Whilst the latter may produce factor solutions that are more readily interpretable, the computing of the factors themselves is rather opaque. By contrast, the components produced by PCA are linear functions directly derived from the observed variables, i.e., in the present context, from the normalised counts of each linguistic feature in each text of the corpora.

Theoretically, too, EFA and PCA differ in that EFA produces latent variables that are assumed to represent real-life constructs. Conceptually, these constructs are thought to “cause” the distributions of the variables to be as they are observed in the dataset whereas, in PCA, it is the resulting components that are “caused by” or that “produce” the observed variables. Thus, the components of PCA can be said to be empirically real factors that directly represent aggregates of the observed correlated variables. However, these do not necessarily reflect any underlying constructs or processes (Tabachnick & Fidell 2014: 662).

In practice, however, specific and error variance (also known as ‘unique variance’) are often not large enough to significantly affect the first few components of PCA, so that, when focusing on these first few dimensions, the choice of EFA over PCA will usually lead to the same interpretation of the results (Lee 2000: 168–170; Tabachnick & Fidell 2014: 634). Furthermore, Henson & Roberts (2006: 398) explain that any differences in the outputs of the two methods will “decrease as (a) the measured variables have greater score reliability or (b) the number of variables measured

<footnote n="50">⁵⁰ Note, however, that this is no longer true if the PCA solution is rotated (see, e.g., Husson, Lê & Pagès 2017: 29).</footnote>
increases.” Indeed, preliminary exploratory factor and principal component analyses conducted on the TEC and its three reference corpora as tagged by the MAT produced very comparable results (Le Foll 2020a). However, for the present chapter, PCA was chosen for its computational stability and its (relative) ease of interpretation. The latter is also true of the feature weights (loadings) which, in PCA, simply represent correlation coefficients between the observed variables and the components, whereas the factor loadings that emerge from EFAs are factor score estimations. These are mathematically much more complex and, although they fulfil a very similar function, are therefore more difficult to accurately interpret.

A quest for simplicity is also what motivated the interpretation of true principal components as opposed to rotated components. In applying the MDA framework, most linguists follow Biber (1988: 84–85) and apply a rotation to the extracted factor loadings of their EFA solutions. In brief, such rotations are often recommended because they transform the solution such that the extracted factors (or in PCA: the components) are strongly correlated with certain variables and uncorrelated with others. Thus, the procedure simplifies the factor loading structure which is often thought to make the results more readily interpretable. However, this supposed simplification comes at a cost: such rotated factors or components no longer maximise group separations and the rotated loadings across the factors are correlated (Rencher 1992: 219) – something which Biber and others tend to brush aside in their interpretations of such rotated solutions (see Lee 2000: 253–256). In addition, Biber (1988) chose to apply a Promax ‘oblique rotation’ which, unlike the more commonly used ‘orthogonal rotation’ methods, allows for the correlation of factors. Biber (1988: 85fn.) convincingly justifies his choice by explaining that:

> [O]blique solutions might be generally preferable in studies of language use and acquisition, since it is unlikely that orthogonal, uncorrelated factors actually occur as components of the communication process. That is, from a theoretical perspective, all aspects of language use appear to be interrelated to at least some extent, and thus there is no reason to expect mathematically uncorrelated factors representing those aspects (see Hinofotis 1983).

In practice, however, producing oblique solutions considerably complicates the interpretation of the results. Since oblique rotations allow for a degree of correlation between dimensions, both the factor pattern and the factor structure matrices need to be interpreted. However, Biber (1988) and seemingly most linguists who follow Biber in applying oblique rotations only interpret the factor pattern matrix. The degree to which this impacts the results of such studies is difficult to evaluate because very few MDA studies conducted with oblique rotations report the inter-factor correlation coefficients. In the few that do (e.g., Biber 1988: 84; Biber & Egbert 2016: 12), significant inter-factor correlations are reported though these are either not commented on or merely brushed aside as “small” (Biber 1988: 85) or “generally
small” (Biber & Egbert 2016: 12)<fnref n="51" /> even when the results actually point to several
inter-factor correlations above 0.3 or even 0.4. As Lee (2000: 247–254) demonstrates
in his replication of Biber’s (1988) seminal MDA, ignoring inter-factor correlations
can have a profound impact on the interpretation of the resulting factor solution.

In the case of rotating PCA solutions specifically, another disadvantage of applying
rotation is that the results are no longer independent of the number of extracted
components. In other words, using the same dataset, the first two components of a
two-dimensional rotated solution will not be the same as the first two components of
a three-dimensional rotated solution (Husson, Lê & Pagès 2017: 29), which once again
raises the million-dollar question of the “correct” number of components to extract
(see above).

As a result of all the above considerations, the MDAs conducted in this chapter rely
on PCA rather than EFA. Visual inspection of eigenvalues scree plots is used as a
first step to determine how many components are to be analysed. To make the most
of different visualisation options, two different R packages are used to conduct the
PCAs: {stats} for its stats::prcomp function that allows for 3-D visualisation of the
results via {pca3d} and {PCAtools} for its highly customisable PCAtools::pairsplot
and PCAtools::biplot functions for 2-D graphs (for details of all the packages,
functions and parameters used, see code in the Online Appendix 7.2–7.7).

7.2.6 Degradation of correlations

As a family of statistical methods, dimension-reduction methods are known to be very
sensitive to outliers and skewed distributions of variables. Tabachnick & Fidell (2014:
665) lament that “problems created by missing data, and degradation of correlations
between poorly distributed variables all plague FA [factor analysis] and PCA”. These
issues, however, are rarely discussed and, up until now, have largely been overlooked
in MDA studies.<fnref n="52" />

In the context of MDAs with linguistic features, it is perfectly possible for features to
be entirely absent from some of the texts in the corpora under study, thus creating
the impression of “missing data” in the count matrices to be entered in such analyses.
Of course, the data is not “missing” in the traditional sense but rather the rate of
occurrence of these features is simply zero. There are, in theory, three reasons why
this might be the case. The first is quite simply that a text genuinely does not feature
this particular lexico-grammatical unit. For instance, it is easily conceivable that a

<footnote n="51">⁵¹ Since Biber & Egbert (2016) conducted a PCA rather than a common factor analysis method such as EFA or CFA, these are in fact inter-component correlations.</footnote>

<footnote n="52">⁵² As the reader will notice in the following, Lee (2000) constitutes a notable exception; however, his PhD thesis has not yet been published and, as a result, his concerns regarding the use of EFA with untransformed variables appear not to have reached the wider (corpus) linguistic research community.</footnote>
novel may not include a single emoji or emoticon (EMO). Thus, especially linguistic
features that have “strong stylistic discriminating properties” (Lee 2000: 173), such
as emojis and emoticons, will necessarily follow very skewed distributions across
multi-register (or multi-dialect, multi-variety, etc.) corpora. Moreover, texts as long
as an entire novel are rarely entered in an MDA. For example, the Youth Fiction
Corpus analysed in the present study consists of extracts of novels of around 5,000
words each. Second, therefore, we may envisage a situation in which a particular
feature is absent from a text extract, thus returning a count of zero, but can actually
be observed in other parts of the full text. For instance, a short extract of a novel
may not happen to include a single verb in the passive voice, yet it is highly unlikely
that the entire novel does not feature a single verb in the passive. Similarly, if a
complete text is very short it is also likely to have zero occurrences of many of the
least frequent linguistic features, even though this may not be representative of the
text register/variety more generally. For instance, if we examined a small corpus of
tweets, we may find that there are zero occurrences of the word *because* in most
tweets. A hasty functional interpretation of this finding could be that Twitter
discourse favours statements of facts and opinions rather than explanations. In reality,
however, we might find that pooling a random sample of texts as short as tweets from
any other register, be it news reports, novels, or speeches, may yield the same result.
In other words, in some cases, and especially for relatively rare linguistic features,
zero counts can occur simply by virtue of texts being very short. Ultimately, both
these issues tie back to the discussion of an appropriate minimum text length for
MDA studies (see 6.2.2). Therefore, solutions such as those adopted for the present
thesis, i.e., the ordered collation of particularly short texts (particularly relevant for
the TEC; see 6.2.2), and the random sampling of extracts of longer texts (see 3.3.2)
must be carefully considered in order to avoid such issues causing undue influence on
models derived from MDA. Finally, it should also be acknowledged that a third reason
why a text may appear to include zero occurrences of a particular feature may be due
to a failure of the automatic tagger used to identify and count the feature in question.
This risk confirms the need to conduct thorough evaluations of the taggers that are
used in MDAs, as discussed in 7.2.3.

Thus, we have seen that there are potentially several reasons why count matrices
destined to be entered in MDAs may include zeros. For some of these reasons,
mitigating steps have already been undertaken as part of the pre-processing of the
corpus data. However, it may still be necessary to remove linguistic features that are
genuinely very poorly distributed as a result of being entirely absent from a large
proportion of the texts to be analysed using factor analysis. In the present analyses,
features with zero occurrences in more than two thirds of texts were therefore either
excluded from the analyses (see Bohmann 2017: 168 for a similar procedure) or,
whenever linguistically meaningful, merged with other features.
Even having removed features with high percentages of zero occurrences, the distributions of many of the remaining features nevertheless remain highly skewed. By way of illustration, the normalised frequencies of occurrence of five features across the TEC are plotted in Fig. 51. A cursory look at these example histograms points to two potential issues. First, unsurprisingly, the ranges of rates of occurrence (plotted on the y-axes) vary considerably. These ranges depend a) on how frequent a particular linguistic feature is (e.g., we would expect nouns [NN] to be generally much more frequent than split auxiliaries [SPLIT]) and b) on each feature's normalisation basis (e.g., here, the normalised counts of nouns represent the number of nouns per 100 words, whereas progressives [PROG] and split auxiliaries are counted per 100 finite verbs). Second, it is obvious that at least three of these distributions are far from normal and, instead, appear to follow distributions sharing similarities with the Zipfian distribution that is very familiar to linguists (see, e.g., Brezina 2018: 44–46).

<figure><img src="image_57.png" /><figcaption>Fig. 51: Distribution of normalised frequencies of occurrence of five features across the TEC (histograms) and visualisations of their correlation (scatterplots)</figcaption></figure>
Dealing with the first issue is relatively trivial: in such cases, it is common practice
in multivariable analyses, and indeed in Biber's MDA framework, too (Biber 1988:
94-95), to standardise variables to z-scores, i.e., to scale all frequencies to a mean of
zero (μ = 0) and a unit variance of one (σ² = 1). This z-transformation ensures that
each feature makes the same overall contribution to the distances between the texts
that will be explored in the following analyses (see Neumann & Evert 2017: 53, see
also 6.2.4). As for the second issue, Hair et al. (2019: 137) note on the assumptions
of exploratory factor analysis that:

[f]rom a statistical standpoint, departures from normality, homoscedasticity, and linearity apply only to the extent that they diminish the observed correlations.

In other words, factorial patterns may be harder to detect if variables are not normally
distributed and if correlations are nonlinear (as shown in the scatterplots in Fig. 51),
but, if/when they *are* detected, there is no reason to assume that they are not real.
Nonetheless, as pointed out at the beginning of this section, such skewed distributions
run the risk of outliers exerting undue influence on the resulting models. Hence,
following Neumann & Evert (2021), the standardised normalised counts were
subjected to a signed log transformation<fnref n="53" /> in order to (partially) deskew their
distributions. The results of these two transformations, z-standardisation and log-
transformation, are exemplified in Fig. 52. A comparison of the scatterplots in Fig.
51 and Fig. 52 shows that, although these transformations have strengthened the
correlations, many of the distributions remain paranormal and the relationships
between some pairs of features remain nonlinear. As emphasised by Tabachnick &
Fidell (2014: 666), however, as long as PCA (or EFA for that matter) is used
descriptively, “assumptions regarding the distributions of variables are not in force.”
However, they also make clear that when normality and linearity fail, the produced
solutions are “degraded” – though this is not to say that they may not “still be
worthwhile” (Tabachnick & Fidell 2014: 666-667). Note that, whilst Neumann &
Evert (2021) apply a signed logarithmic transformation to deskew feature
distributions as an alternative to removing very sparse features, the present
methodology uses a combination of methods: removing any features that occur in
fewer than a third of texts and transforming the remaining features’ standardised
normalised counts.

<footnote n="53">⁵³ The following function was applied to the standardised normalised counts: `signed.log <- function(x){sign(x)*log(abs(x)+1)}` (see <u>Online Appendix 7.4-7.5</u> for details of the procedure)</footnote>
<figure><img src="image_58.png" /><figcaption>Fig. 52: Distribution of signed log transformed standardised normalised frequencies of occurrence of five features across the TEC (histograms) and visualisations of their correlation (scatterplots)</figcaption></figure>

Overall, the factorability of the data depends on both the number and size of its variable intercorrelations. Bartlett's test of sphericity is often used to test whether variables are sufficiently intercorrelated to produce representative factors; however, it is a significance test of the hypothesis that the correlations in a correlation matrix are zero and, as such, is known to be overly sensitive and dependent on the sample size (Hair et al. 2019: 136) so that, in practice, it will always produce significant results in the context of MDAs carried out on sufficiently large corpora. As formulated in the null hypothesis that it is designed to test, Bartlett's test of sphericity merely indicates the presence of non-zero correlations, which is not to say that the pattern of these correlations is actually suitable for factor analysis (see also Hair et al. 2019: 168). In addition, the test assumes that the data is a sample from a multivariate normal population which is rarely, if ever, the case when dealing with linguistic data (Lee 2000: 178). Thus, for MDA studies with many data points and non-normal
variables, it is wiser to first examine feature intercorrelations visually (see, e.g., Fig. 53, see also Neumann & Evert 2021). This can help to identify both extremely high correlations (collinear variables) and very low ones that can skew the results of the MDA and therefore ought to be excluded. In the present thesis, collinear features are defined as those correlating > |0.95|. Whenever this is the case, the less marked of the two collinear variables is excluded from the analysis. For example, in the second MDA presented in this chapter (7.3.2), the present tense (VPRT) and past tense (VBD) variables correlated at -0.96, leading to VPRT being removed from the dataset. In addition, the Kaiser-Meyer-Olkin (KMO) index (1974: 112) is used to further explore the suitability of the feature intercorrelations for factor analysis. In the present thesis, this is achieved using the R psych::KMO function (Revelle 2020) which outputs an overall KMO Measure of Sampling Adequacy (MSA), as well as MSA scores for each individual feature. These can range from 0 (i.e., not in any way correlated with another feature) to 1 (indicating that this feature can be perfectly predicted by another feature) (Kaiser & Rice 1974). Following the procedure described in Hair et al. (2019: 136–137), the features’ individual MSA values are examined and, if any feature has an MSA of ≤ 0.5, the feature with the lowest MSA is removed. The KMO index is then re-calculated and this process of omitting the variable with the lowest MSA value is continued until all features reach an MSA value of ≤ 0.5.

An additional step that is often taken to ensure that the results of factor analysis methods are robust consists in removing variables with low final communalities from the analysis. As Lee (2000: 244ff.) points out, this is not something that Biber (1988) did (though Biber 1995 shows awareness of the issue)<fnref n="54" />. Communality is measured as the sum of all the squared factor/component loadings for any one variable and therefore refers to the proportion of variance within a variable that is explained by the extracted factors. In other words, a low communality indicates that a substantial proportion of a variable’s variance is not accounted for by the reduced solution. There are no hard and fast rules as to what constitutes a reasonable communality cut-off point because it very much depends on how much total variance a solution explains; however, in the context of MDA, Biber (1995: 138) recommends eliminating linguistic features with communalities ≤ 0.20. This is also the cut-off point that is used in the present chapter. As Lee (2000: 246–247) points out, many features with low final communalities are also low-frequency features.

Once these various steps have been undertaken to eliminate very unevenly distributed features, those with overly high or particularly low correlations, and low communalities, the overall MSA can be re-calculated to evaluate the suitability of the

<footnote n="54">⁵⁴ According to Lee (2000: 244ff), eight of Biber’s (1988) features had final communalities of ≤ 0.20 yet were still retained in the final model.</footnote>
dataset for this kind of analysis. The resulting overall KMO values may be interpreted
following Kaiser & Rice's (1974: 112) wonderfully flamboyant approximate scale:

≥ .90 marvellous
≥ .80 meritorious
≥ .70 middling
≥ .60 mediocre
≥ .50 miserable
< .50 unacceptable

Arguably more meaningfully, KMO values may be compared to those of previous MDA studies. Unfortunately, however, few MDA studies report these: out of the 230 MDA studies that Goulart & Wood (2021: 124) surveyed, 26 claim to have checked the factorability of their data using KMO and 24 report exact overall KMO values. For these, Goulart & Wood (2021: 124) calculate a mean KMO value of 0.69 (SD = 0.08, min = 0.43, max = 0.86), which would suggest that correlation matrices typically entered in MDAs are only “mediocrely” to “middlingly” suitable for factor analysis. However, all of the suggested methodological advancements on the traditional MDA framework outlined so far, in particular the removal of “obvious” correlations, the elimination of highly unevenly distributed features, those that have low MSA scores or low communalities, and the transformation of particularly skewed distributions ought to contribute to higher overall KMO values and to correlation matrices that are more suitable for this kind of data-reduction analysis.
<figure><img src="image_59.png" /><figcaption>Fig. 53 Correlation matrix of the signed log standardised relative frequencies of the features analysed in the second PCA-based MDA (see 7.3.2)</figcaption></figure>

### 7.2.7 Arbitrary thresholds in the computation of dimension scores

The MDA framework also foresees the computation of dimension scores (sometimes also referred to as 'factor scores') for each text in the corpus under study. Mean dimension scores for specific (sub-)registers or other subgroups of the corpus can then be compared. As explained in 6.2.4, Biber (1988: 93) and most other linguists following his method exclude features with factor loadings below a pre-determined cut-off point from the computation of their dimension scores (as in Biber's original MDA study, most use ±0.35). In addition, if a feature loads onto more than one factor with a loading above the chosen cut-off point, the feature is only counted for the factor on which it has the highest loading. To calculate the dimension scores that correspond to a particular factor, the standardised normalised frequencies for each of the salient positive-loading features on that factor are added together whilst the salient negative-loading ones are subtracted.
Biber's MDA framework has been criticised for relying on an arbitrary cut-off point to include or exclude certain features from its dimension scores (e.g., Evert 2018: 12). There is, however, a valid rationale for removing low-loading features from dimension scores: they are likely to simply reflect noise in the data that is arguably best removed from models that aim to be representative of a larger population. Statistical methods can be applied to exclude non-significant loadings, i.e., those that are likely to be the result of random patterns of variation in the data (Husson et al. 2018: 220). However, with large data sets such as those typically used in MDA studies, such significance tests are likely to return extremely low thresholds. In relying on a cut-off point of ±0.35, Biber (1988: 93) applied a slightly more conversative version of the common threshold in social sciences of ±0.30. It has the advantage of excluding loadings that, whilst perhaps statistically significant, may not have any practical relevance as they account for less than 12.25% (= 0.35²) of the shared variance (Lee 2000: 207).

Biber's exclusion of features contributing to more than one dimension has also been criticised. On the one hand, it has the advantage of making the dimension scores "experimentally independent, as each feature contributes to only one dimension" (Lee 2000: 209; see also Biber 1988: 93). However, it clearly adds a degree of arbitrariness: for instance, in Biber's (1988) model, past participle WHIZ deletions contribute to Dimension 5 scores, but not to Dimension 1 scores, even though in Biber's rotated factor solution, past participle WHIZ deletions contributed to a very similar extent to Factor 1 (-0.39) and Factor 5 (0.43). Moreover, Biber (1988: 85) advocated for the use of oblique rotation because linguistic dimensions can reasonably be expected to intercorrelate given that "from a theoretical perspective, all aspects of language use appear to be interrelated to at least some extent" (Biber 1988: 85 fn.). If this is the case, it should come as no surprise that individual linguistic features may also make significant contributions to more than one dimension.

Finally, the method that Biber and others following the MDA framework have traditionally used to calculate dimension scores essentially involves dichotomising all feature loadings: all features that have loadings above the chosen cut-off point (while not contributing more to another factor) contribute equally to the dimension scores, whilst those that are excluded from the dimension scores do not contribute at all. The feature contributions are therefore equal to either one or zero. This means that all loadings above the cut-off point are considered equally important, even though they may actually have made substantially different contributions to the original factor solution. For example, in Biber's (1988) model, the standardised frequencies of past tense verbs and present participial clauses are treated as equally important contributors to Dimension 2 scores, even though their factor loadings are widely different (0.90 vs. 0.39). Hence, this approach can be argued to grant less salient linguistic features disproportionate significance. At the same time, however, such a dichotomous approach to calculating dimension scores is not without its advantages.
In non-linguistic uses of factor analysis, discarding the relative importance of features has been shown to distort results only marginally. In fact, dichotomisation increases the chances that the resulting dimension scores may be replicated with a new sample of texts because it essentially removes some of the random noise inherent to small differences between factor loadings (Gorsuch 2014: 275–276).

As mentioned in 7.2.5, with PCA, the component feature loadings are correlation coefficients between the observed features and the components and are therefore much simpler to interpret than the factor loadings computed in factor analyses. Nonetheless, the question still arises as to how much relevant information the exact feature loadings contribute to any component, whether a cut-off point is needed and, if so, which one. In effect, three solutions to calculate dimension scores can be envisaged. The first solution simply consists in using the loadings as they are. In other words, on any one component, the standardised normalised feature frequencies of any one text are multiplied by their respective loadings on this component and these values are added to compute dimension scores. The second consists in applying a cut-off point to exclude low-loading features whilst retaining the other loadings as multiplying factors to calculate the dimension scores. Finally, the third solution is the one typically adopted in MDA studies: as explained above, it consists in dichotomising loadings according to a cut-off point. With a cut-off point of ±0.30, this would mean that to calculate dimension scores only the unweighted standardised normalised features with loadings of ≥ 0.30 or ≤ -0.30 are added (they are in effect multiplied by one, whilst those with loadings between -0.30 and 0.30 are multiplied by zero).

If the aim of an MDA is to produce a model of linguistic variation that is generalisable
beyond the sample under study, as was presumably the case with Biber (1988), then
solution three may well be the wisest. However, the potential issues it causes
downstream should not be downplayed. For a start, the resulting dimension scores no
longer correlate perfectly with the factors/components they purport to quantify. This
is why Lee (2000: 211) argues that they should really be referred to as “estimates”
rather than “scores” (see also Child 1990). Moreover, the reported R² are no longer
true. Hence, whilst Biber’s (1988) Factor 1 accounts for 26.8% of the shared variance
(Biber 1988: 82–83), this is not true of Dimension 1 that explains considerably less
due to the loss of information caused by the dropping of low-loading features, the use
of dichotomous loading weights (1 or 0) for the remaining features, and the exclusion
of features with significant loadings on more than one factor. Additionally, whilst we
have seen that having a cut-off point is not necessarily a bad idea, it nevertheless
involves an arbitrary decision that will inevitably exert substantial influence on the
resulting dimension scores and can therefore potentially be a potential source of bias
(see also Evert 2018). In light of these issues, the first solution was chosen. For the
reasons explained above, the MDAs presented in this chapter also allow linguistic
features to contribute to more than one dimension. Whilst this may somewhat
complicate the interpretability of the resulting dimensions scores, it is in line with
our understanding of communicative processes in which linguistics features are
expected to intercorrelate in many ways. The chosen solution bears the advantage of
not relying on an arbitrary cut-off point, maintains the true correlations of features
to dimensions and thus does not distort the PCA solution. Since the loadings
themselves act as factors in the computation of the dimension scores, the amount of
noise added by low-loading features is assumed to be negligible.

7.2.8 Misleading visualisation of results

To date, the results of MDAs have almost exclusively been visualised by plotting the
mean dimension score of (sub-)registers on a vertical line representing a dimension
cline with the most negative-scoring text categories placed at the bottom of the line
and the highest scoring ones at the top.<fnref n="55" /> This is how Biber first visualised the results
in his seminal 1988 MDA study (see Fig. 29 for a reproduction of such a plot). This
section explains why such visualisations are not satisfactory.

First, plotting only mean dimension scores entirely ignores intra-category variability. It gives no indication of the range of dimension scores covered by a single (sub-)register category and tacitly suggests that the distributions of dimension scores within each category are all normal. In fairness, most MDA studies do report standard deviations alongside these mean dimension scores – however, usually only in tabular form, which makes it very difficult to grasp how much overlap there is between different (sub-)register categories. Whilst it is perfectly understandable that digital plotting methods were more limited in the late 1980s, such dimension plots are still widely used in MDA publications today. For example, Gardner et al. (2019) present the results of an MDA study on learner academic writing with plots such as Fig. 54. An improvement on Biber’s (1988) dimension plots is that, on Fig. 54, the number of texts that constitutes each category is printed in brackets; however, the plot nonetheless provides no indication of the distribution shape of dimension scores in each category. Furthermore, it is impossible to gauge whether any of the observed differences between the categories are likely to be statistically significant. This is why, in Chapter 6, the dimension scores of each individual text in the corpora were plotted, thus helping the reader to easily evaluate and compare the number of texts in each category, as well as to detect outliers. These ‘raincloud plots’ (see, e.g., Fig. 31) also include boxplots for ease of comparison between the different categories: these display the category median value (which is less susceptible to outliers than the mean) and the interquartile range, which, in combination with the individual text points, gives a good impression of the dispersion of the texts along any one dimension. Given “the centrality of text in corpus-linguistic inquiries” (Biber 2021: n.p.) and the fact that

<footnote n="55">⁵⁵ Some studies (e.g., Lee 2000) have visualised the results of MDA using barplots of mean dimension scores, which is arguably even more misleading.</footnote>
the MDA framework applies a text-linguistic research design (Biber et al. 2016: 357), being able to visualise the position of each text on any one dimension is crucial.

## Compressed Procedural Information

<table><tbody><tr><td>+10</td><td>Food Science (n=124) 9.93; Chemistry (n=89) 9.73</td><td>Methodology Recount (n=347) 10.15</td></tr><tr><td>+9</td><td>Engineering (n=238) 9.04</td><td></td></tr><tr><td></td><td>Meteorology (n=29) 8.78</td><td></td></tr><tr><td>+8</td><td></td><td>Design Specification (n=89) 8.15</td></tr><tr><td>+7</td><td></td><td></td></tr><tr><td>+6</td><td>Cybernetics (n=28) 6.58</td><td></td></tr><tr><td>+5</td><td>Biology (n=169) 5.91; Physics (n=68) 5.21</td><td></td></tr><tr><td></td><td></td><td>Exercise (n=102) 4.92</td></tr><tr><td>+4</td><td>Computer Science (n=87) 4.26</td><td>Explanation (n=195) 3.99</td></tr><tr><td></td><td>Planning (n=14) 3.81; Architecture (n=9) 3.73</td><td>Proposal (n=71) 3.34</td></tr><tr><td>+3</td><td>Agriculture (n=134) 3.69</td><td>Research Report (n=61) 3.14</td></tr><tr><td></td><td>Economics (n=96) 2.89</td><td></td></tr><tr><td>+2</td><td></td><td>Case Study (n=189) 2.39</td></tr><tr><td>+1</td><td>Mathematics (n=33) 1.66; Medicine (n=80) 1.39</td><td></td></tr><tr><td></td><td></td><td>Empathy Writing (n=32) 1.65</td></tr><tr><td></td><td>Business (n=146) 0.10</td><td>Critique (n=315) 0.62</td></tr><tr><td></td><td>Archaeology (n=76) -0.57</td><td>Literature Survey (n=35) 0.42</td></tr><tr><td>0</td><td></td><td></td></tr><tr><td>-1</td><td>Hospitality, Leisure & Tourism Management (n=92) -0.77</td><td></td></tr><tr><td></td><td>Health (n=81) -1.90</td><td></td></tr><tr><td>-2</td><td>Publishing (n=30) -2.24</td><td></td></tr><tr><td></td><td>Anthropology (n=49) -2.80</td><td>Problem Question (n=39) -2.70</td></tr><tr><td>-3</td><td>Psychology (n=95) -3.61</td><td></td></tr><tr><td>-4</td><td>Linguistics (n=115) -4.39; Law (n=134) -4.54</td><td>Narrative Recount (n=64) -4.38</td></tr><tr><td></td><td>Sociology (n=110) -4.75</td><td></td></tr><tr><td>-5</td><td></td><td>Essay (n=1221) -5.15</td></tr><tr><td></td><td>Politics (n=110) -5.79</td><td></td></tr><tr><td>-6</td><td>Comparative American Studies (n=74) -6.78</td><td></td></tr><tr><td></td><td>History (n=95) -6.89</td><td></td></tr><tr><td>-7</td><td></td><td></td></tr><tr><td></td><td>English (n=106) -7.38</td><td></td></tr><tr><td>-8</td><td></td><td></td></tr><tr><td>-9</td><td></td><td></td></tr><tr><td>-10</td><td>Classics (n=82) -10.53</td><td></td></tr><tr><td></td><td>Philosophy (n=106) -10.67</td><td></td></tr></tbody></table>

## Stance towards the Work of Others

<caption>Fig. 54: Dimension 1 mean scores for disciplines (left) and genre families (right) from Gardner et al.</caption>

The second issue, common to plots such as Fig. 54 and the raincloud plots of Chapter 6, is that they encourage the reader to only consider linguistic variation on a single dimension at a time. In the case of an additive MDA that compares the dimension scores of previously confirmed individual dimensions, as in Chapter 6, this
is not necessarily a problem. If, however, the results of MDAs are to be genuinely
understood and interpreted as multi-dimensional, it is crucial that the position of
texts be compared along all the potentially relevant dimensions. In practice, of course,
humans struggle to visualise more than two or three dimensions at once. That said,
facetted plots such as Fig. 57 can be used to visualise clusters of texts in all the
possible combinations of dimensions. On these, however, small differences between
text categories are very difficult to discern. This is why this chapter makes extensive
use of 2- and 3-D plots that display the position of individual texts on two or three
dimensions at once.

Similarly, most MDA studies report lists of salient features and their loadings on
individual dimensions in tabular format (e.g., Table 40). Not only do these tables
usually exclude the features with loadings below the arbitrarily chosen cut-off point
(see 7.2.7), they also do not readily allow for any comparisons of loadings across
different dimensions. By contrast, the tables of loadings in this chapter include all
the features entered in the analyses. Cell shading and font colours are used to
highlight the most important contributions (see Tables 68 and 69). Furthermore,
graphs of features (e.g., Fig. 59) are used to visualise the linguistic features that
contribute most to the position of texts on biplots (more on how to interpret these in
7.3.1). Again, these graphs allow for a better visualisation of the extent to which some
features make salient contributions to more than one dimension.

Thus, in terms of visualisation, too, the method proposed in this chapter bears many
similarities with Neumann & Evert's (2021) geometric multivariate analysis (GMA)
approach which is, in turn, inspired by Biber's MDA framework, "but takes a more
geometric perspective emphasizing the visualization of individual texts in
multidimensional feature space" (Neumann & Evert 2021: 148). Linguistic differences
between texts are analysed by examining the Euclidean distances between texts in
this multi-dimensional feature space.

7.2.9 Difficulties in establishing the statistical significance and
robustness of the results

Evert (2018) suggests using bootstrapping and/or cross-validation as a means of
assessing the significance of the results emerging from PCAs. Following a similar
logic, Lee (2000: 346–369) ran EFAs on subgroups of his data to check the robustness
of the results. In the present chapter, the PCAs will be re-run on random subsets of
two-thirds of the data for the same reason.

Issues pertaining to the way in which the significance of the register differences
between dimension scores emerging from MDA studies are usually tested have already
been addressed in 6.2.6. It explained why, in both Chapter 6 and the present chapter,
linear mixed-effects models were computed to compare the dimension scores of the
results of the MDAs to account for multiple variables that may contribute to linguistic variation (register and textbook proficiency level), the fact that interactions of these variables may also make significant contributions, and for the non-independence of texts sourced from the same textbook series, novel, or website (see Candarli 2021 for a similar approach with nested data in a learner corpus). The approach described in 6.2.6 is also followed in the present chapter.

### 7.2.10 Issues related to reproducibility and replicability

Though the terms are sometimes used interchangeably and different (at times incompatible) definitions abound, in computational sciences, ‘reproducibility’ usually refers to the ability to obtain the same results as an original study using the authors’ data and code, whilst ‘replicability’ refers to obtaining compatible results with the same method but different data (Association for Computing Machinery 2020; see Berez-Kroeker et al. 2018 for similar definitions in the context of linguistic studies).<fnref n="56" /> As Popper wrote as early as 1935, “non-reproducible single occurrences are of no significance to science” (English edition from 1959: 86). In other words, sound science requires the results of studies to be reproducible (providing that the data can ethically and legally be made available) or at least conceptually replicable. In other words, if:

a scientist publishes the results of an experiment, there should be enough of the methodology published with the results that a similarly-equipped, independent, and skeptical scientist could reproduce the results of the experiment in their own lab (Gezelter 2009: n.p.).

In the conceptual sense of replication, issues pertaining to the robustness of the results (see 7.2.9) are directly connected to those related to the replicability of the analyses. Of course, it goes without saying that, ideally, direct replication of MDA studies (reproducibility) also ought to be possible. For the vast majority of MDAs, even those analysing publicly available corpora such as Brown, LOB or the BNC1994, this is currently not the case because the most popular software for tagging and counting linguistic features for English MDA studies, the Biber Tagger (Biber 1988; 2019), is not (at the time of writing<fnref n="57" />) available to the wider research community, either under an open-source licence or commercially. Though the explanations of its algorithms in Appendix II of Biber’s 1988 monograph have rightly been praised for their comprehensiveness, when Nini (2014; 2019) attempted to replicate the Biber Tagger, he came across a number of unclear cases. Though the MAT (see 6.2.3) has been shown to produce results highly comparable to the Biber Tagger in its 1988 version, unexplained differences nonetheless remain. Moreover, Biber has continued to improve the tagger with “several major rounds of revision and extension, resulting in

<footnote n="56">⁵⁶ Note that other terms are also frequently used to refer to the same or related concepts, e.g., *repeatability*, *robustness* and *generalisability* (cf. Belz et al. 2021: 2–3).</footnote>

<footnote n="57">⁵⁷ Some of Douglas Biber’s collaborators have been working on a new version of the Biber Tagger (to be named differently) with the aim of making available via a “semi-public” online interface (Douglas Biber, personal communication 2018).</footnote>
the analysis of a much more comprehensive set of linguistic features” (Biber 2019: 14) for which no details of the algorithms are publicly available. Thus, to ensure that the results of an MDA are reproducible, it is crucial that the computer programme(s)/the algorithms used to tag and count the linguistic features entered in the MDA are published or made available on request. This is also one of the reasons why McEnery & Hardie (2011: 112) conclude that replicability remains “something of a concern for the MD [multi-dimensional analysis] framework”.

The Multi-Feature Tagger of English (MFTE) used in the analyses of the present chapter has been released under a GNU General Public licence (see Online Appendix 7.1) and can therefore be scrutinised by the wider research community. Whilst the corpora cannot be released for copyright reasons, the full tabular results of counts as output by the MFTE are included in the Online Appendix 7.2–7.7, as well as the full code to reproduce the PCA results, the statistical tests, and all the figures on the basis of this count data. This is important because, for all the complexity of advanced statistical methods such as those typically used in MDAs, it is worth remembering that such methods nevertheless require researchers to make countless (as we have seen, often rather arbitrary and certainly always subjective) decisions on a host of parameters. Given that “there is nothing sacred about one particular factor [or component] solution or one grouping of features” (Lee 2000: 370), it is crucial that each parameter choice be transparent, and that the robustness of the results can be checked by independent researchers. In addition, and as recommended by Lee (2000: 393; see also Biber 1990), the present results were replicated on various subsets of the data to test the replicability and robustness of the models presented and interpreted in the present chapter (see 7.4).

## 7.3 Results

As in Chapter 6, register variation within the TEC is first explored in 7.3.1 before the three main textbook registers, Conversation, Fiction, and Informative texts, are mapped against the three corresponding reference corpora in 7.3.2.

### 7.3.1 Linguistic variation within the TEC

This section focuses on register variation within secondary school EFL textbooks. The texts of the TEC (as defined in 7.2.1) were all tagged with the MFTE (see 7.2.2–7.2.5). Of the tagger’s three outputs, the complex normalisation table was used as the basis for the PCAs of this section (see 7.2.4). Three features were removed: CD (cardinal numbers) because numbers had to be removed from most textbook texts due to the presence of line numbering and footnote numbers for glosses, as well as the variables LIKE and SO because these categories were created to increase the precision and recall of other linguistic features and cannot be meaningfully functionally interpreted (Le Foll 2021e: 17–18, 33–34). Next, applying the feature
exclusion procedure described in 7.2.6–7.2.7, the counts for the BE (un)able to
construction (ABLE) were merged with the category of predicative adjectives (JJPR),
whilst the counts for passive GET constructions (PGET) were added to the BE passive
counts to create a more general passive category (PASS) because both categories were
absent from more than two-thirds of texts. Other features which were also only
observed in a third or fewer of TEC texts but could not be meaningfully subsumed
with any other features were excluded from this MDA. These are: CONC, DWNT,
ELAB, EMO, GTO, HGOT, HST, MDMM, PRP, QUTAG and URL (see Appendix I
for the full table of features and the code in the *Online Appendix 7.4* for details of
the procedure). The iterative process to arrive at individual feature MSA values of
> 0.5 (described in 7.2.6) led to the exclusion of one additional feature: MDWS.
Finally, four features were removed on the basis of their low final communalities:
STPR, MDNE, HDG and CAUSE. In addition, sixteen outlier texts were removed on
the basis of some extremely high feature counts (see code in *Online Appendix 7.4* for
details).

The following PCA is therefore based on a matrix of 1,961 texts by 61 features, all z- and signed log-transformed (see 7.2.6), with a satisfactorily high overall KMO factor adequacy index of 0.88, or “meritorious” according to Kaiser & Rice (1974: 112). To determine the number of components to be considered in the analysis, a scree plot was first generated, see Fig. 55. It shows the amount of variation each component captures from the TEC data. The “elbow” method (Jolliffe 2002: 115–118) is difficult to apply here because the plot can be said to feature several “breaking points”. Following Biber’s (1988: 84) advice to extract more rather than fewer components to start off with, the first six components were originally retained for further analysis. Together, these account for 50.88% of the total variance.<fnref n="58" />

<figure><img src="image_60.png" /><figcaption>Fig. 55: Scree plot of the eigenvalues of the PCs for the TEC data</figcaption></figure>

<footnote n="58">⁵⁸ For reference, Biber’s (1988) EFA solution of seven factors accounted for 51.9% of the shared variance (Biber 1988: 83).</footnote>
The distribution of the texts on the first three components was first explored interactively in a 3-D visualisation computed using the {pca3d} R package (see Online Appendix 7.6 and snapshots in Fig. 56). Here, and on all subsequent scatterplots, every data point represents a single text from the TEC. The closer points are, the more linguistic similarities they share. At first sight, the most striking aspect of the 3-D visualisation is that there are two clearly separated clusters of texts: one consisting of instructional language (in yellow) and the other of the remaining textbook registers. Within this second, much larger cluster of texts, we find that conversation is concentrated at one end (in red) and informative texts (blue) at the other, with fiction (green) and personal correspondence (purple) interspaced in between. The 3-D visualisation makes clear that all three components contribute to distinguishing register-based intra-textbook variation.

<figure><img src="image_61.png" /><figcaption>Fig. 56: Snapshots from the 3-D visualisation of the first three dimensions of the multi-dimensional model of intra-textbook variation</figcaption></figure>

By contrast, the remaining three dimensions (PC4, PC5 and PC6) do not appear to distinguish between different textbook registers. This is illustrated in the biplot matrix of all combinations of the six retained dimensions in Fig. 57. The same colour scheme is used to encode the different registers as in the 3-D visualisations and, in addition, the proficiency levels of the textbooks from which each text stems are represented by different shapes: beginner textbook texts (level A) are assigned the circle shape, while the texts from the most advanced textbooks in the TEC (level E) are represented by diamonds (see Online Appendix 7.2 for zoomable version of Fig. 57).
<figure><img src="image_62.png" /><figcaption>Fig. 57: Scatterplot matrix of combinations of the first six dimensions of the model of intra-textbook variation (the number before the comma on each axis label shows</figcaption></figure>
which principal component is plotted on that axis; this is followed by the percentage of the total variance explained by that particular component)

Fig. 58 is a more fine-grained projection of the texts of the TEC on the first two dimensions which, together, account for the greatest linguistic differences between the different registers of the TEC. The ellipses represent the 95% confidence intervals around each of the five textbook register centroids. As already observed in the 3-D plot, two clusters of texts are evident: to the right of the plot, instructional texts form a tight cluster whose ellipse does not overlap with any of the other four textbook registers. Thus, we can conclude that instructional language has a very characteristic linguistic profile which clearly sets this register apart. The linguistic features that contribute most to this very specific profile can be seen in the top right panel of Fig. 59. 59, in the area of the plot that corresponds to the area where the cluster of instructional texts can be found in Fig. 59. They are, as illustrated in (215), imperatives (VIMP), verbs of communication (COMM), and verbs depicting mental processes (MENTAL).

(215) Look at the other groups' guides and choose which channel you would like to watch. Use the key phrases for making and justifying a choice. Work in pairs. Answer the questions. <TEC: Solutions pre-intermediate>

In addition, second person referents (SPP2) and WH-questions (WHQU) also feature in the upper-right panel of Fig. 59 and are thus very typical of instructional language, too, e.g., (216); however, these features are situated closer to the y-axis as they also make strong contributions to the positive end of the model's second dimension (PC2), which, much like Biber's (1988) Dimension 1 (see Fig. 29), corresponds to an involved vs. informational language continuum.

(216) Reactions

a) Describe **what happens** in the second half of the story (after line 43). How **do** the customers **react**? How **does** the narrator **react**?

b) Do you **understand** the way they **react**?

Short stories often **start** unusually ("medias in res" - right in the middle of the action) **and end** with a surprise. Look at "Deportation at breakfast" again: find these elements **and say why** they **are** important here. <TEC: Green Line 5>
<figure><img src="image_63.png" /><figcaption>Fig. 58: Projection of the texts of the TEC on the first and second dimensions of the model of intra-textbook variation</figcaption></figure>

<figure><img src="image_64.png" /><figcaption>Fig. 59: Graph of the features with the strongest contributions to the first and second dimensions of the model of intra-textbook variation</figcaption></figure>
The second, larger cluster on Fig. 58 reveals a clear register cline with textbook texts depicting conversations towards the top-left end of the cluster, fiction, and personal correspondence in the middle, whilst informative texts are concentrated at the bottom-right end of the cluster. The features that contribute most to this distribution of registers can be seen in the graph of features on Fig. 59. The advantage of this kind of biplot is that we can see that the features that are on or very close to one of the two axes contribute principally to just one of these first two dimensions, whilst those that draw diagonals contribute to both dimensions. Thus, Fig. 59 shows that the upper end of the large, “non-instructional textbook English” cluster is characterised by high frequencies of fillers and interjections (FPUH), markers of politeness (POLITE), discourse markers (DMA), verbal contractions (CONT) and present tense verbs (VPRT), e.g., (217). By contrast, the lower end of the cluster features texts with longer words (AWL), high frequencies of nouns (NN), prepositions (IN) and subordinate WH-clauses (WHSC), e.g., (218). These two extremes echo the features with the highest estimated factor loadings on the two ends of Biber’s (1988) ‘Involved vs. Informational Production’ dimension (see Table 40).

(217) Can I help you?
Yes, have you got the new ‘Pets’ magazine, please? I can’t find it.
It’s there - next to the sports magazines.
Excuse me. Where can I try on this sweatshirt?
There, on the left.
Thanks. I like the colour, but the size isn’t right.
No problem. We’ve got other sizes, too. <TEC: Green Line 1>

(218) The Aboriginal Memorial is an installation of 200 hollow log coffins from Central Arnhem Land. Artists made it to commemorate all the indigenous people who, since 1788, have lost their lives defending their land. Visitors can see it in the National Gallery of Australia. The artists said the museum authorities must locate this installation in a public place where they could preserve it for future generations.
<TEC: New Missions 2<sup>de</sup>>

The large cluster's slanted shape on Fig. 58 indicates that both the first (PC1) and second (PC2) dimension of this multi-dimensional model of intra-textbook variation capture important aspects of register-based or situational variation. The biplot of the first two dimensions, Fig. 58, clearly shows that many linguistic features (foremost those plotted on Fig. 59) are distributed quite differently across at least three out of the five textbook registers under study: this is illustrated on Fig. 58 by the fact that instructional language forms its own very distinct cluster, and the ellipses of the conversational and informative texts overlap very little. The ellipses for the fiction and personal correspondence texts, however, overlap much more, suggesting that these two textbook registers are not readily distinguishable on these first two dimensions (though see 7.3.1.1 and 7.3.1.2), which brings us to the third and fourth dimensions.
<figure><img src="image_65.png" /><figcaption>Fig. 60: Projection of the texts of the TEC on the third and fourth dimensions of the model of intra-textbook variation</figcaption></figure>

<figure><img src="image_66.png" /><figcaption>Fig. 61: Graph of the features with the strongest contributions to the third and fourth dimensions of the model of intra-textbook variation</figcaption></figure>
Fig. 60 displays the positions of the texts of the TEC on the third (PC3) and fourth dimensions (PC4). The intersection of these two dimensions highlights a distinctive linguistic profile for at least some of the fiction texts of the TEC. Indeed, part of the green ellipse is set apart from the rest of the texts. The features that contribute most to this characteristic linguistic profile can be found in the top right panel of corresponding graph of features on Fig. 61. Just like on the narrative end of Biber's Dimension 2, the frequency of past tense verbs (VBD) and third-person references (TPP3S) make the largest contributions to this characteristically “narrative” cluster.

A closer look at the shapes of the points in this non-overlapping portion of the textbook fiction ellipse in the top-right panel of Fig. 60 reveals that it is foremost composed of narrative texts from intermediate to advanced textbooks (levels C to E). To explore this further, Fig. 62 displays the texts of the TEC on the same two dimensions as in Fig. 60 but this time the colour scheme and the ellipses correspond to the proficiency levels of the textbooks from which the texts have been extracted, rather than the register of each text (which is, instead, coded by the shapes of the points). A comparison of the two biplots (Fig. 60 and Fig. 62) shows that, whilst register-based variation is greater, textbook proficiency level also makes some notable contributions to linguistic variation in Textbook English, as evident on the third and fourth dimensions. These different factors contributing to linguistic variation in the TEC will be tested in more in-depth analyses of the first four dimensions in 7.3.1.1–7.3.1.4.

The fifth and sixth dimensions (PC5 and PC6), however, will not be examined any further as they account for comparatively little of the total variance (PC5 = 3.18% and PC6 = 2.64%). Both the visualisations (Fig. 63 and the 3-D projections of PC4–PC6 in the *Online Appendix 7.6*) and the mixed-effects models conducted to explore these dimensions (see *Online Appendix 7.6*) indicate that they contribute very little to differentiating between different text registers, proficiency levels, or textbook series.
<figure><img src="image_67.png" /><figcaption>Fig. 62: Projection of the texts of TEC on third and fourth dimensions with colours and ellipses indicating the proficiency level of the textbooks (as opposed to register as in Fig. 60)</figcaption></figure>

<figure><img src="image_68.png" /><figcaption>Fig. 63: Projection of the texts of the TEC on the fifth and sixth dimensions of the model of intra-textbook variation</figcaption></figure>
### 7.3.1.1 Variation along Dimension 1: ‘Overt instructions and explanations’

As we saw in Fig. 56–58, the first dimension to emerge from this PCA-based model of intra-textbook variation primarily separates instructional texts and explanations from the rest of the TEC data. Given that the negative end of the dimension does not imply any specific text quality other than being the least “instructional-like”, interpreting the dimension with a bipolar label would be potentially misleading (Bohmann 2017: 326); hence only the positive end of the dimension will be labelled: ‘Overt instructions and explanations’.

As in Chapter 6, for each dimension, linear mixed-effects models were computed to quantify the extent to which register, textbook proficiency level and the individual styles of textbook authors and publishers (as very approximately captured by the textbook series variable) contribute to each textbook texts’ location on each principal component.<fnref n="59" /> For the first dimension, the model featuring only Register as a fixed effect already explains 88% of the total variance in PC1 scores. Adding the nine textbook series as random, varying intercepts only very marginally increases the R² value to 90%, thus indicating that register is a remarkably strong predictor of PC1 scores, whereas the textbook series variable does not make a significant contribution to PC1 scores (see also plots of random effects in the *Online Appendix 7.6*). The anova-based comparison of the PC1 models showed that, whilst modelling Register*Level interactions provides a significantly better fit (as measured using AIC), only three interactions are significant at the level of *p* < 0.01. As shown in Table 66, these are: Instructional register with the textbook proficiency levels C, D and E.

<table><caption>Table 66: Summary of the model: lmer(PC1 ~ Register + Level + Level*Register + (1|Series))</caption><thead><tr><th>Predictors</th><th>Estimates</th><th>CI</th><th>p-value</th></tr></thead><tbody><tr><td>(Intercept) [Conversation, Level A]</td><td>-2.37</td><td>-2.59 - -2.15</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Fiction]</td><td>1.61</td><td>1.36 - 1.87</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Informative]</td><td>2.23</td><td>1.96 - 2.50</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Instructional]</td><td>5.29</td><td>5.10 - 5.47</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Personal]</td><td>0.48</td><td>0.08 - 0.88</td><td><strong>0.019</strong></td></tr><tr><td>Level [B]</td><td>-0.12</td><td>-0.30 - 0.05</td><td>0.167</td></tr><tr><td>Level [C]</td><td>0.12</td><td>-0.05 - 0.29</td><td>0.159</td></tr><tr><td>Level [D]</td><td>0.23</td><td>0.06 - 0.41</td><td><strong>0.01</strong></td></tr><tr><td>Level [E]</td><td>0.27</td><td>0.07 - 0.48</td><td><strong>0.01</strong></td></tr><tr><td>Register [Fiction] * Level [B]</td><td>0.18</td><td>-0.15 - 0.51</td><td>0.284</td></tr><tr><td>Register [Informative] * Level [B]</td><td>0.36</td><td>0.02 - 0.70</td><td><strong>0.038</strong></td></tr><tr><td>Register [Instructional] * Level [B]</td><td>-0.10</td><td>-0.35 - 0.15</td><td>0.434</td></tr></tbody></table>

<footnote n="59">⁵⁹ Detailed statistics and additional plots of all the models computed as part of this chapter can be found in the *Online Appendix 7.6*.</footnote>
<table><tbody><tr><td>Register [Personal] * Level [B]</td><td>0.11</td><td>-0.39 – 0.61</td><td>0.671</td></tr><tr><td>Register [Fiction] * Level [C]</td><td>-0.25</td><td>-0.58 – 0.07</td><td>0.13</td></tr><tr><td>Register [Informative] * Level [C]</td><td>0.00</td><td>-0.32 – 0.31</td><td>0.993</td></tr><tr><td>Register [Instructional] * Level [C]</td><td>-0.39</td><td>-0.62 – -0.15</td><td><strong>0.001</strong></td></tr><tr><td>Register [Personal] * Level [C]</td><td>-0.22</td><td>-0.72 – 0.28</td><td>0.381</td></tr><tr><td>Register [Fiction] * Level [D]</td><td>-0.05</td><td>-0.38 – 0.27</td><td>0.739</td></tr><tr><td>Register [Informative] * Level [D]</td><td>-0.01</td><td>-0.33 – 0.31</td><td>0.946</td></tr><tr><td>Register [Instructional] * Level [D]</td><td>-0.47</td><td>-0.72 – -0.23</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Personal] * Level [D]</td><td>-0.07</td><td>-0.60 – 0.46</td><td>0.8</td></tr><tr><td>Register [Fiction] * Level [E]</td><td>-0.24</td><td>-0.58 – 0.10</td><td>0.173</td></tr><tr><td>Register [Informative] * Level [E]</td><td>0.06</td><td>-0.29 – 0.40</td><td>0.747</td></tr><tr><td>Register [Instructional] * Level [E]</td><td>-0.50</td><td>-0.77 – -0.22</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Personal] * Level [E]</td><td>-0.18</td><td>-0.74 – 0.38</td><td>0.527</td></tr><tr><td colspan="4"><strong>Random Effects</strong></td></tr><tr><td>σ<sup>2</sup></td><td colspan="3">0.45</td></tr><tr><td>τ<sub>00 Series</sub></td><td colspan="3">0.07</td></tr><tr><td>ICC</td><td colspan="3">0.14</td></tr><tr><td>N<sub>Series</sub></td><td colspan="3">9</td></tr><tr><td>Observations</td><td colspan="3">1961</td></tr><tr><td>Marginal R<sup>2</sup> / Conditional R<sup>2</sup></td><td colspan="3">0.890 / 0.906</td></tr></tbody></table>

Detailed inspection of the mean log z-scores of the linguistic features with high absolute loadings on PC1 (see Fig. 59 and Table 68) across the five textbook proficiency levels revealed that these significant Instructional*Level interactions are due to the number of imperative verbs (VIMP) featured in instructions and explanations progressively decreasing as textbook proficiency level increases, whilst the number of present tense verbs increases. Two reasons explain this. First, textbook instructions become more complex as textbook authors expect learners’ proficiency in English to increase. This is illustrated in extracts (219) and (220), which stem from a level A and a level E textbook and which, on PC1, score 3.89 and 0.63 respectively. Second, secondary school beginner textbooks tend to include far fewer explanations in English, preferring to explain, e.g., grammatical concepts in the students’ L1/school language. This means that the level A and B instructional texts of the TEC include fewer explanations than levels C, D and E textbooks. That said, whilst these three Instructional*Level interaction terms are significant and interpretable, the estimated differences in PC1 scores remain small and the marginal R² value of 89% and conditional R² of 90.6% of the model summarised in Table 66 make clear that, as compared to the model that only included Register as a fixed effect, the impact of textbook proficiency level on PC1 scores is only very marginal.

(219) Identify the people on the photograph. Look and describe what you can see. Compare the people. Listen and describe the characters' families. Use the genitive. <TEC: Piece of Cake 6e>
(220) Reactions

a) Describe what **happens** in the second half of the story (after line 43). How **do** the customers react? How **does** the narrator react?

b) Do you understand the way they react?

Short stories often **start** unusually ("medias in res" - right in the middle of the action) and **end** with a surprise. Look at "Deportation at breakfast" again: find these elements and say **why** they **are** important here. <TEC: Green Line 5>

The lack of overlap in the confidence intervals on Fig. 64 and the figures in Table 67 show that all of the register differences in estimated mean PC1 scores are significant, which confirms that PC1 distinguishes remarkably well between the different registers of the TEC data. Since PC1 accounts for 21.08% of the total variance in the TEC data, this confirms that much of the linguistic variation in Textbook English is register-driven.

<table><caption>Table 67: Estimated differences between mean PC1 scores for each TEC register pair (averaged across all textbook levels and series)</caption><thead><tr><th>Contrast</th><th>Estimate</th><th>SE</th><th>df</th><th>t-ratio</th><th>p-value</th></tr></thead><tbody><tr><td>Conversation - Fiction</td><td>-1.55</td><td>0.05</td><td>1962.95</td><td>-30.53</td><td>&lt;.001</td></tr><tr><td>Conversation - Informative</td><td>-2.34</td><td>0.05</td><td>1960.92</td><td>-50.34</td><td>&lt;.001</td></tr><tr><td>Conversation - Instructional</td><td>-4.99</td><td>0.04</td><td>1961.12</td><td>-125.14</td><td>&lt;.001</td></tr><tr><td>Conversation - Personal Correspondence</td><td>-0.41</td><td>0.08</td><td>1958.34</td><td>-5.13</td><td>&lt;.001</td></tr><tr><td>Fiction - Informative</td><td>-0.79</td><td>0.06</td><td>1962.12</td><td>-14.13</td><td>&lt;.001</td></tr><tr><td>Fiction - Instructional</td><td>-3.44</td><td>0.05</td><td>1961.76</td><td>-69.17</td><td>&lt;.001</td></tr><tr><td>Fiction - Personal Correspondence</td><td>1.15</td><td>0.08</td><td>1957.80</td><td>13.65</td><td>&lt;.001</td></tr><tr><td>Informative - Instructional</td><td>-2.65</td><td>0.04</td><td>1956.92</td><td>-59.40</td><td>&lt;.001</td></tr><tr><td>Informative - Personal Correspondence</td><td>1.93</td><td>0.08</td><td>1956.74</td><td>23.69</td><td>&lt;.001</td></tr><tr><td>Instructional - Personal Correspondence</td><td>4.59</td><td>0.08</td><td>1956.63</td><td>58.82</td><td>&lt;.001</td></tr></tbody></table>

Let us now turn to the linguistic features that load onto PC1 to find out which features contribute the largest register-based differences in the TEC. The most important ones are visualised in Fig. 59 and the full list of loadings is displayed in the first column of Table 68. As explained in 7.2.7, in the present framework and unlike in a classic MDA (Biber 1988: 93), every linguistic feature entered in the analysis loads onto each dimension, as opposed to only on the dimension to which they contribute most. As a result, in this model, all 61 linguistic features contribute, to a greater or lesser (sometimes extremely minimal!) extent, to PC1. Table 68 displays the feature loadings (eigenvalues) which shows the degree to which each feature correlates with each component. Positive values (in shades of yellow) contribute to high component scores, whilst negative ones (in shades of purple) contribute to low scores. These normalised weight values correspond to factor loadings in EFA: hence, to calculate a text's position along PC1, all the log z-scores of the 61 features of the text are multiplied by their corresponding PC1 loadings. Thus, if a text has a high average word length (AWL), its high (and positive) log z-score for
AWL will be multiplied by 0.22, which will contribute to placing this text high on the PC1 dimension. Should this text also feature no or very few verbal contractions verbs (CONT), its low (and negative) log z-score for contractions will be multiplied by -0.25, thus contributing to an even higher overall PC1 score. By contrast, a text consisting of mostly short words and featuring many contractions will likely score low on PC1. Very low absolute loadings are printed in light grey to indicate that these feature contributions most likely only represent noise and are therefore not considered in the interpretation of these dimensions of linguistic variation. However, they are not entirely removed from the table as a reminder that no threshold was applied in the calculation of the component scores.

<figure><img src="image_69.png" /><figcaption>Fig. 64: Coefficient estimates and 95% confidence intervals of the fixed effects in the model: lmer(PC1 ~ Register + (1|Series)) (the intercept corresponds to the reference level: Register [Conversation])</figcaption></figure>

<table><caption>Table 68: List of feature loadings (eigenvectors) on the four dimensions of the model of intra-textbook variation</caption><thead><tr><th></th><th>PC1</th><th>PC2</th><th>PC3</th><th>PC4</th></tr></thead><tbody><tr><td>ACT</td><td>0.08</td><td>-0.11</td><td>0.04</td><td>-0.10</td></tr><tr><td>AMP</td><td>-0.12</td><td>-0.10</td><td>-0.11</td><td>0.01</td></tr><tr><td>ASPECT</td><td>0.10</td><td>-0.05</td><td>0.14</td><td>-0.01</td></tr><tr><td>AWL</td><td>0.22</td><td>-0.16</td><td>-0.12</td><td>-0.13</td></tr><tr><td>BEMA</td><td>-0.22</td><td>0.01</td><td>-0.22</td><td>0.02</td></tr><tr><td>CC</td><td>0.05</td><td>-0.21</td><td>-0.19</td><td>0.00</td></tr><tr><td>COMM</td><td>0.20</td><td>0.09</td><td>0.14</td><td>-0.04</td></tr><tr><td>COND</td><td>-0.01</td><td>-0.02</td><td>0.11</td><td>-0.25</td></tr><tr><td>CONT</td><td>-0.25</td><td>0.11</td><td>-0.03</td><td>-0.06</td></tr><tr><td>CUZ</td><td>-0.09</td><td>-0.13</td><td>-0.06</td><td>-0.02</td></tr><tr><td>DEMO</td><td>-0.12</td><td>0.08</td><td>0.03</td><td>-0.09</td></tr><tr><td>DMA</td><td>-0.20</td><td>0.14</td><td>-0.02</td><td>0.00</td></tr><tr><td>DOAUX</td><td>-0.01</td><td>0.20</td><td>0.06</td><td>-0.15</td></tr><tr><td>DT</td><td>0.12</td><td>0.00</td><td>0.31</td><td>-0.02</td></tr><tr><td>EMPH</td><td>-0.19</td><td>-0.02</td><td>0.06</td><td>-0.14</td></tr><tr><td>EX</td><td>-0.10</td><td>-0.05</td><td>-0.12</td><td>0.05</td></tr></tbody></table>
<table><tr><td>EXIST</td><td>-0.02</td><td>-0.15</td><td>-0.09</td><td>-0.09</td></tr><tr><td>FPP1P</td><td>-0.17</td><td>0.01</td><td>-0.07</td><td>0.00</td></tr><tr><td>FPP1S</td><td>-0.23</td><td>0.07</td><td>0.08</td><td>-0.01</td></tr><tr><td>FPUH</td><td>-0.16</td><td>0.15</td><td>-0.09</td><td>0.07</td></tr><tr><td>FREQ</td><td>-0.03</td><td>-0.05</td><td>0.01</td><td>-0.10</td></tr><tr><td>IN</td><td>0.17</td><td>-0.18</td><td>0.02</td><td>-0.08</td></tr><tr><td>JJAT</td><td>-0.06</td><td>-0.18</td><td>0.04</td><td>-0.21</td></tr><tr><td>JJPR</td><td>-0.17</td><td>-0.06</td><td>-0.12</td><td>-0.11</td></tr><tr><td>LD</td><td>0.16</td><td>-0.03</td><td>-0.26</td><td>-0.01</td></tr><tr><td>MDCA</td><td>-0.04</td><td>0.10</td><td>-0.18</td><td>-0.09</td></tr><tr><td>MDCO</td><td>-0.05</td><td>-0.10</td><td>0.22</td><td>0.01</td></tr><tr><td>MDWS</td><td>-0.07</td><td>-0.01</td><td>0.05</td><td>-0.17</td></tr><tr><td>MENTAL</td><td>0.14</td><td>0.13</td><td>0.12</td><td>-0.25</td></tr><tr><td>NCOMP</td><td>0.04</td><td>-0.05</td><td>-0.24</td><td>-0.15</td></tr><tr><td>NN</td><td>0.20</td><td>-0.09</td><td>-0.29</td><td>0.11</td></tr><tr><td>OCCUR</td><td>0.02</td><td>-0.18</td><td>0.03</td><td>0.02</td></tr><tr><td>PASS</td><td>-0.01</td><td>-0.22</td><td>-0.06</td><td>-0.06</td></tr><tr><td>PEAS</td><td>-0.06</td><td>-0.17</td><td>0.13</td><td>-0.13</td></tr><tr><td>PIT</td><td>-0.19</td><td>-0.04</td><td>-0.06</td><td>-0.06</td></tr><tr><td>PLACE</td><td>-0.16</td><td>-0.01</td><td>-0.07</td><td>0.09</td></tr><tr><td>POLITE</td><td>-0.14</td><td>0.13</td><td>-0.07</td><td>0.02</td></tr><tr><td>POS</td><td>-0.01</td><td>0.03</td><td>-0.04</td><td>0.16</td></tr><tr><td>PROG</td><td>-0.12</td><td>-0.02</td><td>0.11</td><td>0.00</td></tr><tr><td>QUAN</td><td>-0.15</td><td>-0.03</td><td>0.12</td><td>-0.19</td></tr><tr><td>QUPR</td><td>-0.10</td><td>-0.05</td><td>0.16</td><td>-0.11</td></tr><tr><td>RB</td><td>-0.19</td><td>-0.08</td><td>0.20</td><td>0.00</td></tr><tr><td>RP</td><td>0.00</td><td>-0.09</td><td>0.14</td><td>0.02</td></tr><tr><td>SPLIT</td><td>-0.11</td><td>-0.18</td><td>0.02</td><td>-0.16</td></tr><tr><td>SPP2</td><td>0.10</td><td>0.22</td><td>-0.01</td><td>-0.25</td></tr><tr><td>THATD</td><td>-0.05</td><td>0.04</td><td>0.16</td><td>-0.24</td></tr><tr><td>THRC</td><td>0.02</td><td>-0.11</td><td>-0.02</td><td>-0.18</td></tr><tr><td>THSC</td><td>-0.06</td><td>-0.17</td><td>0.07</td><td>-0.14</td></tr><tr><td>TIME</td><td>-0.13</td><td>-0.08</td><td>-0.01</td><td>0.06</td></tr><tr><td>TPP3P</td><td>-0.01</td><td>-0.16</td><td>-0.09</td><td>-0.02</td></tr><tr><td>TPP3S</td><td>-0.06</td><td>-0.11</td><td>0.13</td><td>0.30</td></tr><tr><td>TTR</td><td>-0.04</td><td>-0.26</td><td>-0.05</td><td>-0.01</td></tr><tr><td>VBD</td><td>-0.08</td><td>-0.20</td><td>0.23</td><td>0.30</td></tr><tr><td>VBG</td><td>0.04</td><td>-0.18</td><td>0.00</td><td>-0.22</td></tr><tr><td>VBN</td><td>0.03</td><td>-0.18</td><td>-0.07</td><td>-0.04</td></tr><tr><td>VIMP</td><td>0.25</td><td>0.15</td><td>0.04</td><td>-0.08</td></tr><tr><td>VPRT</td><td>-0.16</td><td>0.05</td><td>-0.32</td><td>-0.22</td></tr><tr><td>WHQU</td><td>0.11</td><td>0.23</td><td>0.00</td><td>-0.09</td></tr><tr><td>WHSC</td><td>0.11</td><td>-0.11</td><td>0.03</td><td>-0.15</td></tr><tr><td>XX0</td><td>-0.22</td><td>0.03</td><td>0.06</td><td>-0.06</td></tr><tr><td>YNQU</td><td>-0.03</td><td>0.23</td><td>0.00</td><td>-0.08</td></tr></table>
As shown in Fig. 56 and Fig. 58, the upper end of the PC1 scale, roughly between 2 and 4, is entirely reserved for instructional texts. We can therefore expect the linguistic features that load positively on PC1 to be highly frequent in textbook instructions and explanations. Indeed, these include imperative verbs (VIMP), the semantic categories of communication and mental verbs (COMM and MENTAL) and WH-questions (WHQU), e.g., (215). Other features contributing to high PC1 scores are more akin to the negative, ‘informational’ end of Biber’s (1988) Dimension 1. These include longer words (AWL), nouns (NN), prepositions (IN), and a high ratio of content to function words (LDE), all of which are associated with impersonal and informational writing.

By contrast, the linguistic features with the most negative loadings on PC1 are associated with spontaneous, interactional production: e.g., contractions (CONT), first person singular and *it* pronouns (FFP1S and PIT), negation (XX0), discourse markers (DMA), emphatics (EMPH), fillers and interjections (FPUH), and demonstrative pronouns (DEMO), e.g. (221). These features very much echo the upper, interactional, end of Biber’s (1988) Dimension 1 (see Table 40).

<figure><img src="image_70.png" /></figure>

### 7.3.1.2 Variation on Dimension 2: ‘Involved vs. Informational Production’

The second dimension of variation in the TEC data (PC2) accounts for 14.16% of the total variance. As has been argued in 7.3.1, it shows a high degree of overlap with the linguistic features that contribute to both ends of Biber’s Dimension 1 and will therefore also be labelled: ‘Involved vs. Informational Production’.

A first mixed-effects model with random intercepts for each textbook series and only Register as a fixed effect explained 56% of the variance in scores on this second dimension, whereas the full model adding Register*Level interactions explained 72%. This indicates that linguistic variation along this second dimension is driven by both register and the proficiency level of the textbooks. As with the first dimension, a
comparison of the models showed that textbook series has no significant effect on the
position of textbook texts along this dimension. As a result, only the estimated
coefficients of the fixed effects of the full PC2 model are visualised in Fig. 65. The
colours correspond to the textbook registers as already introduced in the previous
plots of this chapter. The intercept represents Textbook Conversation Level A texts,
and the coefficients are interpreted just like in the model summary tables (e.g., Table
66).

The register cline on this second dimension bears strong similarities to Biber's (1988) Dimension 1, 'Involved vs. Informational Production' (see Fig. 29), which, in various forms, has emerged as the strongest dimension of linguistic variation in many MDAs, across a wide range of domains and languages (Biber 2014). As shown on Fig. 56 and Fig. 58, Textbook Conversation texts are mostly clustered at the upper end of the cline. In addition, Fig. 65 indicates a clear proficiency-level effect – with more advanced textbook conversations scoring lower on average. The lower end of the register cline is dominated by Textbook Informative texts which all have negative PC2 scores. Personal correspondence and fiction are, once again, situated in the middle. The overlapping confidence intervals on Fig. 65 show that the two textbook registers cannot, on average, be reliably differentiated on this second dimension of linguistic variation.

As we would expect given that linguistic variation is multi-dimensional, we find in
Fig. 59 and in the table of feature loadings (Table 68) that some of the linguistic
features with positive loadings on PC1 also contribute to positive values on PC2:
these include imperatives, mental and communication verbs – which are known to be
particularly strongly associated with instructional texts, e.g., (215). In addition, we
find WH- and yes/no-questions (WHQU and YNQU) and second-person references
(SPP2), which are also found in many textbook instructions and task descriptions.
Consequently, many instructional texts score relatively high on PC2, e.g., (222). In
contrast to the first dimension, on this second dimension these features are also
associated with linguistic features typical of spontaneous conversation and
interactional language, in particular with fillers and interjections (FPUH), discourse
(DMA) and politeness markers (POLITE), the modal can (MDCA) and singular first-
person references (FPP1S). As a result, the highest PC2 scores are achieved by
textbook dialogues such as (217), which feature rapid question and answer exchanges.
Some of the highest PC2 scores are found in textbook dialogues that model classroom
interactions. These incorporate spoken instructional language which sometimes
includes some of the instructions that accompany the textbooks' tasks and exercises
verbatim, e.g., compare extracts (222) and (223).
<figure><img src="image_71.png" /><figcaption>Fig. 65: Coefficient estimates and 95% confidence intervals of the fixed effects in the model: lmer(PC2 ~ Register + Level + Register*Level + (1|Series)). The reference levels are Register [Conversation] and Level [A].</figcaption></figure>

(222) Nadia is going on holiday A. Read her email to her friend. Who is going with her on holiday? What does she promise to do after her holiday?
(T) Imagine you are going on holiday.
B. Write an email to an English-speaking friend and tell him/her what you're going to do. Use the information and Nadia's email to help you.
<TEC: English in Mind 2>

(223) Good morning, everyone! Right, sit down! Paul, can you show us where Scotland is?

OK, Miss.

Excellent. Please take your workbooks out. Now, please open your workbooks at page 12. Yes?

Sorry, Miss, my workbook is at home.

Can you work with Paul, please?

OK. <TEC: Access G 1>
The feature with the strongest negative contribution to this second dimension is type/token ratio (TTR) – a feature which, in Biber’s (1988) model, is also strongly associated with the ‘informational’ end of the ‘Involved vs. Informational Production’ dimension. In addition, the lower end of the dimension is characterised by further lexico-grammatical features typical of structurally more complex, meticulously drafted written production such as passives (PASS), coordinating conjunctions (CC), non-finite verb forms ending in -ing and -ed (VBG and VBN), split auxiliaries (SPLIT) and *that*-subordinate clauses (THSC), e.g., (224). Like the informational end of Biber’s (1988) Dimension 1, prepositions (IN), longer words (AWL), and attributive adjectives (JJAT) are also associated with the negative end of PC2 (see Table 68).

(224) Although books are still popular with teenagers, most of them spend more of their leisure time staring at their phone than reading a paperback. And the more versatile phones become, the more reasons young people have for looking at them. In response to this trend, some smart, young authors have changed the way they write. Instead of publishing a whole book at once, they produce very short chapters, which they send once a week to their readers by text message. Some even claim that this style of writing represents a new literary genre: the ‘cell phone novel’. <TEC: Solutions intermediate>

Fig. 65 shows a clear pattern of decreasing PC2 scores as the proficiency level of textbooks increases. A cursory look at the linguistic features with negative loadings on PC2 (see Table 68) suffices to understand why: the majority of these features are not introduced until the second or third year of EFL instruction. The extent to which these features are intrinsically linked to specific registers determines how large the shift to the negative end of the PC2 scale is, as learners are progressively introduced to these features in the more advanced textbooks.

Thus, we find that the median PC2 score for beginner textbook fiction texts is 1.09 (MAD = 0.59) but, as soon as the past tense (VBD) is introduced in level B textbooks, the PC2 scores for fiction texts drop to a median of -0.06 (MAD = 0.74) and then further to -0.82 (MAD = 0.76) in level C fiction. The other features that make significant contributions to this negative shift include higher type/token ratio and lexical density (LD), longer words (AWL), and higher normalised frequencies of the perfect aspect (PEAS), passives (PASS), *could* as a modal (MDCO), occurrence and existential verbs (OCCUR and EXIST), prepositions (IN), attributive adjectives (JJAT), and *that*-subordinate clauses (THSC), e.g., (225). Fig. 66, however, suggests that, on average, the linguistic features that contribute to PC2 are used to a similar extent in level D and E fiction texts. Indeed, apart from type/token ratio, which can be expected to continue to grow as learners become more proficient in English, all the aforementioned features that contribute to the negative end of PC2, e.g., perfect aspect, passives, *could*, etc., can be expected to have been taught by the fourth year of secondary school EFL instruction.
<figure><img src="image_72.png" /><figcaption>Fig. 66: Estimated PC2 scores across each register and the five textbook proficiency levels</figcaption></figure>

(225) "If I just **had** that knife **in** my hand, I **could** ..." Suddenly a smile **lit** up his face as he **thought of** a plan. He **took hold of** his fishing line. The pole **was** still **in** the canoe so he **could** pull the boat **towards** him, get the knife, cut the roots and get free. Carefully, he **began to** pull **on** the line and **felt** the canoe start **to** move. Then, just as he **started** to so hope **that** his plan might be working, he **heard** a **quiet** splash as his fishing pole **fell** out **of** the canoe **into** the water. The canoe **stopped** moving. [...] While he **had watched** the alligator, his canoe **had drifted** up **behind** him and **hit** him **in** the back. <TEC: Access G 4>

For the informative and conversation texts, the shift towards the negative end of the PC2 scale from level A to E is more or less linear (see Fig. 66). This pattern is to be expected for informative texts: as for the narrative texts, textbook authors are necessarily restricted in their choice of grammatical features given that learners have only been introduced to a limited number of grammatical features and basic vocabulary. Many of the negative loading features on PC2 are typical of informational writing so that it comes as no surprise that, on average, the informative texts of the more advanced textbooks score lowest on this dimension, e.g., (226).

(226) A kiss is ambiguous **at the best of times**, **signifying** anything **from** friendliness **to desire**, deference **to insult**. Kissing - **on the lips**, originally - **was**, **in fact**, a common form **of social greeting** **in Britain** **from Roman times at least until** the 1700s, **when** the potential **for misinterpretation led to** its disappearance. Abroad, of course, they've **never really abandoned** the gesture, **although** the rules **governing** its use **are sometimes exceedingly complicated**. In France, **for example**, anything **between one and four kisses** can be acceptable **depending on who you are**,
who you're kissing, how well the two **of** you know each other **and** exactly where you both happen **to** be **in** France. There are so many variables **that** even French people **within** the same region confess **to being** confused.

<TEC: New Bridges 2<sup>de</sup>>

By contrast, the strong interactions between the conversation register and the proficiency levels of the textbooks constitute a more puzzling finding because few of the features with negative loadings on PC2 can be said to be typical of real-life conversation; yet, on average, advanced textbooks nevertheless feature considerably more of these in their representations of spoken language than beginner textbooks (see Fig. 66). The median PC2 value of level A textbook conversations is 2.60 (MAD = 0.76), with many beginner conversations scoring considerably higher, e.g., (217) and (223) with PC2 scores of 4.08 and 3.53 respectively. However, median PC2 scores progressively drop as textbooks become more advanced, reaching -0.16 (MAD = 1.33) for level E textbook conversations. This echoes the results of the additive MDA based on Biber's (1988) Dimension 1 (see 6.3.1.1 and 6.3.2.1). Advanced textbook dialogues with low PC2 scores tend to feature much higher type/token ratios, longer words, more passives, past tense and perfect verbs, split auxiliaries, coordinating conjunctions, nouns, prepositions, and subordinate clauses, e.g., (227)-(228).

(227) We're here **at** the BBC Radio's annual Teen Awards at Wembley. As I'm sure many **of** our listeners know, the **prizes** are awarded **to** the year's best vloggers, sport and music stars and to teenage heroes who have inspired everyone! Best **of** all, they **have been voted for** by Britain's teenagers! So let's find out **what** the fans here **thought of** the show. OK, what did you think **was** the best **moment** of the afternoon?

Well, **for** me, it has to be **when** Jack G got his **award** for standing up to bullying. If I'd been him, I wouldn't **have had** the **courage** to start a **campaign** against the bullies in my school, so I really admire him **for** doing that. <TEC: Solutions intermediate plus>

(228) The state is also very active in limiting air pollution. But what about traffic in places like L.A.?

Yes, the traffic in Los Angeles is a huge problem which has existed since the arrival of the automobile in the late 1800s. Until then people were transported in L.A. by streetcar. Once the automobile arrived, people came to love the freedom it offered. It allowed them to move far from the center of L.A. and still be able to reach downtown - something the streetcars couldn't offer. <TEC: Green Line New 5>

The pattern observed in the instructional register on Fig. 66 is in line with that observed in the previous section (7.3.1.1). Linguistically, it is driven by more complex sentence structures in the explanations of more advanced textbooks which are characterised by higher frequencies of subordinate clauses (THSC and WHSC) and coordinating conjunctions (CC) – three features which contribute to lower PC2 scores. This leads to the modest, but nevertheless significant, interaction effects between the instructional register and textbook proficiency level reported in the mixed-effects
model of PC2 scores (see Fig. 65, Fig. 66 and, for the full results of the model, the
*Online Appendix 7.6*).

7.3.1.3 Variation on Dimension 3: ‘Present/factual vs. Past/speculative’

The third dimension that emerges from this intra-textbook PCA accounts for just
5.31% of the variance. As already identified in 7.3.1, this dimension of variation
appears to involve both register-based and proficiency level-based linguistic variation
(see Fig. 60 and Fig. 62). This visual observation was confirmed in the mixed effects
models computed to model PC2 scores: a comparison of models for this dimension
shows that register alone explains 27% of the variance in PC3 scores; the proficiency
level variable alone accounts for 12%, but by modelling the interactions between the
two variables 44% of the variance in PC3 can be explained. The fixed effects
coefficient estimates of the full model that includes these interactions are displayed
in Fig. 67.

<figure><img src="image_73.png" /><figcaption>Fig. 67: Coefficient estimates and 95% confidence intervals of the fixed effects in the model: lmer(PC3 ~ Register + Level + Register*Level + (1|Series). The intercept corresponds to the reference levels Register [Conversation] and Level [A].</figcaption></figure>
The effects of the interactions are also illustrated in Fig. 68. Across all five registers, the largest jump in PC3 scores is observed between Level A and Level B textbook texts. It is largely driven by the near absence of past tense verbs in Level A textbooks contrasted by the highly frequent use of the past in Levels B and C textbooks as learners are taught to comprehend and produce this grammatical feature (see 6.3.1.2 for a similar observation on Biber's Dimension 2 including examples). Other strongly loading features on PC3 whose frequencies are also strongly mediated by the proficiency level of the textbooks include the perfect aspect (PEAS), the use of the modal *could* (MDCO), and conditional subordination (COND) – all three of which make positive contributions to PC3 scores.

Interestingly, aside from the present tense variable (VPRT), the frequencies of all the other strong contributors to the negative end of this dimension are not correlated with textbook proficiency and hence do not contribute to the effects visualised in Fig. 68. Instead, this cluster of negative-loading features – which includes the total frequency of nouns (NN), lexical density (LD) and noun-noun compounds (NCOMP) – describes the nominal style of the negative end of this dimension. This is a characteristic of Textbook English that is stable across all proficiency levels. The multitude of effects at play in this dimension make it difficult to interpret but, for now, this dimension is tentatively labelled ‘Present/factual vs. Past/speculative’.

<figure><img src="image_74.png" /><figcaption>Fig. 68: Estimated PC3 scores across each register and the five textbook proficiency levels</figcaption></figure>

### 7.3.1.4 Variation on Dimension 4: ‘Clausal complexity’

Similarly to PC3, the fourth dimension that emerged from this intra-textbook PCA only accounts for 4.50% of the variance in the texts of the TEC. It, too, reflects both
register- and proficiency-level-based variation – this time to almost equal degrees: register alone explains 20% of the variance in PC3 scores, whilst the five levels of the proficiency level variable account for 19%. The full mixed effects model involving the interactions between the register and proficiency levels, however, explains a total of 43% of the variance in PC4 scores. The estimated coefficients of the fixed effects of this model are plotted in Fig. 69.

At first glance, the highest loading features on this fourth dimension might suggest a second ‘narrative’ dimension since some of these features (e.g., past tense verbs and third-person reference) are shared with the positive-loading features of the third dimension (see Fig. 61). However, this dimension appears to be much more driven by the interaction effects of proficiency level and register. These effects, predicted by the model summarised in Fig. 69, are illustrated in Fig. 70. Their direction is somewhat surprising given that the second highest loading feature on PC4 is past tense verbs (VBD, see Table 68) and, whilst we have seen in Fig. 47 in 6.3.2.2 that the occurrence of past tense verbs drastically increases from level B textbook onwards, Fig. 70 show that PC4 scores actually decrease as the proficiency level of the textbook increases.

Thus, this proficiency-level-based effect must be driven by other lexico-grammatical phenomena that contribute to negative scores on this dimension and which are gradually introduced in the later years of secondary school EFL teaching. These include *if*-conditionals that lead to an increase in conditional subordinators (COND) and the modals *will* and *should* (MDWS), e.g., (230). Furthermore, this dimension points to linguistic variation that, at the upper end, is characterised by narrative discourse that relies foremost on individual words for brief descriptions (e.g., adverbs of time and place and use of the *there is/are* construction, see (229)), whilst, as attested by the negative loadings of features such as THATD, VBG, THRC, WHSC and THSC (see Table 68), the lower end of this fourth dimension is characterised by more complex sentence structures that, in fiction, allow for more detailed descriptive passages and in conversation, personal correspondence, instructional and informative texts, convey more sophisticated explanations, opinions or arguments, e.g., (230). Hence, this dimension can be summarised as representing a complexity cline with the simplest constructions clustering at the upper end of the dimension and the most complex at the bottom. The label ‘Clausal complexity’ was therefore chosen to describe this fourth dimension.
<figure><img src="image_75.png" /><figcaption>Fig. 69: Coefficient estimates and 95% confidence intervals of the fixed effects in the model: lmer(PC4 ~ Register + Level + Register*Level + (1|Series)). The reference levels are Register [Conversation] and Level [A].</figcaption></figure>

(229) In December 1980, ex-Beatle John Lennon and his wife Yoko Ono were in New York. In the afternoon, they were on their way to a recording studio to work on a new song. There was an American called Mark Chapman in the street. In his hand, there was a piece of paper and a pen. 'Mr Lennon,' he said. 'Can I have your autograph?' John Lennon signed his name and Chapman went away. In the evening, John and Yoko were in front of their apartment building. There was a man at the door. It was Mark Chapman. This time, there wasn't a pen in his hand, but a gun. 'Mr Lennon!' he said. Suddenly, there were five shots and John Lennon was dead.

<TEC: English in Mind Starter>

(230) On the other hand, opponents of nuclear weapons argue that it will not be long before some countries develop a defence system that makes them immune to nuclear attack, while still being able to launch a nuclear offensive of their own. When this happens, the world will suddenly become an extremely dangerous place. Furthermore, they point to the huge
cost of the weapons and say **that** the world would be a better and safer
place **if** the money were spent on health and education. <TEC:
Solutions Intermediate plus>

<figure><img src="image_76.png" /><figcaption>Fig. 70: Estimated PC4 scores across each register and the five textbook proficiency levels</figcaption></figure>

7.3.2 PCA of Textbook English vs. ‘real-life’ English

This section presents the results of a PCA-based MDA that attempts to capture the similarities and differences between the texts of the Conversation, Fiction and Informative subcorpora of the TEC (see 3.3.1.4) and the three corresponding reference corpora outlined in 3.3.2: the Spoken BNC2014, the Youth Fiction corpus and the Informative Texts for Teenagers Corpus (Teens Info). Methodologically, it follows the same procedure as for the intra-textbook variation PCA reported on in 7.3.1.

First, the exclusion of linguistic features that are absent from more than two-thirds
of the texts of the TEC and reference corpora together led to the merging of the
ABLE and JJPR categories and the PGET and PASS (as in 7.3.1), as well as the
removal of the EMO, HST, PRP and URL features. Due to otherwise very low final
communalities, the singular third-person (TPP3S) and the plural third-person
referent categories (TPP3P) were merged into a single TPP3 category. For the same
reason, the adverbs of frequency (FREQ) and time (TIME) were also combined into
a more general FQTI category. The normalised counts of all the features were then
standardised before plotting their distributions (see Online Appendix 7.5). Next, 115
outlier texts (= 2.28% of the texts) were identified on the basis of excessively high z-
scores. The majority of these outlier texts (n = 74) stem from the Info Teens corpus.
Many of these outliers are articles composed of bullet point lists featuring very few
finite verbs, leading to improbably high counts of some of the linguistic features
normalised per finite verb phrase (FVP) (see 7.2.4). These texts were therefore
removed from the dataset to be entered in the following PCAs to avoid them exerting
undue influence on the model and inflating or distorting differences between texts
(see, e.g., Le Foll 2021a: 110–113 on the consequences on leaving such texts in such
analyses). Here, too, signed log transformation was applied to tame some of the highly
skewed feature distributions (see *Online Appendix 7.5* for details of the procedure and
plots of the distributions before and after transformation). The dataset was checked
for collinearities and excessive correlations with the help of a correlation matrix
display (see Fig. 53 and *Online Appendix 7.5*). This led to the exclusion of the present
tense feature (VPRT) because, with a correlation of -0.97, its normalised counts per
FVP are almost the perfect mirror image of past tense counts per FVP (which, given
that finite verbs can either be tagged as past tense, present tense, or modal does not
come as a surprise). As a result of the above steps, none of the remaining features
returned individual MSA values < 0.5 or final communalities of < 0.2 so that the
final data matrix consisted of the signed log transformed standardised normalised
counts of 72 features in 4,980 texts. Its overall KMO value of 0.95 suggests that it is
“marvellously” suitable for factor analysis (Kaiser & Rice 1974: 112). The scree plot
of eigenvalues suggested a four-component solution (see Fig. 71). The cumulative
proportion of variance explained by these four principal components is 47.15%.

<figure><img src="image_77.png" /><figcaption>Fig. 71: Scree plot of the eigenvalues of the PCs for the Textbook English vs. ‘real-life’ English PCA</figcaption></figure>

As with the exploration of intra-textbook linguistic variation in 7.3.1, the retained
components were first explored using 3-D projections of the position of the texts along
the four dimensions. The projection of the first three principal components (see Fig.
72) reveals three very well-defined clusters for the three reference corpora (in red,
bright green and dark blue) – with hardly any overlap. By contrast, the three TEC
subcorpora cluster across much larger areas of the 3-D plot, among them the intercept of the plot, which is an area in which hardly any of the reference texts are found.

<figure><img src="image_78.png" /><figcaption>Fig. 72: Snapshots from the 3-D representation of texts along PC1-PC3</figcaption></figure>

This pattern can also be observed in the scatterplot matrix of all combinations of the four dimensions in Fig. 73. At first sight, it would appear that Textbook Fiction texts and those of the Youth Fiction corpus share many similarities on all dimensions. This is in stark contrast to the Textbook Conversation and Spoken BNC2014 clusters, which overlap very little on all four dimensions. In addition, Fig. 73 indicates that there is a lot of internal variation within the Info Teens corpus.

Together, the first two principal components explain 37.85% of the total variance. As shown in more detail in Fig. 74, both these dimensions of linguistic variation appear to capture differences between the three textbook registers and their corresponding reference corpora. The corresponding graph of features (Fig. 75) shows that these first two dimensions share many similarities with Biber's (1988) first two dimensions. The first dimension (PC1) places highly interactional, spontaneous speech at the upper end, whilst informationally dense, edited texts score lowest. It will therefore be labelled 'Spontaneous interactional vs. Edited informational'. On both ends of the dimension, many of the strongest contributing features overlap with those that also have the highest absolute loadings on Biber's (1988) Dimension 1, e.g., verbal contractions (CONT) and discourse markers (DMA) at the positive pole and nouns (NN), longer average word length (AWL), prepositions (IN) and higher lexical diversity (TTR) at the negative pole.
<figure><img src="image_79.png" /><figcaption>Fig. 73: Scatterplot matrix of combinations of the four dimensions of the model of Textbook English vs. ‘real-life’ English (the number before the comma on each axis label shows which principal component is plotted on that axis; this is followed by the percentage of the total variance explained by that particular component)</figcaption></figure>
The second dimension (illustrated on the *y*-axes of Fig. 74 and Fig. 75) is also very comparable to Biber’s (1988) Dimension 2 and will therefore also be labelled: ‘Narrative vs. Non-narrative’. Although the normalisation bases are very different, the highest loading features are the same: past tense verbs (VBD) and third-person referents (TPP3). The perfect aspect (PEAS), which is the third highest-loading feature on Biber’s Dimension 2, ranks eighth on PC2 (see Table 69), whilst public verbs (e.g., EXPLAIN, SAY, TELL), the fourth most important positive contributor to Biber’s Dimension 2, corresponds broadly to the high contribution of verbs of communication (COMM) on PC2. However, unlike Biber’s (1988) Dimension 2, the ‘Narrative vs. Non-narrative’ dimension to emerge from the present MDA involves a number of significantly loading features with negative contributions to the dimension. These include noun compounds (NCOMP), BE as a main verb (BEMA) and the modal can (MDCA).

The three ellipses of the TEC registers are noticeably “shifted” towards the middle of the biplot depicting texts on the first and second dimensions (Fig. 74). In the following sections, the reason for this “shift” towards the middle of the biplot will be explored. To this end, both the full table of feature loadings (Table 69) and graphs of variables such as Fig. 75 are examined to understand the linguistic specificities of these three textbook registers. Linear mixed-effects models were also computed for each principal component but, for reasons of space, only the most salient findings are reported in this chapter. All the models, tables and plots that were explored as part of the following analyses can be found in the *Online Appendix 7.2*.
<figure><img src="image_80.png" /><figcaption>Fig. 74: Projection of the texts of the three subcorpora of the TEC and the reference corpora on PC1 and PC2</figcaption></figure>

<figure><img src="image_81.png" /><figcaption>Fig. 75: Graph of the features with the strongest contributions to the first and second dimensions</figcaption></figure>
<table><caption>Table 69: List of feature loadings (eigenvectors) in the Textbook English vs. 'real-life' English PCA-based model</caption><thead><tr><th></th><th>PC1</th><th>PC2</th><th>PC3</th><th>PC4</th></tr></thead><tbody><tr><td>ACT</td><td>-0.10</td><td>0.01</td><td>-0.01</td><td>0.12</td></tr><tr><td>AMP</td><td>0.00</td><td>-0.05</td><td>-0.05</td><td>0.09</td></tr><tr><td>ASPECT</td><td>-0.08</td><td>0.10</td><td>-0.01</td><td>0.00</td></tr><tr><td>AWL</td><td>-0.21</td><td>-0.13</td><td>-0.06</td><td>-0.01</td></tr><tr><td>BEMA</td><td>0.08</td><td>-0.24</td><td>0.06</td><td>-0.02</td></tr><tr><td>CAUSE</td><td>-0.08</td><td>-0.12</td><td>0.06</td><td>0.14</td></tr><tr><td>CC</td><td>-0.14</td><td>-0.13</td><td>-0.09</td><td>-0.09</td></tr><tr><td>COMM</td><td>-0.03</td><td>0.19</td><td>0.03</td><td>0.20</td></tr><tr><td>CONC</td><td>-0.03</td><td>-0.03</td><td>-0.18</td><td>-0.06</td></tr><tr><td>COND</td><td>0.08</td><td>-0.02</td><td>-0.17</td><td>0.23</td></tr><tr><td>CONT</td><td>0.22</td><td>-0.05</td><td>0.03</td><td>0.00</td></tr><tr><td>CUZ</td><td>0.10</td><td>-0.14</td><td>-0.20</td><td>-0.18</td></tr><tr><td>DEMO</td><td>0.15</td><td>-0.12</td><td>-0.08</td><td>-0.04</td></tr><tr><td>DMA</td><td>0.20</td><td>-0.09</td><td>-0.04</td><td>-0.16</td></tr><tr><td>DOAUX</td><td>0.18</td><td>-0.02</td><td>0.06</td><td>0.00</td></tr><tr><td>DT</td><td>0.08</td><td>0.16</td><td>-0.24</td><td>-0.03</td></tr><tr><td>DWNT</td><td>-0.04</td><td>0.10</td><td>-0.12</td><td>0.09</td></tr><tr><td>ELAB</td><td>-0.07</td><td>-0.17</td><td>-0.04</td><td>0.07</td></tr><tr><td>EMPH</td><td>0.17</td><td>-0.07</td><td>-0.09</td><td>-0.03</td></tr><tr><td>EX</td><td>0.06</td><td>-0.02</td><td>-0.04</td><td>0.00</td></tr><tr><td>EXIST</td><td>-0.13</td><td>-0.09</td><td>-0.05</td><td>0.00</td></tr><tr><td>FPP1P</td><td>0.08</td><td>-0.02</td><td>0.09</td><td>0.19</td></tr><tr><td>FPP1S</td><td>0.17</td><td>0.06</td><td>0.06</td><td>0.08</td></tr><tr><td>FPUH</td><td>0.18</td><td>-0.10</td><td>-0.05</td><td>-0.19</td></tr><tr><td>FQTI</td><td>-0.07</td><td>0.03</td><td>0.01</td><td>0.14</td></tr><tr><td>GTO</td><td>0.14</td><td>0.00</td><td>-0.04</td><td>0.00</td></tr><tr><td>HDG</td><td>0.10</td><td>-0.07</td><td>-0.14</td><td>-0.12</td></tr><tr><td>HGOT</td><td>0.16</td><td>-0.05</td><td>-0.01</td><td>-0.11</td></tr><tr><td>IN</td><td>-0.21</td><td>-0.01</td><td>-0.08</td><td>0.01</td></tr><tr><td>JJAT</td><td>-0.05</td><td>-0.13</td><td>-0.25</td><td>0.07</td></tr><tr><td>JJPR</td><td>-0.03</td><td>-0.17</td><td>-0.03</td><td>0.21</td></tr><tr><td>LD</td><td>-0.17</td><td>-0.16</td><td>0.13</td><td>-0.04</td></tr><tr><td>MDCA</td><td>0.05</td><td>-0.21</td><td>0.11</td><td>0.22</td></tr><tr><td>MDCO</td><td>0.00</td><td>0.19</td><td>-0.15</td><td>0.10</td></tr><tr><td>MDMM</td><td>-0.02</td><td>-0.10</td><td>-0.14</td><td>0.17</td></tr><tr><td>MDNE</td><td>0.06</td><td>-0.02</td><td>-0.06</td><td>0.22</td></tr><tr><td>MDWO</td><td>0.07</td><td>0.11</td><td>-0.18</td><td>0.04</td></tr><tr><td>MDWS</td><td>0.06</td><td>-0.02</td><td>-0.01</td><td>0.25</td></tr><tr><td>MENTAL</td><td>0.11</td><td>-0.02</td><td>-0.05</td><td>0.16</td></tr><tr><td>NCOMP</td><td>0.00</td><td>-0.27</td><td>-0.05</td><td>0.03</td></tr><tr><td>NN</td><td>-0.21</td><td>-0.10</td><td>0.10</td><td>-0.07</td></tr><tr><td>OCCUR</td><td>-0.13</td><td>-0.02</td><td>-0.05</td><td>-0.06</td></tr></tbody></table>
<table><tr><td>PASS</td><td>-0.14</td><td>-0.13</td><td>-0.09</td><td>-0.10</td></tr><tr><td>PEAS</td><td>-0.06</td><td>0.12</td><td>-0.19</td><td>0.12</td></tr><tr><td>PIT</td><td>0.15</td><td>-0.10</td><td>-0.15</td><td>-0.07</td></tr><tr><td>PLACE</td><td>0.02</td><td>0.09</td><td>0.09</td><td>0.07</td></tr><tr><td>POLITE</td><td>0.09</td><td>0.00</td><td>0.20</td><td>0.11</td></tr><tr><td>POS</td><td>0.02</td><td>0.09</td><td>0.04</td><td>-0.05</td></tr><tr><td>PROG</td><td>0.09</td><td>0.08</td><td>-0.04</td><td>0.15</td></tr><tr><td>QUAN</td><td>0.17</td><td>-0.01</td><td>-0.16</td><td>0.01</td></tr><tr><td>QUPR</td><td>0.08</td><td>0.11</td><td>-0.12</td><td>0.21</td></tr><tr><td>QUTAG</td><td>0.15</td><td>-0.04</td><td>-0.07</td><td>-0.15</td></tr><tr><td>RB</td><td>0.14</td><td>0.15</td><td>-0.18</td><td>0.07</td></tr><tr><td>RP</td><td>-0.01</td><td>0.22</td><td>-0.09</td><td>0.15</td></tr><tr><td>SPLIT</td><td>-0.03</td><td>-0.11</td><td>-0.21</td><td>0.08</td></tr><tr><td>SPP2</td><td>0.17</td><td>-0.07</td><td>0.10</td><td>0.16</td></tr><tr><td>STPR</td><td>0.10</td><td>0.01</td><td>0.01</td><td>-0.04</td></tr><tr><td>THATD</td><td>0.16</td><td>-0.01</td><td>-0.14</td><td>-0.02</td></tr><tr><td>THRC</td><td>-0.05</td><td>-0.17</td><td>-0.15</td><td>-0.02</td></tr><tr><td>THSC</td><td>-0.02</td><td>-0.08</td><td>-0.27</td><td>0.07</td></tr><tr><td>TPP3</td><td>-0.05</td><td>0.30</td><td>-0.04</td><td>-0.15</td></tr><tr><td>TTR</td><td>-0.19</td><td>0.07</td><td>-0.02</td><td>0.16</td></tr><tr><td>VBD</td><td>-0.10</td><td>0.35</td><td>-0.05</td><td>-0.20</td></tr><tr><td>VBG</td><td>-0.14</td><td>-0.02</td><td>-0.14</td><td>0.12</td></tr><tr><td>VBN</td><td>-0.14</td><td>-0.10</td><td>-0.08</td><td>-0.07</td></tr><tr><td>VIMP</td><td>0.01</td><td>-0.07</td><td>0.21</td><td>0.21</td></tr><tr><td>WHQU</td><td>0.13</td><td>-0.02</td><td>0.20</td><td>0.07</td></tr><tr><td>WHSC</td><td>-0.09</td><td>-0.10</td><td>-0.20</td><td>0.05</td></tr><tr><td>XX0</td><td>0.18</td><td>0.01</td><td>-0.06</td><td>0.06</td></tr><tr><td>YNQU</td><td>0.18</td><td>-0.03</td><td>0.14</td><td>-0.02</td></tr></table>

Given that the first two dimensions appear to be functionally and linguistically analogous to Biber's (1988) dimensions, we can expect that many of the similarities and differences between Textbook English registers and situationally comparable, reference registers observed in Chapter 6 using additive MDA will be confirmed in the present analysis.

The third dimension (PC3), however, is not represented in Biber's (1988) model. Indeed, Fig. 76 shows that this third dimension appears to directly model some specificities of Textbook English as opposed to non-textbook, naturally occurring English. It is therefore labelled 'Pedagogically adapted vs. Natural'. Whilst the ellipses of the three reference corpora are entirely superimposed onto each other along PC3, with centroids around zero on the PC3 axis, the ellipses of the three TEC registers are notably shifted towards the positive end of the dimension. In addition, the elongated shapes of the ellipses of the textbook registers on this biplot shows that these texts cover a wide range of PC3 scores. Fig. 77 demonstrates that much of this
intra-register variation is driven by the proficiency levels of the textbooks: it displays the texts on PC3 and PC4 in exactly the same position as in Fig. 76 but, on Fig. 77, the ellipses correspond to the Level variable rather than, as in Fig. 76, the (Sub-)corpus variable. These ellipses show that, on average, level A textbook texts score highest on this third dimension and that, as the proficiency level of the textbooks increases, PC3 scores decrease.

<figure><img src="image_82.png" /><figcaption>Fig. 76: Projection of the texts of the three subcorpora of the TEC and the reference corpora on PC3 and PC4</figcaption></figure>

<figure><img src="image_83.png" /><figcaption>Fig. 77: Projection of the texts of the three subcorpora of the TEC and the reference corpora on PC3 and PC4 with ellipses representing the five textbook proficiency levels vs. the reference corpora</figcaption></figure>
This effect is confirmed in the summary of the linear mixed-effects model computed to predict scores on this third ‘Pedagogically adapted vs. Natural’ dimension (see Table 70). Note that, in this and all other models explored in the following sections, the reference level is the Reference (Ref.) Conversation data, i.e., the Spoken BNC2014. As opposed to the PCA-based model of intra-textbook variation discussed in 7.3.1, only three registers are now modelled: Conversation, Fiction, and Informative. In addition, the Level variable now includes a sixth level for the Ref. corpora alongside the familiar proficiency levels of the TEC (A to E). This means that, on this third dimension, Table 70 shows us that Textbook Conversation texts from beginner (level A) textbooks score, on average, 4.25 points more than the intercept of -0.64, i.e., 3.61. Hence, we observe a particularly large difference between beginner Textbook Conversation texts and the transcripts of the Spoken BNC2014. Following the same logic, Table 70 shows that Textbook Fiction texts from the most advanced textbooks in the TEC have a mean PC3 score of -0.05 (= -0.64 + 1.29 + 0.43 + -1.03), which, by contrast, is remarkably close to the Youth Fiction mean PC3 score of -0.21 (= -0.64 + 0.43), which suggests that, on this dimension, there are probably no meaningful linguistic differences between these texts. For ease of interpretability, PC3 scores as predicted by the model are plotted on Fig. 78.

<table><caption>Table 70: Summary of the model: lmer(PC3 ~ 1 + Level + Register + Level*Register + (1|Source))</caption><thead><tr><th>Predictors</th><th>Estimates</th><th>95% CI</th><th>p-value</th></tr></thead><tbody><tr><td>(Intercept) [Conversation] [Ref.]</td><td>-0.64</td><td>-1.99 - 0.71</td><td>0.354</td></tr><tr><td>Level [A]</td><td>4.25</td><td>2.82 - 5.69</td><td>&lt;0.001</td></tr><tr><td>Level [B]</td><td>3.09</td><td>1.66 - 4.52</td><td>&lt;0.001</td></tr><tr><td>Level [C]</td><td>2.12</td><td>0.69 - 3.55</td><td>0.004</td></tr><tr><td>Level [D]</td><td>1.64</td><td>0.21 - 3.07</td><td>0.024</td></tr><tr><td>Level [E]</td><td>1.29</td><td>-0.15 - 2.73</td><td>0.078</td></tr><tr><td>Register [Fiction]</td><td>0.43</td><td>-0.92 - 1.79</td><td>0.533</td></tr><tr><td>Register [Informative]</td><td>0.43</td><td>-0.96 - 1.83</td><td>0.544</td></tr><tr><td>Level [A] * Register [Fiction]</td><td>-1.5</td><td>-2.89 - -0.12</td><td>0.033</td></tr><tr><td>Level [B] * Register [Fiction]</td><td>-1.39</td><td>-2.76 - -0.01</td><td>0.048</td></tr><tr><td>Level [C] * Register [Fiction]</td><td>-1.34</td><td>-2.71 - 0.03</td><td>0.056</td></tr><tr><td>Level [D] * Register [Fiction]</td><td>-1.35</td><td>-2.72 - -0.03</td><td>0.055</td></tr><tr><td>Level [E] * Register [Fiction]</td><td>-1.03</td><td>-2.41 - 0.35</td><td>0.142</td></tr><tr><td>Level [A] * Register [Informative]</td><td>-1.92</td><td>-3.35 - -0.49</td><td>0.008</td></tr><tr><td>Level [B] * Register [Informative]</td><td>-1.45</td><td>-2.86 - -0.03</td><td>0.045</td></tr><tr><td>Level [C] * Register [Informative]</td><td>-1.36</td><td>-2.77 - 0.05</td><td>0.058</td></tr><tr><td>Level [D] * Register [Informative]</td><td>-1.43</td><td>-2.84 - -0.02</td><td>0.047</td></tr><tr><td>Level [E] * Register [Informative]</td><td>-1.53</td><td>-2.95 - -0.11</td><td>0.034</td></tr><tr><td colspan="4" style="text-align: center;">Random Effects</td></tr><tr><td colspan="4">σ<sup>2</sup> 0.52</td></tr></tbody></table>
<table><tr><td>τ<sub>00 Source</sub></td><td>0.48</td></tr><tr><td>ICC</td><td>0.48</td></tr><tr><td>N Source</td><td>325</td></tr><tr><td>Observations</td><td>4980</td></tr><tr><td>Marginal R<sup>2</sup> / Conditional R<sup>2</sup></td><td>0.425 / 0.700</td></tr></table>

<figure><img src="image_84.png" /><figcaption>Fig. 78: Predicted PC3 scores of the texts of the TEC and the reference corpora</figcaption></figure>

At first glance, the clear proficiency level patterns displayed in Fig. 78 would suggest that the negative loading features on this third dimension are all advanced linguistic features that are typically not taught until after learners have been acquainted with more basic lexico-grammatical phenomena. Looking at Fig. 79 and Table 69, however, we find that the linguistic features that make the greatest negative contributions to PC3 scores include subordinate clauses (THSC and WHSC) and causative subordinators (CUZ) per 100 finite verb phrases, as well as the frequency of attributive adjectives (JJAT) and determiners (DT) per 100 nouns – which are all features that are introduced early in the curricula followed by the textbooks included in the TEC. Only split auxiliaries (SPLIT) and the perfect aspect (PEAS) further
down the list are unambiguous examples of features not usually introduced until the
third year of secondary EFL instructions.

<figure><img src="image_85.png" /><figcaption>Fig. 79: Features that make the most important contributions to the third and fourth dimensions</figcaption></figure>

Some of the features that characterise the positive end of the third dimension are
reminiscent of the instructional end of the first dimension of the intra-textbook PCA-
based model (see 7.3.1.1): e.g., imperative verbs (VIMP), WH-questions (WHQU),
and second-person referents (SPP2). Indeed, many of the texts that score particularly
high on the present third dimension model classroom dialogues. Such dialogues also
include high frequencies of the modal verb *can* (MDCA), *yes/no* questions (YNQU),
and politeness markers (POLITE) which also contribute to high PC3 scores, e.g.:

(231) **What's your name young man?**
I'm Chad, Sir, Chad O'Malley.
Well I'm Mr Lloyd, the headmaster. **Remember** my name. Now, **act** your age
and **stop** chatting!
Patrick, **can you** open the window, **please?**
**Sorry** Mrs Preston, I'm late. **Can I come in?**
It's alright for today Scarlett, but don't **forget** the room number next
time. Well, children, let's start. Let's talk about kings and queens!
**Can you** give me the name of...the Queen of England?
Queen Elizabeth!
Well. That's easy. More difficult... **Can you** give me the name of a
Norman king... Tom! **Can you** repeat the question?
... a Norman king!
William the Conqueror!
Congratulations Scarlett!
Wow!
And now [a] question for the champions. Can you tell me who this man is?
He had six wives. His first wife was Catherine of Aragon, his second
wife was Ann Boleyn and...
Miss... Miss... I know! I know!
Yes Patrick?
It's King George.
No, it isn't.
Scarlett! Can you answer?
Henry VIII.
Brilliant! Join the school history club!
She's incredible! <TEC: Join the Team 6°>

In general, this third dimension indicates that, as opposed to non-textbook language, Textbook English is characterised by high lexical density (LD) and, in particular, a high frequency of nouns (NN). Encouragingly, Fig. 78 shows that, on this 'Pedagogically adapted vs. Natural' dimension, texts from more advanced textbooks tend to score most like their corresponding reference corpora, although the gap between Level E textbook dialogues and the Spoken BNC2014 remains alarmingly wide.

The fourth dimension to emerge from this 'Textbook English vs. real-life English' PCA accounts for just 3.42% of the total variance. However, it is considered of relevance in the present study because it further contributes to identifying aspects of Textbook Conversation that distinguish this text variety from naturally occurring conversation. Whilst the ellipses of the other two TEC ellipses largely overlap with those of their corresponding reference corpora, it is clear from Fig. 76 that the vast majority of Textbook Conversation texts score higher on this dimension than those of the Spoken BNC2014. As shown on Fig. 79, the features that characterise its positive pole include conditional clauses (COND), the (semi-)modals *must*, *need* (MDNE), *will*, *shall* (MDWS), *may* and *might* (MDMM), the progressive (PROG) and perfect aspect (PEAS) and imperative verbs (VIMP), whilst its negative end is associated with causative subordinators (CUZ), hedges (HDG), and the past tense (VBD). The texts that score highest on this dimension are informative texts that provide advice (e.g., many texts from the teenkidsnews.com and teenvogue.com subcorpora of the Info Teens corpus) whilst those that score lowest are factual texts reporting on historical events (e.g., many of the texts of the factmonster.com, and historyforkids.net subcorpora of the Info Teens corpus). This could hint at a 'Factual vs. Speculative' dimension, were it not for the fact that hedging devices (HDG) are a major contributor to the 'factual' end of the dimension. Thus, this dimension does not lend itself well to a purely functional interpretation. In fact, it foremost differentiates between texts with simple verb forms and those with complex ones (modals, perfect and progressive aspects, etc.). The result is that a register that generally employs high rates of simple verb forms per FVP, e.g., natural face-to-face
conversation, scores low on this dimension whilst textbooks' synthetic representations of conversation feature many more complex verb forms and therefore score higher.

Given that this fourth dimension (at least partly) captures verb phrase complexity, it is not surprising that, as in the third dimension, it is considerably influenced by the different target proficiency levels of the textbooks. On this fourth dimension, the proficiency level effects are evidently driven by the fact that many of the aforementioned positive-loading features representing more complex verb forms are not introduced until the second or third year of English tuition. The mixed-effects models computed for PC4 scores make clear that textbook proficiency level affects dimension scores differently depending on the register under study. These effects are visualised in Fig. 80. Thus, we observe that, as the proficiency level of the textbooks increases, the dimension scores of the Fiction and Informative textbook subcorpora converge to that of their corresponding reference corpora whereas, curiously, the opposite pattern emerges for Textbook Conversation. The driving forces behind this effect will be examined in the following section (7.3.2.1), which explores the linguistic differences between the texts of the Textbook Conversation subcorpus and those of the Spoken BNC2014.

<figure><img src="image_86.png" /><figcaption>Fig. 80: Predicted PC4 scores of the texts of the TEC and the reference corpora</figcaption></figure>
### 7.3.2.1 Textbook Conversation vs. the Spoken BNC2014

As seen in Fig. 81, the majority of Textbook Conversation texts differ quite substantially from the texts of the Spoken BNC2014 on the first ‘Spontaneous interactional vs. Edited informational’ dimension, which explains nearly a third of the total variance in the full data matrix. The projection of texts in Fig. 81 also shows that, whilst the reference conversation data forms a relatively tight cluster of texts (mean = 3.74, SD = 0.49), the ellipse of the conversation subcorpus of the TEC (mean = 1.56, SD = 1.25) spreads across a much larger area. On this first dimension, therefore, Textbook Conversation texts vary a lot more than the texts of the Spoken BNC2014 that resemble each other much more.

To explore this variation further, PC1 scores were modelled by entering Register, Level and their interactions in a mixed-effects model. The full model (see summary in Table 71) explains 92% of variation in PC1 scores (conditional R²). Removing the random slopes and intercepts accounting for different text sources (see 7.2.9) does not lead to a substantial drop in predictive power (adjusted R² = 87%). Table 71 confirms that Textbook Conversation at all proficiency levels scores significantly lower on PC1 than the Spoken BNC2014 (here, the reference level). The results also show that the proficiency levels of the textbooks do interact significantly with scores on this first dimension but, as already observed on Biber’s (1988) Dimension 1 in the additive MDA in 6.3.1.1, for the conversation register, not in the expected direction: the more advanced textbooks present dialogues that are even less similar to naturally occurring conversation than the beginner textbooks! This is illustrated on Fig. 81, which displays the predicted PC1 scores across the three reference corpora and their corresponding textbook registers subdivided by proficiency levels. It confirms that this unexpected finding is only true for the conversation register.

<table><caption>Table 71: Summary of the model: lmer(PC1 ~ 1 + Level + Register + Level*Register + (Register|Source))</caption><thead><tr><th>Predictors</th><th>Estimates</th><th>95% CI</th><th>p-value</th></tr></thead><tbody><tr><td>(Intercept) [Conversation] [Ref.]</td><td>3.71</td><td>2.46 - 4.96</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [A]</td><td>-1.73</td><td>-3.06 - -0.40</td><td><strong>0.011</strong></td></tr><tr><td>Level [B]</td><td>-1.65</td><td>-2.97 - -0.32</td><td><strong>0.015</strong></td></tr><tr><td>Level [C]</td><td>-2.08</td><td>-3.40 - -0.76</td><td><strong>0.002</strong></td></tr><tr><td>Level [D]</td><td>-2.47</td><td>-3.80 - -1.15</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [E]</td><td>-2.73</td><td>-4.06 - -1.40</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Fiction]</td><td>-4.20</td><td>-5.45 - -2.95</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Register [Informative]</td><td>-6.75</td><td>-8.03 - -5.47</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [A] * Register [Fiction]</td><td>2.18</td><td>0.86 - 3.49</td><td><strong>0.001</strong></td></tr><tr><td>Level [B] * Register [Fiction]</td><td>1.71</td><td>0.41 - 3.01</td><td><strong>0.01</strong></td></tr><tr><td>Level [C] * Register [Fiction]</td><td>2.12</td><td>0.83 - 3.42</td><td><strong>0.001</strong></td></tr><tr><td>Level [D] * Register [Fiction]</td><td>1.93</td><td>0.64 - 3.23</td><td><strong>0.003</strong></td></tr><tr><td>Level [E] * Register [Fiction]</td><td>2.41</td><td>1.10 - 3.71</td><td><strong>&lt;0.001</strong></td></tr></tbody></table>
<table><tbody><tr><td>Level [A] * Register [Informative]</td><td>3.37</td><td>2.01 – 4.73</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [B] * Register [Informative]</td><td>3.17</td><td>1.83 – 4.51</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [C] * Register [Informative]</td><td>3.24</td><td>1.90 – 4.57</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [D] * Register [Informative]</td><td>3.20</td><td>1.87 – 4.53</td><td><strong>&lt;0.001</strong></td></tr><tr><td>Level [E] * Register [Informative]</td><td>3.14</td><td>1.80 – 4.48</td><td><strong>&lt;0.001</strong></td></tr><tr><td colspan="4" style="text-align: center; font-weight: bold;">Random Effects</td></tr><tr><td>σ<sup>2</sup></td><td>0.59</td><td></td><td></td></tr><tr><td>τ<sub>00</sub> Source</td><td>0.41</td><td></td><td></td></tr><tr><td>τ<sub>11</sub> Source . RegisterFiction</td><td>0.12</td><td></td><td></td></tr><tr><td>τ<sub>11</sub> Source . RegisterInformative</td><td>0.20</td><td></td><td></td></tr><tr><td>ρ<sub>01</sub></td><td>-0.05</td><td></td><td></td></tr><tr><td>ICC</td><td>-0.48</td><td></td><td></td></tr><tr><td>N Source</td><td>0.41</td><td></td><td></td></tr><tr><td>Marginal R<sup>2</sup> / Conditional R<sup>2</sup></td><td>0.870 / 0.923</td><td></td><td></td></tr></tbody></table>

<figure><img src="image_87.png" /><figcaption>Fig. 81: Predicted PC1 scores of the texts of the TEC and the reference corpora</figcaption></figure>

We have already noted that the first dimension to emerge from the present PCA-based MDA bears many similarities to Biber's (1988) Dimension 1. As shown in Fig. 74 and Table 69, many of the features with particularly high and low loadings are
shared. In particular, the upper end of the dimension is dominated by texts that contain high frequencies of many features of unplanned speech, e.g., discourse markers (DMA), fillers and interjections (FPUH) and causative subordinators (CUZ), as well as features typical of social interaction, e.g., questions and question tags (YNQU, WHQU, QUTAG), first-person and second-person references (FPP1S, FPP1P, SPP2), e.g., (232). All of these features are less frequent in Textbook Conversation than in the Spoken BNC2014, which is one of the reasons why, on average, the dialogues of the TEC score significantly lower on PC1 than the transcripts of the Spoken BNC2014, e.g., (233). As already observed in 6.4, this is likely due to the fact that, in their carefully crafted, scripted dialogues, textbook authors very rarely model any disfluencies in spoken interactions, e.g., in the form of hesitations, interruptions, backtracking, or repair. In addition, many Textbook Conversation texts are much more like mini-monologues than genuine interactions. On average, turns are much longer and feature far fewer signs of unplanned speech, e.g., (234).

(232) yeah well there's a few things I need to do in um town
mum was thinking of dropping me off at Zumba and then going
yeah well I need to pay those cheques in
can you pay my cheque in? and get some money out for the rent. can you
pay my cheque in?
yeah
thanks
pay your cheque in and anything else we need in town?
I can't think but we're gonna have to get up early
what? How early's early?
well I well not too early we just leave at nine we have to be ready and
I have to have my breakfast before then so I can
okay
but I'm not too sure if I'm gonna go to Zumba or come to town with you
right
but either way we're going to town at nine
okay what do you do in Zumba? What is it?
dance it's just that you remember those those videos you used to have
like aerobic dance videos
well I never had any but yeah I know what you mean
no no you've seen me do them before <BNC2014: SWU3>

(233) What are you up to at the weekend, Toby?
I'm going to go for a bike ride on Saturday. Do you fancy coming too?
I can't, I'm afraid. I'm going to help my dad with some gardening. We're
going to do some work for a neighbour.
That doesn't sound like the best way to spend your weekend. Gardening is
hard work! And according to the forecast, the weather isn't going to be
good.
I know. But the neighbour is going to pay us for it. And my dad's a
gardener so he's got all the right tools.
Really? I'll come and help you. I mean, if that's OK with you and your
dad...
Sure. We'll share the money with you: £10 an hour. But what about the
bike ride?
I'll go on Sunday instead. The weather will probably be better then. Do
you want to come?

Yes, please. I love bike rides. But let's go in the afternoon. I'll be
exhausted when I wake up! <TEC: Solutions Intermediate>

(234) **That's right.** This passion for having a perfect body is definitely a negative product of Hollywood. There seems to be an unwritten rule that in order to make it in the movies you must be beautiful. Your looks are almost like your business card and are often the only way to make a good first impression. But things get dangerous when they're taken to extremes. Examples of this are the body builders at Venice Beach, like in this picture, or people who spend tens of thousands of dollars a year on taking care of their looks and on plastic surgery - and they're not just celebrities. But is taking care of your body all bad?

As June said, it's when things are taken to extremes that they're harmful. There's also a very positive side to this Californian symbol. A lot of new kinds of sports were invented in California - here are some examples - and a lot of Californians lead very active and healthy lifestyles. In addition, California is often the country's leader when it comes to making public health laws. The Golden State was the first to ban smoking in workplaces, bars and restaurants in 1998 and now about half of the states in the US have similar laws. The state is also very active in limiting air pollution. But **what about** traffic in places like L.A.? <TEC: Green Line New 5>

Another factor that contributes to lower PC1 scores is the fact that some of the features with high loadings on PC1 are more likely to be used by language users who are physically in the same space, e.g., demonstratives (DEMO) and the pronoun *it* (PIT). Such features are equally frequent in the transcripts of the textbooks' video materials as in those of the Spoken BNC2014, but less frequent in audio-only materials and it is worth nothing that audio materials, which make up the largest proportion of the Conversation subcorpus of the TEC. By virtue of sharing a common environment, the speakers of the Spoken BNC2014 can also afford to resort to much vaguer language. This difference is also captured on this first dimension. In addition to the extensive use of demonstratives and the pronoun *it*, it is observable in the considerably more frequent use of quantifiers (QUAN), quantifying pronouns (QUPR) and hedges (HDG) than in Textbook Conversation.

(235) oh right so you **sort of** get so you **sort of** get things that **kind of** yeah
I guess localised

yeah so you got a potential for these vast metropolises filled with
**loads of like** diverse em cultures and backgrounds **and stuff** but you end
up getting **these** close communities em so I wanted to explore **that** idea
and and in the video at the end when asked how what what would be the
role of an artist em to explore **this** idea or to confront **it** and em he
suggested that perhaps the artist could expand em **sort of** enlarge the
the garden gnome to the size of the statue of liberty **or something** just
just poke fun of out of **it** and em so so that that was part of the
reasoning for **this** first art project that I was going to do em and
entering into of course what I thought I was going to be doing which was
creative education they asked us to get completely obsessed with
something just one particular theme
oh right
just to explore what **it** was to get obsessed and eh I'd recently
bought Minecraft so I decided I'd go for **that** and just learn absolutely
everything about **it** and em eh **something** that came across when I was
looking at **bits and bobs** is that a **lot of** people on Minecraft servers
are in their own little worlds they tend to make **these** monuments to
Mario **it's** a **It's** a very easy statue to make there's there's pixel art
of Mario and **it** happens **all** the time so I had **sort of** combined those
ideas in my head and thought it would be a **bit of** fun to make a **sort of**
combination of a garden gnome the statue of liberty and super Mario
oh wow
in eh on on the server and eh so when I announced **it** to people that **sort
of** got excited about **it** and **sort of** made an island for **it** to sit on and
eh **some** people were trying thought they might make a little **sort of**
hidden illuminati style base
wow <BNC2014: SCJL>

Since the positive- and negative-loading features that contribute to each dimension
are complementary, it is important to not only consider which positive high-loading
features are absent from or less frequent in the Conversation subcorpus of the TEC
to understand the nature of Textbook Conversation, but also which negative-loading
ones are markedly more frequent in Conversation subcorpus of the TEC than in the
reference Conversation corpus. These include nouns (NN), prepositions (IN), longer
words (AWL), a higher lexical density (LD), and lexical diversity (TTR). As already
noted in 6.4, this shows that, on average, Textbook Conversation is considerably more
nominal than naturally occurring conversation. The fact that dialogues from more
advanced textbooks score even lower than beginner textbooks on PC1 is due to higher
z-scores of these features. Excerpts (227), (228) and (234), for instance, are examples
of particularly low-scoring texts from advanced textbooks (level E), whereas (236)
stems from a second-year secondary school textbook and, on this dimension, is
situated in the same region as the majority of the Spoken BNC2014 texts. It features
much shorter turns than the average textbook dialogue.

(236)      **This** is lovely.
             Isn't **it** lovely?
             What is **it** exactly?
             **It's** a coffee machine.
             **Oh, yes.** Of course. Is **it** battery powered?
             **No, it's** mains powered.
             Look, the cable's here, under the base. If **you** press **this** button, the
             plug appears.
             **That's** clever. I love **it**. **It's** perfect for my kitchen at home. I'll come
             back later today and buy **it**.
             Would **you** like to try a cup before **you** go?
             I'm sorry?
             A cup of coffee?
             **Oh, no** thanks. I never drink coffee. Horrible stuff.
             <TEC: Solutions Pre-intermediate>
As already explained in 7.3.2, the distribution of texts along the third dimension (PC3) confirms that, as opposed to real-life conversation, Textbook Conversation tends to feature exclusively well-formed polite interactions, with very few disfluencies. Whilst the dialogues of the more advanced textbooks of the TEC score closest to the reference Spoken BNC2014 texts (see Fig. 81), a noticeable distance can nevertheless be observed between the two varieties. On this third ‘Pedagogically adapted vs. Natural’ dimension, it is mostly driven by the high nominal content of Textbook Conversation leading to high noun counts (NN), high lexical density (LD), and proportionally fewer of those nouns being qualified by an attributive adjective (JJAT), as well as generally fewer causative subordinators (CUZ), subordinate clauses per finite verb phrases (THSC and WHSC) and general adverbs (RB). In addition, the modals *could* and *would* and phrasal verbs (as imperfectly captured by the RP variable) are, across all proficiency levels, less frequent in Textbook Conversation than in the Spoken BNC2014, e.g., (237). These three features also contribute to negative PC3 scores.

(237) I **would** like something to extract all the vegetable
I don't know I I like the idea of eating **very** little meat I **really** love
molluscs and they can't and no one else even likes them why can't I have
them?
I I doubt they're the worst I mean
do you think should I investigate **whether** the fishing practices **that** get
molluscs are harmful to the environment?
yes I also think just reducing the amount you **would** eat **would** be doing a
favour to the animals in some ways
but you should also look at erm **who** was involved with the farming
industry problem okay? there might be [...]
there might be people treated **badly**
mm mm but that's almost in everything and once you **really** get **down** to it
**exactly**
like you you **basically**
it's terrifying
stop eating
so that's **probably** so that's why I was a bit like well I **couldn't** commit
myself to this erm
no no
lab meat thing **cos** there'll be cruelty somewhere
yeah and
**cos** there's cruelty in everything there's **probably** cruelty in this
potato.
cruel-free parsnips
and if you eat things that are **only** grown or made **locally** it's **very**
expensive. mm mm
you know erm you know if if I had all the money in the world it **would** be
very nice <BNC2014: SFC2>

On the fourth dimension, more advanced Textbook Conversation texts are situated
further away from the reference Spoken BNC2014 than beginner and intermediate
ones (see Fig. 80). One of the driving factors behind this surprising finding is certainly
that textbook authors use printed dialogues and audio and video materials to
introduce new vocabulary and “recycle” previously introduced lexical items, leading
to higher type/token ratios in more advanced textbooks, even though real-life
conversations tend not to be characterised by particularly high lexical diversity, e.g.,
compare (237) and (238).

(238) Good morning Mr. Stone. Good morning. So, first, could you tell us more about the movie?

Sure, well... As a historian, Modern Times seems particularly interesting to me because Chaplin showed the effect of mechanization on the lives of people in the modern industrialized world. The film is about the hardship of an ordinary man who struggles to survive in the depressed economy of 1930s America. The factory scene is just brilliant! It is exactly what modernity was about at that time: the assembly line, the division of labour, mass production and the daily grind of many industrial workers. They were exploited by their bosses, who made them work from dawn to dusk in very tough conditions. The film shows the workers' repetitive labour and how some of them went crazy as they repeated the same task over and over again, tightening bolts for instance.

Some people say that industrialization has only had negative effects and they are nostalgic about the past, when things were simple and technology had not spoiled everything yet. Do you agree with that?

No, I don't quite agree with them because industrialization was also a progressive revolution; it did have very positive effects. [...] <TEC: Piece of Cake 4º>

7.3.2.2 Textbook Fiction vs. the Youth Fiction corpus

As already observed in the comparative additive MDA (see 6.3.2), of the three TEC
registers compared to reference corpora, Textbook Fiction is most similar to its
corresponding target reference corpus, Youth Fiction. Where differences are observed,
they are mostly due to beginner textbooks having not yet introduced more advanced
grammatical features. The strong proficiency level effects observed along the second
'Narrative vs. Non-narrative' dimension (see Table 72) are largely driven by the
absence of the past tense in Level A textbooks (see also 6.3.2.2). Other high positive-
loading features on PC2 that are almost entirely absent from Levels A and B
textbooks include the modals *could* and *would* (MDCO and MDWO), and the perfect
aspect (PEAS). In addition, the PC2 scores of Level A Textbook Fiction texts are
lower due to a strong over-representation of two features with some of the largest
negative-loading weights on this second dimension (see Table 69): the modal *can*
(MDCA) (median = 6.25 per 100 FVPs) as compared to Youth Fiction
(median = 1.65 per 100 FVPs) and BE as a main verb (BEMA) (19.2 per 100 FVPs
compared to 14.2). It is, however, important to remember that there are relatively
few Level A texts in the Fiction subcorpus of the TEC so that these figures ought to
be interpreted with caution. Nonetheless, excerpt (239) constitutes a representative