ICLE - RC : International Corpus of Learner E nglish for Relative Clauses (2025)

Abstract: We present the ICLE-RC, a corpus of learner English texts annotated for relative clauses and related phenomena. The corpus contains a collection of 144 academic essays from the International Corpus of Learner English (ICLE; Granger et al., 2002), representing six L1 backgrounds – Finnish, Italian, Polish, Swedish, Turkish, and Urdu. These texts are annotated for over 900 relative clauses, with respect to a wide array of lexical, syntactic, semantic, and discourse features. The corpus also provides annotation of over 400 related phenomena (it-clefts, pseudo-clefts, existential-relatives, etc.). Here, we describe the corpus annotation framework, report on the IAA study, discuss the prospects of (semi-)automating annotation, and present the first results from our corpus analysis. We envisage the ICLE-RC to be used as a valuable resource for research on relative clauses in SLA, language typology, World Englishes, and discourse analysis.

## introduction relative clauses (henceforth rcs) are a type of subordinate clauses that typically modify nouns or noun phrases, and sometimes also adjectives foot_0 , adverbs foot_1 , pps foot_2 , vps foot_3 , and even entire clauses 5 . rcs in english (and beyond) have extensively been studied for a wide range of themes, such as syntactic and typological variation @xcite , semantic features @xcite , discourse functions @xcite , diachronic development @xcite , fla/sla @xcite , parsing @xcite , and processing @xcite , to name but a few of more recent work. in this paper, we present the icle-rc, a new corpus of english rcs and related phenomena. the latter includes constructions such as it-clefts, pseudo-clefts, and existential-relatives that employ words like that, which, or who, which are otherwise known as relative markers, frequently used to introduce relative clauses. the icle-rc uses a subset of the international corpus of learner english (icle; @xcite . the first version of the icle-rc contains 144 icle texts, covering six l1 backgrounds -finnish, italian, polish, swedish, turkish, and urdu -with 24 texts from each. these texts are annotated for 924 rcs, with respect to a wide array of lexical, syntactic, semantic, and discourse features. these texts are also annotated for 407 related phenomena, which we call other constructions (henceforth ocs). the paper is structured as follows: section 2 outlines the motivation behind the creation of the icle-rc. the composition of the corpus is described in section 3. we describe the annotation framework for rcs and ocs in section 4 and section 5, respectively. section 6 reports on an iaa study, and highlights challenges in our rc annotation. the prospects of (semi-)automating the rc annotation is discussed in section 7. we present the first results from our corpus analysis in section 8. related work is briefly described in section 9. section 10 concludes the paper with an outlook on the future work and applications of the corpus. ## motivation the development of the icle-rc stems from a number of reasons. first, the corpus would provide real language data to assess english learners' use of rcs against the standard rules of english grammars (e.g., the use of which for a human referent, or the use of a comma for integrated rcs). second, the six l1 backgrounds covered in the icle-rc represent six different language families @xcite ) -finnish: uralic; italian: romance; polish: slavic; swedish: germanic; turkish: turkic; and urdu: indo-aryan foot_5 . this would allow identifying typological patterns for certain rc features potentially resulting from cross-linguistic influence (e.g., the use of extraposed rcs). this would also offer significant implications for research in world englishes, in comparison to native varieties of english (e.g., by comparing the icle-rc with comparable corpora such as icnale @xcite as well as those of native academic english such as loc-ness @xcite ). third, the corpus would help us explore english learners' use of other constructions as alternative strategies of information structuring, in addition to rcs. finally, although corpus-based studies exist for english rcs, they have mostly used small-size data sets designed to tackle very specific rc-oriented issues (see section 9). to our knowledge, there is no large-scale corpus of english rcs with a feature-rich annotation framework. the icle-rc is designed to accommodate a wide variety of english texts, and support the annotation of rcs therein with a comprehensive coverage of linguistic features pertaining to lexical, syntactic, semantic, and discourse domains. ## data selection and setup of the corpus the icle-rc derives from the icle @xcite , which is a corpus of academic essays written by undergraduate students from a given set of topics. these students are intermediate or advanced learners of english, coming from different l1 backgrounds such as chinese, dutch, finnish, french, german, greek, hungarian, italian, japanese, polish, russian, spanish, swedish, turkish, and urdu. the data collection for the icle was initiated in the late 1990s, and has since been coordinated by sylviane granger at the centre for english corpus linguistics at the university of louvain. the corpus has grown over the years as a result of close collaboration with a large number of partner universities around the world. the most recent version of the corpus (iclev3) includes over 5.5 million words covering 25 l1 backgrounds 7 . the icle-rc includes 144 icle essays (100k+ words), which are equally distributed into 24 essays from six l1 backgrounds, namely finnish, italian, polish, swedish, turkish, and urdu. these 24 essays for each language are compiled from three institutions (with 8 essays from each), which are further balanced for the gender of the writer foot_7 , whenever possible. the detailed distribution of the essays in the icle-rc is provided in ## annotation framework for rc the relative clauses (rcs) foot_8 in the icle-rc are annotated for a wide range of lexical, syntactic, semantic, and discourse features. these features are grouped into seven primary categories, as listed in relative marker (rm): rms are words that introduce an rc. rms include the subordinator that and relative pronouns such as which, who, or whose. in the icle-rc, the rm feature includes three sub-features: that, wh-word, and zero (i.e., the absence of an overt rm for bare-relatives). these categories are exemplified below foot_9 . ## annotation framework for oc in addition to rcs (and their linguistic features), the texts in the icle-rc are also annotated for a wide range of ocs (other constructions). ocs either resemble rcs (particularly because of the use of words such as that and which) but are not rcs proper, or they are a special type of rcs. ocs comprise six types, as defined and exemplified below. it-cleft: in a cleft construction, a single clause is split up into two clauses, each containing its own verb. an it-cleft construction begins with a dummy it, which is typically followed by a copula and an np. the information in the it-clause is emphasised for the listener (foregrounded information). the clause that follows the it-clause is introduced by that (sometimes also which or who), and it contains information that is already understood (backgrounded information). relative-there: this feature refers to existential clauses (introduced by the dummy pronoun there) that are followed by an rc. fused relative: fused relatives are a special type of rc in which the referent and the relativised element are fused together instead of being expressed separately as in regular rcs. fused relatives are introduced by a wide range of rms (otherwise used in regular rcs), such as who(ever), what(ever), which(ever), or where(ever). ## reliability of annotation the icle-rc is aimed to offer gold-standard data, and is entirely created from human annotation. the possibility of pre-annotating the source texts using heuristics based on (dependency or constituency) parsing output from parsers was excluded due to their limited success on learner english data 13 . the icle essays typically contain grammatical errors, missing words, truncated or incomplete sentences, and non-standard usages, and our preliminary experiments based on spacy dependency parses were not sufficiently satisfatory. the rcs and ocs in the icle-rc were annotated by two annotators (two of the authors), who have many years of experience with various kinds of linguistic annotation. on average, the annotators took between 30 minutes and one hour to annotate a single essay (including revisions). the annotators used the uam corpustool (version 2.8.16) (o @xcite to perform the annotation. a screenshot of an rc-annotation in uam corpus-tool is provided in figure 1 in the appendix. 13 for an overview of applying (ud) parsers to learner data, see @xcite and @xcite . in order to test the reliability of the corpus, we conducted an iaa study. the annotators independently annotated all 24 texts for the polish part of the corpus. given our multi-layered, featurerich annotation scheme ( it was found that the two annotators individually identified 163 rcs and 157 rcs, respectively, while both identified 151 common rcs foot_12 . according to cohen's kappa @xcite importantly, the variation in agreement can be interpreted as indicative of the relative complexity of the annotation task for a target feature type. first, syntactic features (e.g., referent func-tion, marker function), in comparison to other feature types, are relatively more objective in nature. hence, their identification is quite straightforward, which caused a very high degree of agreement. second, the identification of rm (a lexical/syntactic feature) is quite uncomplicated when it is explicitly marked by that or a wh-word, but not necessarily the same when there is no overt rm (for bare-relatives). in our iaa study, the annotators also agreed overwhelmingly more on the presence of an rm than on their absences, which resulted in a higher degree of substantial agreement. third, the identification of referent type operates on a semantic/discourse level, which brings subjectivity into analysis. this is evidenced by a lower degree of substantial agreement between the annotators. for instance, (21) presents such a case in which the referent 'a merciful god' was annotated as entity by the first annotator, but as abstract-entity by the second annotator. @xcite we treat it like a valuable gift from a merciful god who enabled us to use our skills and abilities ... [polish; @xcite finally, restrictiveness presents an interesting case. restrictiveness distinguishes integrated and supplementary rcs, and is determined based on syntactic cues; e.g., use of a comma for supplementary rcs, or the non-use of that for supplementary rcs (according to standard english grammars). restrictiveness is also conveyed through discourse meaning, i.e., whether the rc presents an integral part of the meaning of the matrix clause, or as a separate, additional unit of information. in the icle(-rc), which is a corpus of l2 english student essays, the students did not seem to have strictly adhered to the standard grammatical rules for marking integrated and supplementary rcs. ( 22 ) presents such a case (an rc with who), where the annotators disagreed on identifying the restrictiveness value. in those circumstances, the icle-rc annotators had to rely only on the available discourse meaning, which invited a greater amount of subjectivity in the interpretation. the challenge of determining restrictiveness has also been addressed in the rc literature @xcite . ambiguities of this kind probably caused only a moderate degree of agreement between the annotators. ## (semi-)automating annotation the same classification set-up is used to train and predict the values on the second level of the taxonomy in ## first results an important theme of investigation in our work is whether/how different rc features (and subfeatures) vary across languages. for the purpose of illustration, we only provide the distribution of two features: rm and restrictiveness. first, 17 the official icle instructions stipulate ca. 600 words. by contrast, the occurrence of bare-relatives (with a zero marker) is found to be a highly frequent feature exploited by both the finnish and swedish students (about 20% of all rcs). furthermore, the distribution of the overt rms vary across these languages. for example, the subordinator that is used more frequently for finnish, swedish, and turkish. by contrast, italian, polish, and urdu show a more frequent use of a wh-word. furthermore, the distribution of the wh-words shows a consistent pattern across these languages, with which being the most frequent wh-word, followed by who and then where (albeit with a larger margin). the remaining wh-words (when, whose, or whom) occur rarely in the corpus. next, the distribution of rcs for restrictive-ness (in it might be the case that (some of) these observed variations originate from the ways rcs are structured in the corresponding l1s. this can be validated by thoroughly examining the rc-related ## related work although there are no large-scale corpora exclusively annotated for rcs, there exists a rich body of corpus-based studies on rcs in english. weichmann (2015) provides a detailed, usage-based analysis of rcs (in 500 texts, with 80,000 parse trees) in the british component of the international corpus of english (ice) 19 . @xcite corpus-based account of english grammar, among many other grammatical phenomena, describes the use and distribution of rcs in a variety of registers. more commonly, specific aspects of rcs have been 19 https://www.ice-corpora.uzh.ch/en.html subject to corpus-based scrutiny, such as modified entity @xcite , type of modification @xcite , relativisers and their functions @xcite , referents of rcs @xcite , (non-)humanness @xcite , restrictiveness @xcite , and bare-relatives @xcite . a significant line of research involves the analysis of rcs in historical corpora @xcite @xcite and diachronic changes in the use of rcs @xcite @xcite ). yet another important theme in rc research concerns the usage and variation of rcs in regional varieties of l1 english @xcite @xcite as well as in world englishes @xcite . finally, corpus-based research also explored phenomena related to rcs (ocs), such as pseudo-cleft @xcite and relativethere @xcite . ## conclusions and outlook the icle-rc is an extension of a subset of the icle, and it provides annotation for rcs and related phenomena, based on a comprehensive, multi-layered, feature-rich taxonomy. the first and present version of the icle-rc contains a collection of 924 rcs (and 407 ocs) from 144 academic essays, representing six l1 backgrounds and six corresponding language families. the annotations in stand-off xml format and the code for our classification experiments are available on github 20 . the corpus is now in the post-production stage, and will soon be published as an open-access resource. our future work includes expanding the size and coverage of the corpus by adding more texts for the existing six languages as well as incorporating texts from other l1 backgrounds (from the icle), representing new (sub-)language families, such as cantonese (sino-tibetan), dutch (west germanic), greek (hellenic), japanese (japonic), farsi (indo-iranian), russian (slavic), and tswana (bantu). the extended corpus would enable us to employ statistical modeling on the data and draw reliable and comprehensive conclusions about the use of rcs by l2 english users. we envisage that the icle-rc would be used as a valuable resource for research on rcs in various areas of linguistic analysis. in sla and language typology, the corpus would help identifying varying patterns in the use of english rcs by l2 learners, and checking whether those patterns result from specific l1 backgrounds, or they, for example, conform to those stipulated by the np accessibility hierarchy @xcite . the icle-rc can also be used to (re-)examine the properties of rcs in regional varieties of english, and validate or revise the resulting findings against the existing research in world englishes. furthermore, the corpus offers a rich repository of informationstructuring devices (ocs, in addition to rcs), and this would aid research on discourse structure, supporting the analysis of fore-/back-grounding strategies, discourse referents, discourse segments, and discourse relations.