Research Notes

What does
plurilingualism
mean for language
assessment?
Issue 78
September 2020
ISSN: 2632-3109

Research Notes
Issue 78/September 2020

Guest Editor
Graham Seed, European Research Manager, Cambridge Assessment English

EditorSian Morgan, Senior Research Manager, Cambridge Assessment English

Production Team
Alison French, Proofreader
John Savage, Publications Assistant, Cambridge Assessment English

Typeset in the United Kingdom by George Hammond Design

Research Notes: 78
CoNteNts

2

Editorial

5

What is plurilingualism and what does it mean for language
assessment?
Graham seed

16

Taking account of plurilingualism in Cambridge Assessment
English products and services
Graham seed and Martine Holland

26

Assessing learners’ bilingual skills across different levels of the CEFR

35

Functional use of multilingualism in assessment:
Opportunities and challenges

elena Anna Maria Gandini

Fauve De Backer, stef slembrouck and Piet Van Avermaet

43

The challenge of assessing plurilingual repertoires:
The EVAL-IC project
elisa Fiorenza and elena Diego-Hernández

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 1

Editorial

Bienvenue et chào mù’ng ban. zu dieser Ausgabe vonResearch Notes, mam nadzieję, że utaelewa un pocomás de lo que es el plurilingualism.
the original volume of the Common european
Framework of Reference for Languages (CeFR, Councilof europe 2001) pointed to the growth in importanceof plurilingualism in language learning. the online
publication of the CeFR’s Companion Volume in 2018,
which was printed in 2020 (Council of europe 2020),
fleshed out more of what this meant in practice by
providing scales and descriptors explicitly about thedevelopment of plurilingual competences.
the concepts of multilingualism and plurilingualism,
defined in the first article, have been frequently
discussed in the field of language education andlanguage assessment over the last 20 years. It has beena recurring theme of discussion within the Associationof Language testers in europe (ALte),a grouping of language testing organisations ofwhich Cambridge Assessment english (henceforth
Cambridge english) is an important member.
ALte’s 7th International Conference had been
due to be held in Madrid in April 2020, but
understandably had to be postponed to April 2021
due to the CoVID-19 pandemic, which disrupted
travel plans, let alone affected the health of millions
of people globally.
the theme of the conference was, and still will be,
Safeguarding the future of multilingual assessment,with one particular strand on Defining the construct ofmultilingualism in language assessment. this follows on
from previous ALte events focusing on the relevanceof multilingualism and plurilingualism in language
assessment, notably the immediately preceding
conference in Ljubljana, slovenia, in November 2019,
which had as its theme Monolingual testing in a
multilingual reality?, a discussion of an apparent
dichotomy in today’s language testing field.

2| CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78

Many of the papers submitted for the 7th International
Conference focus on the construct and the practiceof integrating plurilingualism into language
assessment, and this issue of Research Notes takes
up the discussion. In doing so, some of the authorsof papers submitted for the conference have written
articles for this issue, in the hope of shedding
important light on how to understand the plurilingual
dimension in language assessment. the authors
are from, or writing about, ALte Member andAffiliate organisations.
Language testing bodies, and Cambridge english is no
exception, are now engaging in the debate and arerecognising plurilingual abilities in language learners,
not only in theory but also in practice.
In the first article, I present plurilingualism and its rolein english language learning and assessment, making
the case that the integration of learning andassessment helps us understand the construct ofplurilingualism in language assessment. In addition,a simple framework is presented which aims tocategorise the different ways in which plurilingualcompetence is recognised in assessment.
the second article aims to show how Cambridge
english has begun to respond to the need forrecognising the value of learners’ plurilingual
repertoires in discussion with international bodies
and policy-makers, as well as in some of the projects,
products and tests that they create.
the final three articles each look at what may beconsidered a form of plurilingual assessment, reflecting
on the integration of learning and assessment in theirrespective, and very different, cases.
Gandini discusses how more than one language isused in assessment from an ALte Institutional Affiliate,designed to suit the real-life communicative needs ofpeople in the south tyrol border region of Italy.

©UCLES 2020

she explores the use of cross-linguistic mediation andreports on a study currently in progress.
De Backer, slembrouck and Van Avermaet thenexplore the use of test takers’ plurilingual repertoires
as an education and assessment accommodationin non-language school subjects in an attempt attest fairness. this project comes from an ALte
Member and Van Avermaet is one of ALte’s
Individual expert Members.

References
Council of europe (2001) Common European Framework of Reference
for Languages: Learning, Teaching, Assessment, Cambridge:
Cambridge University Press.
Council of europe (2020) Common European Framework of Reference
for Languages: Learning, Teaching, Assessment. Companion Volume
with New Descriptors, strasbourg: Council of europe.

Finally, Fiorenza and Diego-Hernández, who comefrom two different universities where ALte Members
and Affiliates are based, report on a project aimed at
encouraging and assessing language learners’ generalplurilingual abilities.
I hope this short collection of articles will provoke
and inspire current and future language educators
and assessors to adopt appropriate stances on
plurilingualism, relevant to the context for the learner.
Graham seed
Cambridge, september 2020

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78

| 3

4

| CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

What is plurilingualism and what does it mean for
language assessment?
Graham Seed european Research Manager, Research and thought Leadership Department, Cambridge Assessment english

A starting point: Integrated learning and assessment
Among others, Jones and saville (2016) argue that assessment should be informed by, and lead to,better learning, as part of an ecosystem of learning: ‘A systemic and ecological approach seeks complementarity:
informal classroom assessment and formal large-scale assessment should both contribute to the two key
purposes of assessment: to provide evidence of learning and evidence for learning’ (Jones and saville 2016:2;
emphases in original).
Jones and saville go on to state that this ecosystem of learning takes place both inside and outside the classroom.
King (2018:33) agrees, and encourages:… looking beyond the classroom for the sources of language learning. We know that much language
learning takes place informally – listening to music, playing games and watching films, using theInternet and communicating electronically, and increasingly in our multilingual cities in the diverse
street. How will educators respond to this reality, treating it not as diversion but as a major source ofknowledge and incorporating what learners bring with them from their outside world … ?
Placing the learner themselves at the centre of the ecosystem of learning and assessment, as well as the integrationof learning and assessment, have been recurring themes throughout recent editions of Research Notes (see issues
70, 75 and 77), as researchers seek to understand the different sources and contexts of learning that language
learners find themselves in. Understanding these contexts will hopefully lead to more meaningful and more
effective learning, teaching and assessment.

other languages in the world of the english language learner
one source in the context of language learning is to recognise the language ability that language learners alreadyhave when learning english. they already have at least one language, the language they use at home, but more
often than not, they may also have additional languages – if the language of schooling is different, if they speak a
regional dialect or variant, or if they speak another lingua franca, for example. this multilingual world ‘is the natural
way of life for three-quarters of the human race. [this] principle … has been obscured in parts of europe as aconsequence of colonial history. We urgently need to reassert it, and to implement it in practical ways, for, in themodern world, monolingualism is not a strength but a handicap’ (Crystal 2006:409).
In europe, recent migration, as refugees, economic or academic migrants, have added to the number of people who,
like Basque or Luxembourgish speakers, already usually have more than one language before they start learning
english. even when mobility has been reduced, such as in the coronavirus lockdowns of 2020, the use ofelectronically mediated communication has kept the need for operating in more than one language alive.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 5

the use of many languages may at first seem in stark contrast to the growing importance of english globally.
In the eU, 97% of school pupils learn english as their first foreign language (european Commission 2018b).
Perhaps more than three-quarters of academic journal articles are written in english (Montgomery 2013), and in
the 2019 shanghai Jiao tong index of universities, 19 out of the 20 top-ranked universities were in the UsA or UK,
english-language settings.1 this has led to the growth in english as a Medium of Instruction (eMI).
But it is increasingly being recognised that english is not enough. employability may demand english language skills
as a given – other language proficiencies as well as multicultural sensitivities can mark employees out. these skills
enable confidence in less familiar situations and domains that employees can find themselves in. Furthermore,using these linguistic resources – of the home language(s) and any other languages the individual knows – now has
to be seen as an aid, not an impediment, to learning and teaching english (Cenoz and Gorter 2013, Chabert andAgost 2020, Duarte and van der Ploeg 2019, Lau and Van Viegen (eds) 2020, Ziegler, Durus and sert 2013).
this means a change from the ‘native speaker standard’ as the ideal for learning english to the recognition thatknowing other languages is a tool for learning english. there is a move away from siloed language lessons in schools
to the practice of content and language integrated learning (CLIL), which evans (2018:1) states is a ‘more effective
approach than eMI’. there is an increase in recognition of knowing other languages as being useful tools in learning
english. this paradigm shift is becoming known as the ‘multilingual turn in education’ (Conteh and Meier (eds)
2014, May (ed) 2014). King (2018) summarises this as ‘a new model for language’:We are moving towards a new kind of paradigm for languages. this is based on a plurilingual andasymmetric model where not all language competence needs to be the same, and where experienceand learning out of school, whether in the street or on the Internet, will also contribute to thelanguage profile of the individual ... In this paradigm english as the major vehicular language ofcommunication has a key role to play. (King 2018:31).

terminology
Before proceeding, it is important to examine some key terms. the Common european Framework of Reference
for Languages (CeFR), first published in 2001 by the Council of europe who added to it with the 2020 Companion
Volume, sees a distinction between multilingual and plurilingual, as found in the King quote above.
Multilingualism is ‘the coexistence of different languages at the social or individual level’ (Council of europe
2020:28). For example, the fact that the United Nations has six official languages that all documents are available
in shows that it is multilingual; or that by speaking english and German I can be considered multilingual.
the fact that I can speak english and German, as well as a number of other languages to different proficiency levels,means that I can call myself plurilingual, defined by the CeFR as ‘the dynamic and developing linguistic repertoire of
an individual user/learner’ (Council of europe 2020:28). the Companion Volume goes on to describe this more:
the ability to call flexibly upon an inter-related, uneven, plurilinguistic repertoire to:lswitch from one language or dialect or variety to anotherlexpress oneself in one language and understand a person speaking anotherl

Call upon the knowledge of a number of languages to make sense of a text

1. www.shanghairanking.com/ARWU2019.html

6| CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

l

Recognise words from a common international store in a new guiselMediate between individuals with no common language, even with only a slight knowledge oneselflBring the whole of one’s linguistic equipment into play, experimenting with alternative forms ofexpression (Council of europe 2020:28).

Furthermore, just as the CeFR has provided reference descriptors of language ability in areas such as written
production or spoken interaction graded on a scale between ‘Pre-A1’ (the least proficient) to ‘C2’ (the mostproficient), the Companion Volume also provides two specific sets of descriptors, plurilingual comprehension andbuilding on plurilingual repertoire, which are attempts to exemplify what it means to have plurilingual abilities atdifferent levels of proficiency. table 1 shows some examples of these descriptors.
Table 1: Selected examples of descriptors about plurilingualism in the CEFR Companion Volume
Descriptor

Scale in which descriptor is found

CEFR level

Can recognise internationalisms to deduce meaning of
simple signs

Plurilingual comprehension

A1

Can understand short, clearly written messages andinstructions by piecing together what he/she understands
from the versions in different languages

Plurilingual comprehension

A2

Can use a word from another language in his/her plurilingual
repertoire to make him/herself understood in a routine
everyday situation, when he/she cannot think of an adequate
expression in the language being spoken

Building on plurilingual repertoire

A2

Can extract information from documents written in different
languages in his/her field to include in a presentation

Plurilingual comprehension

B1

Can alternate efficiently between languages in his/her
plurilingual repertoire in order to facilitate comprehension
with and between third parties who lack a common language

Building on plurilingual repertoire

B2+

In reading the very first paragraph of the editorial in this Research Notes issue, perhaps you were able to work out
the message through some of your linguistic abilities, or at least make a guess using contextual clues too. In thisway you might be able to claim your plurilingual comprehension ability is (at least) A2 according to the second
descriptor in table 1.
In academic literature on the subject, the concept of plurilingualism is also known as individual multilingualism,
individual bilingualism or other terms. one example of the concept in use is code-switching, described ‘as (part of)a verbal action, the alternating use of two or more “codes” within one conversational episode’ (Auer 1998:1),
with ‘codes’ at its most basic meaning languages and language variants, dialects and accents. A similar, butarguably slightly different term is translanguaging, originating in Wales by Williams (1994) and developed by Garcia
and others to mean ‘the deployment of a speaker’s full linguistic repertoire without regard for watchful adherence
to the socially and politically defined boundaries of named languages’ (otheguy, Garcia and Read 2015:283).

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 7

A multilingual turn in education policy?
As discussed in the section above, the academic field has pointed towards the value of this multilingual turn(especially in the 2010s), being able to cite studies of the value of promoting a multilingual – or plurilingual –agenda in education. some policy-makers have also begun to advocate such a stance.
In european policy-making, the 2002 Barcelona european Council envisaged all eU citizens being able to speak
their ‘mother tongue + two’ additional languages. Despite attempts to put this into practice, the First European
Survey on Language Competences (european Commission 2012) found that this objective had not yet succeeded,which in part led to the european Commission's adoption of the 2019 Council Recommendation on acomprehensive approach to the teaching and learning of languages. this stated that ‘all Member states haveacknowledged the need to enhance multilingualism and develop multilingual competence in the Union’(european Commission 2019a:17).
Gutierrez eugenio and saville (2017) document the progress of european policy-makers’ relationship with language
assessment, concluding that since 2015 there has been a shift towards recognising integrated language learning,teaching and assessment, with a focus on multilingual and plurilingual agendas.
there has recently therefore been a focus on language-aware schools by embracing translanguaging and plurilingual
education practices. this has been easier to implement in existing bilingual areas such as the Basque country(e.g. Department of education, Language Policy and Culture of the Basque Government and the sociolinguistics
Cluster 2013) and Valencia (Generalitat Valenciana 2018), but the european Commission has been keen to promote
innovative practices in this area (Le Pichon-Vorstman, siarova and szőnyi 2020).
other parts of the world have also recognised value in promoting more than the national language as a mode forstudy. the second article in this issue will investigate some ways in which Cambridge english has assisted this,as in many of these cases it is english that has been chosen as one of the languages that the education system
places value in, alongside national, regional and other globally important languages.

the multilingual turn in assessment?
the multilingual (or plurilingual) turn in education must affect such a turn in assessment too, with the starting
point being the integration of learning and assessment. Yet very little attention has been paid to plurilingualism in
the field of assessment: ‘the contrast between the expanding use of multilingual practices in pedagogy, and theabsence of multilingual approaches in assessment and evaluation measures is striking’ (schissel, De Korne andLópez-Gopar 2018). Gorter and Cenoz also believe in the link between learning and assessment in the context ofplurilingualism: ‘tests should match actual language practices and multilinguals use resources from their whole
linguistic repertoire. If teaching is going in the direction of a multilingual focus, assessment should also follow thesame path’ (Gorter and Cenoz 2017:243). the Association of Language testers in europe (ALte)’s recently revised
Principles of Good Practice (2020a) similarly agrees that assessment needs to correspond to the language contexts
that the learners find themselves in. this may include ‘different languages, in many different contexts with manydifferent other users, and [learners] will acquire new language to bridge communicative gaps that exist between
themselves and others. these interactions mean the language user becomes a plurilingual, pluricontextual language
learner, and sites the acquisition of language very much within a socio-cognitive understanding’ (ALte 2020a:9).

8| CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Figure 1: The role of language assessment focusing on the plurilingual language learner (ALTE 2020a)

Assessment therefore must be an opportunity to demonstrate the relevant language skills by observing
performance on a relevant and authentic task, and providing appropriate feedback to the learner, as well as toother stakeholders as appropriate. this model is shown in Figure 1.
However, to date it has been difficult to operationalise this construct within existing language assessment tasks.
What would be included in a relevant and authentic test task would depend on the contexts that each plurilingual,pluricontextual language learner finds themselves in, and that will vary from individual to individual. After all, eachlanguage learner has a unique plurilingual repertoire based not only on their proficiency in different languages, butalso on any exposure to other languages, however small, they have encountered during their life – ‘each individual
uses their different linguistic abilities in different ways in different situations’ (saville and seed forthcoming).
A further issue would be ensuring that assessors have similar plurilingual backgrounds to the test takers in order toreward the test taker appropriately, scoring reliably across multiple test versions (Lopez, turkan and Guzman-orth2017). Practical constraints for test providers such as the need for assessing multiple learners at one time add to anargument that true assessment of an individual learner’s plurilingual abilities is a near impossibility.
Assessing plurilingual abilities is therefore a complex issue and requires the construct of plurilingual assessment
to be defined. In fact, it is necessary to recognise that there is more than one type of plurilingual assessment,each serving a different purpose depending on the goal of what is to be assessed, and how that stems from andcontributes to further learning.

Assessment in plurilingual situations
to date, some assessments in plurilingual situations have been created. the literature and resources around thesemany different assessments have been analysed in order to categorise them into four broad areas, resulting in theframework shown in table 2. this framework can therefore help understand the different constructs of assessments
which tap into an individual’s plurilingual abilities and must be seen in the context of integrated learning andassessment.
Table 2: A framework of assessment in plurilingual situations
A. Drawing on one’s
plurilingual repertoire toaid learning or proving skills
in one named language(e.g. english)©UCLES 2020

B. Drawing on one’s
plurilingual repertoire toaid learning or proving skills in
more than one named
language

C. Drawing on one’s
plurilingual repertoire toaid learning or proving skills
in other subjects

D. Learning and developing
plurilingual competence tofunction with languages notknown, or only partiallyknown

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 9

A. Drawing on one’s plurilingual repertoire to aid learning or proving skills in one named language(e.g. English)
the ability to communicate in one named language (english, in this case) is a skill that people need to have as theircontext demands it, usually in use as a common language where neither side knows each other’s language, or incommunication with someone whose home language is english. Because of the necessity of using english in areal context, it is therefore valid to provide an assessment of someone’s ability to communicate in english.
the input language received (read or heard) is in english, and the output language produced (spoken or written)
should be in english. Note that ‘english’ can of course be substituted for any other language, particularly thoselanguages which are used in global communication: spanish, French, Portuguese and Mandarin may also be
considered examples of this.
A user of english, whose home language is another language, may however be internally, cognitively, using anotherlanguage to help them communicate in english: that is to say, they are drawing on their plurilingual competence,or repertoire, to process the communicative task at hand (Cenoz and Gorter 2013). As this is the case, testingorganisations have a duty to respond to this fact in the two ways described in the paragraphs below.
the first is that test takers’ plurilingual abilities should be recognised as a help, and not a hindrance incommunication, just as was acknowledged in the section about education above. this means not treating evidenceof plurilingual abilities as errors but as an imperfect attempt at communication in english. the case study in thenext article about Cambridge english Qualifications for young learners develops and exemplifies this point.
secondly, language testing organisations with tests where the input and output are in one and the same language,
must see these tests not in isolation but as part of a multilingual profile of evidence of language learning (savilleand seed forthcoming). the CeFR (2001, 2020) agrees by stating that learners can profile their language ability.
Rather than demanding full proficiency in a language to claim to be able ‘to speak’ a language, learners can indicate
which languages, and to which proficiency level, they can operate in. Figure 2 is taken from the CeFR Companion
Volume as an example of a plurilingual proficiency profile, which is also broken down into different language skills
or modes of communication (oral reception, written production, etc.).
Pre-A1

A1

A2

A2+

B1

B1+

B2

B2+

C1

C2

Above C2

EnglishGermanFrenchSpanishItalianFigure 2: A plurilingual proficiency profile (Council of Europe 2020:40)

Quality language testing organisations with tests in different languages should collaborate with each other so thatlearners are able to create their own plurilingual proficiency profile based on being able to prove their language
proficiency in each language relevant to their own context. ALte’s objectives are to encourage multilingualism
in this way, and the ALte Framework, an extract of which is shown in Figure 3 (ALte 2020b), is a grid whichenables tests of one language to be compared with tests of other languages, categorised according to CeFR level.
the comparability of the tests across different languages is achieved through linking to CeFR levels, and successfully
auditing tests through ALte’s quality management system to ensure that each of them meets ALte’s minimum
standards of quality (ALte 2020a:24–25).

10 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

ALTE Framework 2020

Language

Dutch
Nederlands

Organisation

Pre-A1

A1
ALTE BreakthroughA2
ALTE Level 1

B1
ALTE Level 2

The Interuniversity
Testing Consortium(IUTC)

Cambridge
Assessment
EnglishB2
ALTE Level 3

C1
ALTE Level 4Interuniversitaire
Taaltest Nederlands
voor Anderstaligen –ITNA – B2• Audit valid until:
15 June 2021

Interuniversitaire
Taaltest Nederlands
voor Anderstaligen –ITNA – C1• Audit valid until:
15 June 2021

Cambridge English:
Advanced• Audit valid until:
30 April 2023

Cambridge English:
Key• Audit valid until:
30 April 2023

Cambridge English:
Preliminary• Audit valid until:
30 April 2023

Cambridge English:First• Audit valid until:
30 April 2023

Cambridge English:
Key for Schools• Audit valid until:
30 April 2023

Cambridge English:
Preliminary for Schools• Audit valid until:
30 April 2023

Cambridge English:
First for Schools• Audit valid until:
30 April 2023

Cambridge English:
Flyers• Audit valid until:
9 April 2025

Cambridge English:
Business Preliminary• Audit valid until:
30 April 2023

Cambridge English:
Business Vantage• Audit valid until:
30 April 2023

C2
ALTE Level 5

Cambridge English:
Proficiency• Audit valid until:
30 April 2023

EnglishCambridge English:
Starters• Audit valid until:
9 April 2025

Cambridge English:
Movers• Audit valid until:
9 April 2025

Cambridge English:
Business Higher• Audit valid until:
30 April 2023•

Figure 3: Extract from ALTE Framework (ALTE, latest version 2020b)

B. Drawing on one’s plurilingual repertoire to aid learning or proving skills in more than one named language
Practical reasons demand that language learners take various standardised tests of named languages in order toprove proficiency of their own personal plurilingual profile. theoretically, a test could be devised to replicate theauthentic multilingual situations individuals find themselves, in order to create a plurilingual proficiency profile.
But the reality would mean creating numerous bespoke tests for each individual which would not be practicallypossible at present. therefore, the solution is to prove proficiency through a number of quality tests, as describedabove. However, there are some plurilingual situations which are the same for a number of individuals, and in thesecases, a test which combines two or more languages at the same time can be made a reality. the term for this,used by the CeFR and elsewhere, is mediation, or more specifically in this case, cross-linguistic mediation, and the
CeFR Companion Volume (Council of europe 2020) provides a number of illustrative descriptors for thiscommunicative purpose; see table 3.
Table 3: Example descriptors of cross-linguistic mediation from the CEFR Companion Volume (adapted from Council of Europe
2020:108, 111, 202)
Descriptor

Scale in which descriptor is found

CEFR level

Can provide a simple, rough, spoken translation into (Language B) ofshort, simple everyday texts (e.g. brochure entries, notices, instructions,
letters or emails) written in (Language A).

Mediation – translating a written text in speech

A2+

Can summarise in (Language B) the main points made during a
conversation in (Language A) on a subject of personal or current interest,
provided that the speakers articulated clearly in standard language.

Mediation – processing text in speech

B1

Can relay in writing in (Language B) the relevant point(s) contained in anarticle (written in Language A) from an academic or professional journal.

Mediation – relaying specific information in writing

B2©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 11such plurilingual situations often occur in border regions, where two or more languages are actively used.
south tyrol is such a region, and a language test which exemplifies what this means in practice is detailed by Gandiniin the third article in this issue.
Another type of plurilingual situation where this ‘B’ category of assessments is applied can often be found in tests of
the national language and a language used for global communication, often english. the KPG exams in Greece test
mediation skills between Greek and other languages, for example by writing an email in english in response to
having read a text in Greek. Plurilingual repertoires have to be deployed in order to function in this multilingual
test environment, aimed at replicating authentic tasks (see stathopoulou 2015).
the Austrian plurilingual oral exam for upper secondary vocational colleges (Piribauer et al 2015) involves
mediation across three languages: German, as the language of schooling, english, as the first foreign language,
and either French, Italian, Russian or spanish as the second. the assessment criteria explicitly reward ‘language
switch and interaction’.
C. Drawing on one’s plurilingual repertoire to aid learning or proving skills in other subjects
this categorisation of plurilingual assessments is of those primarily testing something non-linguistic, perhaps ascience subject (e.g. shohamy 2019) or maths (e.g. Heugh, Prinsloo, Makgamatha, Diedericks and Winnaar 2017).
Assessments in this category are given in two or more languages, with the test takers having a free choice as towhich language they use to answer the science, maths, geography or other subject questions. In this regard, the tests
may be seen as a natural assessment response to the educational method of CLIL. In this form of assessment, Gorterand Cenoz recognise that ‘participants are assessed differently according to their linguistic background and not as
deficient speakers of their second languages’ (2017:242).
A fundamental value in these tests is not only of plurilingualism but of fairness, in order to make sure that certain
test takers are not excluded from demonstrating their knowledge of a particular subject because their proficiency inthe favoured language of schooling is not good enough (shohamy 2011, shohamy and Menken 2015).
Cambridge’s critical thinking admissions test for Uzbek Presidential schools is an example of this type C category, andmore details on the test administered in either Uzbek, Karakalpak or english is given in the second article in this issue.
A development of the type C category is where test takers do not have to choose which language to use before thetest but during it. the test paper contains the questions given in more than one language at the same time, allowing
test takers to receive input in either language and produce output in either language, maximising the useof their plurilingual repertoires to answer the questions. De Backer, slembrouck and Van Avermaet’s article in thisissue provides some rich examples of this sort of assessment.
these assessments especially embody integrated plurilingual learning and assessment in all school subjects,
ensuring that language classes are not seen as separate silos but have worth in the whole curriculum.
D. Learning and developing plurilingual competence to function with languages not known, or only
partially known
Plurilingualism has been said to be an attitude as well as an ability (Piccardo 2017). While learning and assessment
using one’s plurilingual abilities can point towards increased proficiency in and across different languages, a casecan be made for developing plurilingual competence more generally – to have the confidence to be able to operatewith languages not known, or only partially known, as far as is possible. the CeFR Companion Volume (Council ofeurope 2020:157) points out some important concepts relating to their descriptors of plurilingual competence,
among which are:

12 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

l

All knowledge and experience of languages contribute to building up communicative competence.

l

the proactive capacity to use knowledge of familiar languages to understand new languages,
looking for cognates and internationalisms in order to make sense of texts in unknown languages –while being aware of the danger of “false friends”.

the value of demonstrating and encouraging these attitudes and behaviours in education – for children andlifelong – has been shown (Bak and Mehmedbegovic 2017) and it has led, amongst others, to the european
Commission’s call for language-aware schools (european Commission 2019b:3). Curricula, often language-neutral,that focus on the development of plurilingual (and pluricultural) competences have been emerging over the last
decade, often accompanied by a framework around which a curriculum is built.
the Framework of Reference for Pluralistic Approaches to Languages and Cultures (FRePA, or CARAP in French)(Candelier et al 2012) is one such example of a framework, detailing descriptors around developing plurilingual
knowledge, attitude and skills. PlurCur (Allgäuer-Hackl, Brogan, Henning, Hufeisen and schlabach 2018) showed
how a plurilingual curriculum was piloted in different schools across europe.
Assessment that comes from learning and leads to further learning is just as important in this category. However,assessment of attitudes and behaviours is different from language proficiency assessment, and therefore theassessments that were suggested in categories A, B and C, though helpful, may not be wholly relevant for thiscategory. Assessment techniques such as observations, monitoring, learning portfolios, language learning diaries
and self-assessment may be more appropriate (saville and seed forthcoming), though little attention has been
paid to assessment in plurilingual curricula. eVAL-IC, profiled in the final article of this issue by Fiorenza andDiego-Hernández, is however one example where this has taken place.

A digital future for authentic plurilingual assessment
the CeFR’s action-oriented approach has highlighted the value of an authentic task-based approach to learning andassessment of language communication (Piccardo and North 2019). the development of technological solutions inthis space has helped to create more quickly processed and more accurate assessments, for example by using
adaptive tests (Walczak 2015) and automarking of writing (Yannakoudakis, Øistein, Geranpayeh, Briscoe andNicholls 2018) and speaking (Wang et al 2018). In the current era, assessments should more authentically reflect
real-life communication – all the more so as a result of the change in lifestyle of many people across the world
due to the coronavirus lockdowns of 2020.
the recognition of plurilingual competence enables going further in communication than in just one language,
and this can be replicated to the learning as well as the assessment spaces using digitally assisted authentic tasks,
personalised to the learner and localised to the context as much as possible. the example of IeLts smart Learning,
given in the following article, can be viewed as a format for a tool which could be replicated in many different
geographies, and ties together recognition of plurilingual abilities, cultural sensitivities, motivation in learning,
integrated learning and assessment, with the use of digital technology.
Mobile devices, through the use of video capture, can be used to collate evidence of an individual’s plurilingual
abilities not just in education and assessment but in the other two ‘worlds of learning’ that Jones and saville (2016)
describe in the learning-oriented assessment model: the personal world and the social world. For example, a videoof a language learner using plurilingual resources to interact with locals and other tourists while on holiday could be©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 13

securely uploaded to a portfolio of evidence, so that a teacher or the user themselves would be able to assess theirlanguage learning and make decisions as to what should be the next focus in their language learning journey.
the use of technology is therefore able to capture far more evidence about not just language proficiency but also
plurilingual abilities than in the past, and this is likely to be the future of plurilingual assessment.

References
Allgäuer-Hackl, e, Brogan, K, Henning, U, Hufeisen, B and schlabach, J (2018) More languages? – PlurCur!: Research and practice regarding
plurilingual whole school curricula, strasbourg: Council of europe.
ALte (2020a) Principles of Good Practice, available online: www.alte.org
ALte (2020b) ALTE Framework, Version 32, dated 9 April 2020, available online: www.alte.org
Auer, P (1998) Code-switching in Conversation. Language, Interaction and Identity, London: Routledge.
Bak, t and Mehmedbegovic, D (2017) Healthy linguistic diet: The value of linguistic diversity and language learning across the lifespan, available
online: www.meits.org/policy-papers/paper/healthy-linguistic-diet
Candelier, M (Coord.), Camilleri Grima, A, Castellotti, V, de Pietro, J F, Lőrincz, I, Meißner, F J, schröder-sura, A, Noguerol, A and Molinié, M (2012)
FRePA – A Framework of Reference for Pluralistic Approaches to Languages and Cultures: Competences and Resources, strasbourg: Council ofeurope.
Cenoz, J and Gorter, D (2013) towards a plurilingual approach in english language teaching: softening the boundaries between languages,
TESOL Quarterly 47 (3), 591–599.
Chabert, A and Agost, R (2020) Communicative language teaching: is there a place for L1 in L2 learning?, European Journal of Language Policy
12 (1), 55–83.
Conteh, J and Meier, G (eds) (2014) The Multilingual Turn in Languages Education: Opportunities and Challenges, Bristol: Multilingual Matters.
Council of europe (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge: Cambridge
University Press.
Council of europe (2020) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume with
New Descriptors, strasbourg: Council of europe.
Crystal, D (2006) How Language Works, New York: the overlook Press.
Department of education, Language Policy and Culture of the Basque Government and the sociolinguistics Cluster (2013) Talking Pupils: theArrue Project 2011, Vitoria-Gasteiz: eusko Jaurlaritzaren Argitalpen Zerbitzu Nagusia.
Duarte, J and van der Ploeg, M (2019) Plurilingual lecturers in english medium instruction in the Netherlands: the key to plurilingual approaches
in higher education?, European Journal of Higher Education 9 (3), 268–284.
european Commission (2012) First European Survey on Language Competences: Final Report, Brussels: european Commission.
european Commission (2019a) Proposal for a Council Recommendation on a comprehensive approach to the teaching and learning of languages,
Brussels: european Commission.
european Commission (2019b) Commission Staff Working Document Accompanying the Document ‘Proposal for a Council Recommendation on acomprehensive approach to the teaching and learning of languages’ Part 1, Brussels: european Commission.
evans, M (2018) Advantages of Multilingualism: What is the impact on school learning? Cambridge education Research Reports, University ofCambridge, available online: www.cambridge.org/files/8215/5005/8132/Advantages-of-multilingualism.pdf
Generalitat Valenciana (2018) LLEI 4/2018, de 21 de febrer, de la Generalitat, per la qual es regula i promou el plurilingüisme en el sistema educatiu
valencià. [2018/1773] [Law 4/2018, 21 February, for regulation and promotion of plurilingualism in the Valencian educational system].
Gorter, D and Cenoz, J (2017) Language education policy and multilingual assessment, Language and Education 31 (3), 231–248.
Gutierrez eugenio, e and saville, N (2017) Policy review: The role of assessment in European language policy – a historical overview, available online:
www.meits.org/policy-papers/paper/the-role-of-assessment-in-european-language-policy-a-historical-overview
Heugh, K, Prinsloo, C, Makgamatha, M, Diedericks, G and Winnaar, L (2017) Multilingualism(s) and system-wide assessment: a southern
perspective, Language and Education 31 (3), 197–216.
Jones, N and saville, N (2016) Learning Oriented Assessment: A Systemic Approach, Studies in Language Testing volume 45, Cambridge:
UCLes/Cambridge University Press.
King, L (2018) The Impact of Multilingualism on Global Education and Language Learning, Cambridge Assessment english Perspectives, Cambridge:
Cambridge Assessment english, available online: www.cambridgeenglish.org/Images/539682-perspectives-impact-on-multilingualism.pdf

14 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Lau, s M C and Van Viegen, s (eds) (2020) Plurilingual Pedagogies: Critical and Creative Endeavors for Equitable Language in Education, Heidelberg:
springer International Publishing.
Le Pichon-Vorstman, e, siarova, H and szőnyi, e (2020) The Future of Language Education in Europe: Case Studies of Innovative Practices,
Neset Report, Luxembourg: Publications office of the european Union.
Lopez, A A, turkan, s and Guzman-orth, D (2017) Conceptualizing the Use of Translanguaging in Initial Content Assessments for Newly Arrived
Emergent Bilingual Students, Princeton: educational testing service.
May, s (ed) (2014) The Multilingual Turn Implications for SLA TESOL and Bilingual Education, New York: Routledge.
Montgomery, s L (2013) Does Science Need a Global Language?, Chicago: University of Chicago Press.
otheguy, R, García, o and Reid, W (2015) Clarifying translanguaging and deconstructing named languages: A perspective from linguistics,
Applied Linguistics Review 6 (3), 281–307.
Piccardo, e (2017) Plurilingualism as a catalyst for creativity in superdiverse societies: A systemic analysis, Frontiers in Psychology 8, 21–69.
Piccardo, e and North, B (2019) The Action-oriented Approach: A Dynamic Vision of Language Education, Bristol: Multilingual Matters.
Piribauer, G, Atzlesberger, U, Greinix, I, Ladstätter, t, Mittendorfer, F, Renner, H and steinhuber, B (2015) Designing and Implementing Plurilingual
Oral Exams: Framework for the Austrian upper secondary level oral leaving examination at colleges for higher vocational education, Vienna: CeBs.
saville, N and seed, G (forthcoming) Plurilingual assessment, in Lawrence, G, Piccardo, e and Germain-Rutherford, A (eds) Routledge Handbookof Plurilingual Education, London: Routledge.
schissel, J L, De Korne, H and López-Gopar, M e (2018) Grappling with translanguaging for teaching and assessment in culturally andlinguistically diverse language learning contexts: teacher perspectives from oaxaca, Mexico, International Journal of Bilingual Education and
Bilingualism.
shohamy, e (2011) Assessing multilingual competencies: Adopting construct valid assessment policies, The Modern Language Journal 95 (3),417–429.
shohamy, e (2019) ‘Language and Knowledge of Society’ requirements, and human rights, presentation given at Achieving equal opportunities for
All Migrants through Learning and Assessment Conference at Council of europe, strasbourg, 16 october 2019.
shohamy, e and Menken, K (2015) Language assessment: Past to present misuses and future possibilities, in Wright, W e, Boun, s and Garcia, o(eds) Handbook of Bilingual and Multilingual Education, Hoboken: Wiley-Blackwell, 253–269.
stathopoulou, M (2015) Cross-language Mediation in Foreign Language Teaching and Testing, Bristol: Multilingual Matters.
Walczak, A (2015) Computer-adaptive testing, Research Notes 59, 35–39.
Wang, Y, Gales, M J F, Knill, K M, Kyriakopoulos, K, Malinin, A, van Dalen, R C and Rashid, M (2018) towards automatic assessment of
spontaneous spoken english, Speech Communication 104, 47–56.
Williams, C (1994) Arfarniad o ddulliau dysgu ac addysgu yng nghyd-destun addysg uwchradd ddwyieithog [An evaluation of teaching and learning
methods in the context of bilingual secondary education], unpublished PhD thesis, University of Wales.
Yannakoudakis, H, Øistein, e A, Geranpayeh, A, Briscoe, t and Nicholls, D (2018) Developing an automated writing placement system foresL learners, Applied Measurement in Education 31 (3), 251–267.
Ziegler, G, Durus, N and sert, o (2013) Plurilingual repertoires in the esL classroom: the case of the european school, TESOL Quarterly 47 (3),643–650.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 15

Taking account of plurilingualism in Cambridge
Assessment English products and services
Graham Seed european Research Manager, Research and thought Leadership Department, Cambridge Assessment english
Martine Holland senior Assessment Manager, IeLts UK

english is necessary – but not sufficient
A first glance at the name Cambridge Assessment english implies that the focus is only on english. However,
especially in the last 30 years, the organisation has been keen to promote multilingualism, with the english
language playing its part in a diverse, multilingual and multicultural world.
since the increased recognition of plurilingual repertoires in learning and assessment following the publication of
the Companion Volume to the Common european Framework of Reference for Languages (CeFR) (Council ofeurope 2020), Cambridge english has naturally mainly responded within category A of the framework outlined intable 2 in the first article of this issue, that is to say the recognition of plurilingualism in learning-oriented
assessments that require input and output in one language, english. Where this happens, english is to be seen asnecessary – but english alone is not sufficient, in that it is part of a plurilingual profile of language ability that aperson can demonstrate.
this article gives an overview of some of the ways Cambridge Assessment english (henceforth Cambridge english)
takes account of plurilingual repertoires, more widely in society and policy, and also in the lives of the individuals
who interact with their products and services, using three particular case studies to exemplify this. the first section
looks at recognition of plurilingualism in the work that Cambridge english engages in through international
co-operation and projects, concluding with a case study of a bespoke project Cambridge english is involved inwith the Uzbekistan Presidential schools. the second is in integrated learning and assessment, examining the useof technology in a product called IeLts smart Learning. Finally, a revised Cambridge english Qualification for young
learners exemplifies how to take account of plurilingualism in more formal assessment contexts.

taking account of plurilingualism in international co-operation andpolicy-making contexts and projects
evidence of Cambridge english’s multilingual focus can be found through collaboration on the european andglobal stage, notably through the Association of Language testers in europe (ALte), of which Cambridge english,
at that time UCLes, was a key founding member following a visit to the spanish testing organisation at the
University of salamanca back in 1989 (ALte 2020b). ALte’s aims are to promote both quality in language
assessment and diversity in multilingualism, and since then Cambridge english has played a pivotal role in ALte,hosting its secretariat and contributing widely to the debate.
through its role in ALte, Cambridge english has been able to participate in events promoting multilingualism within
european institutions such as the annual celebration of the european Day of Languages at the european Parliament(Cambridge english 2018a) .

16 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

the european Commission also appointed a consortium led by Cambridge english and other ALte Members to carryout the 2008–2012 surveyLang project, which measured the language skills of 15-year-olds from across europe inorder to assess the progress towards the ‘mother tongue + two’ goal of the 2002 Barcelona declaration (european
Commission 2012). Cambridge english also led the 2015 Study on Comparability of Language Testing in Europe
for the european Commission (european Commission 2015) and provided strategy input for implementing
multilingualism (saville and Gutierrez eugenio 2016).
Collaboration with the Council of europe as well as its european Centre for Modern Languages (eCML) has takenplace, most notably with Cambridge english input into the CeFR, both in the initial publication in 2001 and theCompanion Volume over 2018 to 2020, which promote plurilingualism.
Cambridge english has not just worked at the europe-wide policy level, but at national and regional levels, helping
play a part in the promotion of a plurilingual education in both europe and beyond. It has supplied expertise andassessments in enabling education systems to develop a bilingual model so that learners have the essential
plurilingual repertoires that involve english to be able to enter higher education and find employment (Cambridge
Assessment english 2018b). examples are with spanish in Madrid (2018b:50) and Panama (2018b:68), and with
Arabic in egypt (2018b:72) and saudi Arabia (Cambridge Assessment english 2020a).
A CLIL approach was taken to improve the Mongolian and english language abilities in a state-funded curriculumbuilding support project in Mongolia (Cambridge Assessment english 2018b:22), just as in Kazakhstan, where a pilot
was run in 50 state schools across the country to train teachers and provide them with the tools to take thisapproach in learning and assessment (Cambridge Assessment english 2020b).
Also in Kazakhstan, education and curriculum standards, teacher support and assessment were developed for
the Nazarbayev Intellectual schools in the trilingual environment of Kazakh, Russian and english (Cambridge
Assessment english 2018b:20). selection for places at higher education is highly competitive and being able tooperate in english and the home language(s) in all subjects gives plurilingual learners an advantage.
the following case study shows a project which values learners’ language abilities to identify those suitable foradmission to education, but does so in a way that can be categorised in the ‘C’ column of the framework forplurilingual assessment described in table 2 of the first article in this issue, in that test takers are free to choose
the language of the test taken, thus helping achieve fairness in testing.
Case study: Uzbekistan Presidential Schools
As part of their selection tests, Cambridge Assessment Admissions testing provided tests aimed at measuring
applicants’ ability in thinking skills for entry into ‘Presidential schools’ – a prestigious network for schools which is
being established across Uzbekistan, the main goal being to select the brightest children in the country to be thefuture leaders of the country.
the project sponsors were keen to ensure that the tests provided opportunities for the brightest children fromacross the country. three different versions of the tests were designed for different school years (Grades 5 and 6;
Grades 7 and 8; Grades 9 and 10), and each version of the test was available in three languages: Uzbek, Karakalpak(spoken in Karakalpakstan, an autonomous republic within Uzbekistan), and english. Candidates were free tochoose whichever language version they wanted to take the test in. Providing the test in Karakalpak was seen as
important to give all children equal opportunities, and taking the test in english did not mean the candidate wasgiven any special recognition.
the 75-minute test was designed to target thinking skills (critical thinking and problem solving), as these skills arerecognised as extremely important for the modern day, as well as being curriculum-independent, not testing the©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 17

candidates’ ability to memorise information, but rather the skills required to think through unfamiliar problems,
working quickly and accurately. the test system was first used in July 2019 in four regions of Uzbekistan.
Table 1: % of candidature in each language by region (N = 11,473)
RegionEnglish

Uzbek

Karakalpak

Khiva

0.4%

99.6%

–

Namangan

1.1%

98.9%

–

Nukus (Karakalpakstan)

1.6%

36.0%

62.4%

tashkent

8.3%

91.7%

–

overall

2.8%

81.5%

15.7%

1

Prezident maktablariga kirish test sinovlari – Namunaviytest savollari hamda javoblar izohi

President mekteplerine kiriw testi - Úlgi test sorawlari
hám juwaplar túsindirmesi

Test A: Muammoga yechim topish

Test A: Máselege sheshim tabıw

Anora o‘z pullarini hisoblab ko‘rgach, uning 12000 so‘m ekanligini bildi. U barcha pullarini uch
qismga bo‘lib, 100 so‘mlik, 200 so‘mlik va 500 so‘mliklarga ajratib chiqdi. Natijada unda 100
so‘mlik pullar soni 200 so‘mlik pullarga nisbatan uch marta ko‘pligi, 200 so‘mlik pullar esa 500
so‘mlik pullarga nisbatan ikki marta ko‘pligi ma‘lum bo‘ldi.!

1

Anorada umumiy hisobda nechta pul birliklari mavjud?

Anora aqshalarin esaplap korip, oniń 12000 som ekenin anıqladı. Ol hámme aqshasın úsh
bólekke bólip, 100 somlıq, 200 somlıq hám 500 somlıqlarǵa ajıratıp shıqtı. Nátiyjede, onda 100
somlıq aqshalar sanı 200 somlıq aqshalarǵa qaraǵanda úsh esege kóp ekenligi, al 200 somlıq
aqshalar bolsa 500 somlıq aqshalarǵa qaraǵanda eki ese kóp ekenligi anıqlandı.
Anorada ulıwma esaplaǵanda neshe pul birlikleri bar?

A

42

B

48

A

42

C

60

B

48

D

72

C

60

E

84

D

72

E

84

Figure 1: Presidential Schools critical thinking test item in both Uzbek (left) and Karakalpak (right)

Table 2: Average score of candidates by language, broken down by school grade versions
Grade

Uzbek version

English version

Karakalpak version

Average

5–6

12.75

12.95

11.18

12.52

7–8

11.81

11.56

10.28

11.56

9–10

12.32

12.35

11.02

12.12

the results show the average score across the Uzbek and english candidature to have no significant difference(t(9501) = .69, p = .49). Despite a slightly lower average score for Karakalpak test takers, the Uzbek Ministry ofeducation was satisfied that, for this first administration, their results were broadly in line with candidates taking
the tests in Uzbek in that region. they particularly valued the inclusion of candidates from that background who
otherwise may not have been able to access these high-stakes tests in order to support learners’ further education.
this is achieved through the provision of appropriate test specifications and item writer training to raise awarenessof the cultural and educational context in Uzbekistan. there is also a strong focus on the iterative process of refining
various quality assurance processes in light of evidence from the first delivery of these tests. For the future,
the ministry is committed to continuing to provide the tests in both Uzbek and Karakalpak, and working with
Cambridge Assessment to provide selection tests suitable for the Presidential schools across Uzbekistan as they
move towards having one of these schools in each of the 14 regions across the country.

18 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

taking account of plurilingualism in integrated learning and assessment
the case study above shows the value of recognising the individual learner’s needs. Jones and saville (2016)
place the learner at the centre of the ecosystem of learning, where assessment is learning-oriented. ALte’s
Principles of Good Practice document (2020a) puts the plurilingual, pluricontextual language learner at the centreof assessment activity.
the first article in this issue recognised the existence of other languages in the world of the english language learner,
most notably their home language (variant or dialect) and other languages they encounter in their daily life, as wellas any additional exposure to other languages they have had in their lifetime. Cambridge english has promoted thevalue of this plurilingual competence in english language education, for example in the english Profile studies series(e.g. Ćatibušić and Little 2014) and through webinars (Cambridge Assessment english 2019).
the english Profile studies series also sought to highlight some of the challenges that certain language groups tend
to have based on the linguistic distance that needs to be breached between that language and english. In integrated
language learning and assessment contexts, it is important to understand the linguistic features that may provedifficult for someone learning english, but additionally the social and cultural differences. the following case study
is an example of how Cambridge english responded to these needs in an integrated learning and assessment
product: IeLts smart Learning.
Case study: IELTS Smart Learning
IeLts smart Learning (IsL) is a speaking-focused, learning-oriented assessment app designed for and sold in the
Chinese market, targeted towards teenagers. It is designed and owned by the IeLts partnership – Cambridge
Assessment english, the British Council and IDP – in collaboration with the Chinese technology firm iFlytek,
and was launched in April 2020.
IsL is made up of a series of units covering four CeFR levels from A1 to B2: each unit represents one week’s study
and covers one topic, such as holidays, healthy food, or technology. each unit is carefully scaffolded, moving fromread-aloud tasks at word and chunk level, through more extended read-aloud tasks, to translation, and finally to an
IeLts task for freer speech practice. Learners are given personalised feedback for each task on a range of features
including pronunciation, fluency, grammatical accuracy and lexical level.
the app is designed to support the communicative approach through placing the learners in real-life contexts where
possible, and also uses principles of gamification to support younger learners in developing motivation andconfidence.
Needs analysis
Before the design phase of app production, research was conducted into the key needs of learners and school
teachers in China, through exam data analysis (IeLts 2018) and focus group interviews. speaking was often found to
be a weaker skill among learners for the following reasons:lthere is a lack of practice and targeted feedback: Class sizes can be particularly large in China, meaning both that
communicative methodologies are more difficult and that learners do not often receive targeted feedback.
Focus groups reported carrying out speaking practice through drilling, and that some learners only received
targeted feedback when giving class presentations.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78

| 19

l

A lack of feedback leads to a lack of improvement: Analysis of IeLts exam results among Chinese test takers
suggests that fewer than 30% of Chinese test takers get the band score they need first time, and that about
50% will retake IeLts three or more times.

lA lack of practice and feedback leads to a lack of confidence: Focus groups reported a general lack of confidence
when speaking, making comments such as ‘Chinese students are too shy. Not brave enough to express ideas.’
It was also suggested that some teachers chose not to focus on speaking in class because they themselves
struggled with this skill.

these issues with speaking are being brought into focus with the upcoming revisions in the national China exams:
the Zhongkao (secondary school entrance exam) and Gaokao (university entrance exam) are rolling out new exam
specifications that include, for the first time, a productive spoken component. Both exams also include translation,
which would need to be addressed in our app for face validity.
Following proof-of-concept trialling and user feedback, it became clear that input such as context screens and task
instructions would need to be available in both english and Chinese, to serve a range of teaching and learning styles.
For example, some learners would spend limited time on these input screens, preferring to skim this content andfocus instead on the task itself. this style of engagement meant that these screens were being ignored when in
english, leading to a poor user experience and minimal activation of schemata.
Development
From these problems, it became clear that an app to develop speaking needed to support teachers by providing
self-study activities with a variety of task types and clear, targeted feedback, while also providing a safe andmotivating learning environment that included recognition of plurilingual competence. Learner and teacher focus
groups were held throughout the app’s development to make sure these problems were addressed. Additionally,
the app was tested by over 1,500 beta users, who completed surveys fortnightly over a 4-month period on avariety of areas, including their confidence and their perception of their own language progress.
Recognition of plurilingual competence in ISL
Instructions, context setting and feedback screens for each task will shortly be available in either Chinese or english.
Having this choice allows the learner to work with their own learning preferences, deciding for themselves which
language, or both, they prefer to receive information in. this also allows tasks to be set in authentic contexts atlower levels without the risk that such contexts will not be understood. From B1, the learner is encouraged to moveto english if possible, but messaging in the app makes it clear that they can return to Chinese at any point. this fits
into the overall philosophy of the app by encouraging the learners to take small risks while preserving a sense ofsafety and not negating their plurilingual competence.
In addition, each unit includes a translation task. this is partly because translation is part of Chinese national tests
and therefore fundamental to the value of the app. Importantly though, it is right to recognise that translation is in
itself a real-life communicative skill, included in the CeFR descriptors under mediation (see the first article in thisissue). each translation task puts the learner in a real-life context where translation is necessary, such as helping afriend talk to an assistant in a shop or facilitating information-sharing between two people from different cultures.
the IsL translation task has proved popular with learners, being rated the most useful of the different task types.

20 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Figure 2: Context setting prompts available in either language

IsL is still new on the marketplace, and so substantialuser data from the full launch is not yet available.
However, beta users gave the app very positive
feedback: having used the app for several months,
95% felt that it helped them to improve their oral
english, and 97% rated the feedback given as very
helpful (n=662). there are plans to continue toanalyse data and iterate app development to ensure
IsL remains as useful to users in the future.

Figure 3: Example of translation activity in ISL©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 21

taking account of plurilingualism in more formal assessment contexts
While apps such as IsL specifically integrate learning with assessment, the Cambridge english ‘traditional’ productsof exam papers remain a component of the ecosystem of learning and assessment, and have always taken into
account communicative ability along the lines of the socio-cognitive framework (Weir 2005). this has meant,for example, the focus on clear pronunciation rather than native-speaker-like pronunciation (Cambridge
Assessment english, no date) and the need for speaking examiners to be of sufficient proficiency rather thannative speakers.
the need for a plurilingual profile approach to assessment can be evidenced by the BULAts test, in use from 1998
to 2016, which had versions in english, French, German and spanish. the recent focus on a learner’s plurilingualcompetence has had an effect on taking this resource into consideration as a linguistic repair strategy in Cambridge
english exam papers, rather than seeing evidence of this as an error. the following case study shows how this is put
into practice in the revised Cambridge english A1 Movers exam.
Case study: Revised Cambridge English Qualifications for young learners
In 2014, Cambridge english commenced a revision project for Pre A1 starters, A1 Movers and A2 Flyers, as part oftheir continual improvement cycle. the primary aim of this revision project was to ensure that the exams continue
to reflect young learners’ english language achievement accurately (Albrecht and Dunlop 2018). the revisionproject was conducted in phases and involved the collection of quantitative and qualitative data from a range ofsources, such as questionnaires and interviews with key stakeholders; content and psychometric analysis of thetasks and papers; and reviews conducted by examiners, test developers and other assessment experts. the revised
Pre A1 starters, A1 Movers and A2 Flyers exams were launched in January 2018.
As part of the revision, new Reading/Writing tasks were introduced to A1 Movers and A2 Flyers, to give learners
further opportunity to demonstrate their reading and productive writing skills (Davies and Dunlop 2018). Part 6 of
the revised A1 Movers Reading and Writing paper requires candidates to complete sentences, answer questions andwrite full sentences about an illustrated scene. the items require written responses ranging from one or two words
up to full sentences. Part 7 of the revised A2 Flyers Reading and Writing paper requires the candidate to write a shortstory based on three pictures. the task requires candidates to write 20 or more words communicating a progressionof events based on the pictures. the assessment criteria for these two tasks emphasise comprehensibility overspelling and grammatical accuracy in line with what can be expected of learners at CeFR Levels A1 and A2(Cambridge Assessment english 2018c).
An example of the three pictures found in Part 7 is given in Figure 4 below:

Figure 4: A2 Flyers sample Writing picture prompt

22 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

one candidate from China replied with the following response:

A maximum of 5 marks is available, and the assessment criteria for achieving 5 marks is given in Figure 5.

Score

5

Criteria•

Response describes a progression of events which are explicitly connected to each other and•Response is based on all three pictures and•Minimal effort is required on the part of the reader to make sense of the response

Figure 5: Assessment criteria for a score of 5 in Part 7 of the A2 Flyers Writing test

the candidate above meets the criteria as shown in this mark scheme in that the response describes the progressionof events in all three pictures, and although the reader requires a small amount of effort to make sense of the
response, it is minimal.
It is clear that the candidate did not know the english word for ‘dolphin’ and rather than not displaying any
communicative ability at all, uses their plurilingual competence by transliterating the Chinese word 海豚 into theword haitun. It is of course wrong and not an english word, but succeeds in at least making the message complete.
the assessment criteria, mindful of the test’s target candidature and purpose, is not focused on accuracy of
vocabulary but on communicative message, and therefore does not penalise.
In fact, not only is the candidate demonstrating an A2 level of english writing for this context, but also A2
plurilingual abilities if we match the candidate’s attempt to an A2 descriptor found in the CeFR scale for ‘building on
plurilingual repertoire’: ‘Can use a word from another language in his/her plurilingual repertoire to make him/
herself understood in a routine everyday situation, when he/she cannot think of an adequate expression in thelanguage being spoken’ (Council of europe 2020:162).

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 23

Conclusion
the recognition of plurilingual competence and abilities within individual learners is starting to have an effect on
language learning and teaching and should also have an effect on assessment, particularly if an integrated learningand assessment approach is taken, as the CeFR has done. the effect can be felt not only on the individual learner,
but in all worlds that the learner finds themselves in within the ecosystem of learning, as well as at national andsupranational levels. With the learner at the centre, this is important to note for educators and assessors, even fora large assessment body like Cambridge english which may be viewed as ‘monolingual’.
It has been clear that in today’s multilingual world ‘english is necessary, but not sufficient’. National education
bodies, for example, have been keen to invest in english language education and assessment, but not at the expenseof other languages, especially the home languages and languages of schooling at the regional and national levels.
the increase in digital technology is increasingly becoming useful in enabling learners to utilise their plurilingual
repertoire to help them learn english and meet their real-life communicative needs, and Cambridge english has,
and will continue to have, its part to play.

Acknowledgements
special thanks are given to those who have helped in providing input for this article in a short space of time:
Daniel Brooker, Gwendydd Caudwell, Paul Crump, Amy Devine, Alice Gresham, Victoria Laxton, safiatu Lopes,
sian Morgan, John savage, Nick saville and Lynne stevenson.

References
Albrecht, s and Dunlop, M (2018) overview: the 2018 Pre A1 starters, A1 Movers and A2 Flyers revisions, in Cambridge english (ed) The 2018
Pre A1 Starters, A1 Movers and A2 Flyers Revisions, Cambridge: Cambridge Assessment english, 3–6.
ALte (2020a) Principles of Good Practice, available online: www.alte.org
ALte (2020b) The History of ALTE: The Association of Language Testers in Europe – The first 30 years, available online: www.alte.org
Cambridge Assessment english (no date) Should my child learn American or British English? available online: www.cambridgeenglish.org/learningenglish/parents-and-children/information-for-parents/tips-and-advice/should-my-child-learn-american-or-british-english/
Cambridge Assessment english (2018a) Cambridge Assessment English, ALTE and the European Parliament celebrate multilingualism, available
online: www.cambridgeenglish.org/news/view/cambridge-assessment-english-alte-and-the-european-parliament-celebratemultilingualism/
Cambridge Assessment english (2018b) English language transformation services: Case studies, available online:
www.cambridgeenglish.org/Images/472257-case-studies-ministries-education-institutions-employers.pdf
Cambridge Assessment english (2018c) Cambridge English Qualifications. Pre A1 Starters, A1 Movers, A2 Flyers. Handbook for Teachers,
available online: www.cambridgeenglish.org/Images/357180-starters-movers-and-flyers-handbook-for-teachers-2018.pdf
Cambridge Assessment english (2019) Using other languages to learn English, available online: www.youtube.com/watch?v=sHp9Fu-4akoCambridge Assessment english (2020a) National Initiative for English Excellence in the Schools of Saudi Arabia, Cambridge: Cambridge
Assessment english internal report.
Cambridge Assessment english (2020b) Enhancing English language teaching, learning and assessment in state schools in Kazakhstan, Cambridge:
Cambridge Assessment english internal report.
Ćatibušić, B and Little, D (2014) Immigrant Pupils Learn English: A CEFR-related Study of L2 Development, english Profile studies volume 3,
Cambridge: UCLes/Cambridge University Press.

24 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Council of europe (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge: Cambridge
University Press.
Council of europe (2020) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume with
New Descriptors, strasbourg: Council of europe.
Davies, K and Dunlop, M (2018) New writing tasks for young learners: A1 Movers and A2 Flyers, in Cambridge english (ed)The 2018 Pre A1 Starters, A1 Movers and A2 Flyers Revisions, Cambridge: Cambridge Assessment english, 18–24.
european Commission (2012) First European Survey on Language Competences: Final Report, Brussels: european Commission.
european Commission (2015) Study on Comparability of Language Testing in Europe, Brussels: european Commission.
IeLts (2018) Test statistics, available online: www.ielts.org/teaching-and-research/test-statistics
Jones, N and saville, N (2016) Learning Oriented Assessment: A Systemic Approach, studies in Language testing volume 45, Cambridge:
UCLes/Cambridge University Press.
saville, N and Gutierrez eugenio, e (2016) European Strategy on Multilingualism – Policy and Implementation at the EU Level (European Parliament,
Policy Department, Culture and Education Publication no. IP/B/CULT/IC/2016-063), Brussels: european Parliament.
Weir, C J (2005) Language Testing and Validation: An Evidence-based Approach, Basingstoke: Palgrave Macmillan.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 25

Assessing learners’ bilingual skills across different
levels of the CEFR
Elena Anna Maria Gandini UCLan (University of Central Lancashire)

Introduction
the multilingual turn we are witnessing in the academic and theoretical debate on second language acquisition
research and language teaching clearly responds to the needs of today’s multilingual society, where language users
are more and more often required to act as mediators, moving flexibly across languages and cultures, and drawing
upon their entire linguistic and cultural repertoire, partial as it may be in some languages. Language testing andassessment seem to be much slower to follow through in this change of approach, as highlighted by several
prominent voices in the field (among others shohamy 2011, 2013, stathopoulou 2015, Gorter and Cenoz 2017).
Indeed, plurilingualism has always been a key part of the linguistic policy promoted by the european Union (eU),both as a symbol of the eU ideals, as well as a means to facilitate intercultural exchanges and foster a common
identity. For the same reason, mediation had been recognised, together with reception, production and interaction,
as one of the crucial linguistic activities by the Common european Framework of Reference for Languages (CeFR)
already in its first edition, which stated that ‘Mediating language activities … occupy an important place in thenormal linguistic functioning of our societies’ (Council of europe 2001:14). However, the lack of more specificdescriptors articulating how this skill develops throughout the levels identified by the CeFR led to this skill being
often overlooked by practitioners, as it was assimilated alternatively to production and reception, especially in thefield of language assessment. A first step towards the assessment of mediation in its own right came with the
publication of Relating Language Examinations to the Common European Framework of Reference for Languages:
Learning, Teaching, Assessment (CEFR). A Manual (Council of europe 2009), which includes spoken and written
mediation in the communicative activities listed in Form A1 (General examination Description), and more
specifically in Forms A17 (spoken Mediation), A18 (Written Mediation) and A22 (Aspects of Language Competencein Mediation). More recently, the call to look at language assessment from a different perspective, more suited toour multilingual and globalised world, was made even more explicit with the publication of the new Companion
Volume to the CeFR (Council of europe 2018), which includes descriptors for various mediation activities andstrategies, as well as for plurilingual and pluricultural competence.
equipped with these new instruments, it is therefore vital, as recommended by Chalhoub-Deville (2019), that
language assessment broadens its perspectives to consider multilingual constructs and include them, if they
appear relevant for the specific purpose of the examination. this allows exams to better reflect the complexity ofmany contexts, since multilingualism is becoming the norm for many communities across the world, and the‘full (linguistic) humanity’ (schissel, Leung and Chalhoub-Deville 2019:373) of a growing number of people withcomplex linguistic biographies, who grow up to be bi- or plurilingual adults despite being born in an apparent
monolingual reality. A multilingual approach in language testing and assessment could then take multiple forms,
depending on the purpose of the test, the test takers involved and the context the testing is in.
this article will look at an example of bilingual assessment, developed and delivered in the multilingual regionof south tyrol (Italy), and in particular at how translanguaging is incorporated in its reading and speaking tasks,and how well the tasks match the CeFR descriptors for mediation and plurilingual competences. It will also outline
an ongoing follow-up study, which investigates exam strategies and washback through in-depth interviews withexam candidates.

26 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Background and context
south tyrol is a mountainous region situated in the northeast of Italy, on the border with Austria, with a complex
ethnolinguistic makeup, which includes three ethnic groups each with their own official language: German,
Italian and Ladin, spoken in different proportions across different parts of the region. While the Ladin minority(4.53% of the population) and the Italian community (26.06%) are mainly concentrated in specific areas of theregion (respectively Gardena and Badia Valley for the Ladin community, and the larger centres of Bozen/Bolzano
and Meran/Merano, as well as the southern area, bordering the province of trento), the German-speaking
majority is well distributed across the region, and reaches 90–100% of the population in most of the otherareas (Astat 2019).
the history of the region explains the complexity of its ethnolinguistic makeup and the importance given tolanguage, culture and ethnicity issues, which results from a difficult process that led from the promise of an
administrative and cultural autonomy made by King Vittorio emanuele III shortly after the annexation to theKingdom of Italy in 1919 (Von Hartungen 2002) to the implementation of the autonomy statute in 1972 andsuccessive improvements in 2001.
Crucial importance in the autonomy statute is given to the decree regulating several aspects of public life in theregion (e.g. access to public employment, composition of administrative organs and distribution of resources)
according to ethnolinguistic proportion among the different groups (DPR 572/1976). Also, the decree grants equal
status to all three languages in public administration and guarantees to each citizen the right to use their preferredlanguage when approaching institutions. therefore, civil servants and employees of privatised firms offering a public
service (such as the postal service and public transport) are required to reach a certain level of proficiency in Italianand German (or Italian, German and Ladin in the area where the majority of Ladin speakers live). expected levels of
proficiency are linked to specific job descriptions, and are now expressed with reference to the CeFR (whereas in thepast they were linked to the level of education required for a certain job and were expressed according to a different
system of levels). Language proficiency can be demonstrated in different ways, most commonly through an exam ofbilingualism in Italian and German (esami di bilinguismo/Zweisprachigkeitsprüfungen).

History of the exams of bilingualism
since its inception in 1977, the exam of bilingualism has undergone three quite radical reforms, which have altered
its structure completely, reflecting to some extent the major theoretical developments in applied linguistics andlanguage teaching methodology. From January 1977 to December 1998, the exam consisted of two translation
tasks (German into Italian, and Italian into German). Both texts were approximately at the same level and a rather
strong emphasis was placed on the accuracy of the translation, which is consistent with the dominant position ofgrammar-translation methods in language teaching and testing at the time, following closely the methodology
applied to the teaching of classical languages. However, a debate soon started around the use of translation as ameans to assess general language skills (egger 2001), as the influence of the communicative approach progressively
reduced the importance of translation, which started to be considered a language skill per se, not necessarily a good
representation of the general level of proficiency of candidates and/or of their level of ability in other skills. once a
candidate had passed the written exam, there was an oral examination, which could take place no sooner than
30 days after the written exam.
From January 1999 to December 2013, a new exam format was introduced, which began to draw upon the conceptof mediation skills. Instead of a translation, candidates were given two short reading tasks (one in Italian, one inGerman) with six comprehension questions each, to be answered using the ‘other’ language.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 27

Figure 1: Example of Italian reading task with comprehension
questions (to be answered in German)

Figure 2: Example of German reading task with comprehension
questions (to be answered in Italian)

As can be seen in the examples, which were part of a collection of tasks that was made available to candidates toprepare for the exams in what were formerly called levels A and B (with reference respectively to graduate careerand post-high school career paths), the questions covered most of the content of the text, but the need to answer
the questions required some minimal reworking of the sentence structure and the concepts. A relatively strong
grammar focus was still evident, as the rubric specifically asked to answer the questions using one or two full
sentences, which is not so common in real-life communication. Despite the initial attempt at creating a task thatrevolves around the use of mediation and plurilingual skills, which responds better to the specific needs of abilingual region, it seems that the level of authenticity would have been improved by asking the questions in thesame language the candidates had to answer them, which would probably better mimic what might happen in areal-life situation where a bilingual person is required to use their cross-linguistic mediation skills (i.e. by answeringsomeone’s questions about a text written in a language they do not fully understand). except for the length andlevel of complexity of the text, the task seems to match relatively well the descriptors for overall Mediation at
Level B2/C1, as candidates are expected to be able to demonstrate a clear understanding of a text in language A
by conveying ‘detailed information and arguments reliably’ (from the B2 descriptor, Council of europe 2018:105)
in language B, with precision of expression, clarity and fluency being a clear plus that can bring the skill level
required towards C1 (Council of europe 2018:105).
During this phase (1998–2013), a significant increase in the number of exam dates available and a general
streamlining of the procedure made things much easier, with both the written and oral exams being taken on thesame day.
the endeavour towards efficiency continued with the implementation of the current version of the exam,
introduced in January 2014, which was mapped against the CeFR and brought on a rather radical restructure, sothat for the first time since the exam of bilingualism was created candidates are tested on all four skills in bothlanguages. the new features of the exam consist of a brand-new listening section and an expanded reading andwriting section, with a clearer distinction between the two skills, and a language in use task. to facilitate thecomparison with the previous version of the exam, and to better understand the role that the two languages play inthe various sections, examples of tasks that represent the mediation and plurilingual skills construct for Level B2/C1
are included in the next section.

28 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

In this version, the receptive tasks are all based on multiple-choice tasks rather than open questions to facilitateand speed up the marking process of these sections, allowing candidate results to be ready within a very shortturnaround time. the listening and reading/writing sections both take place in the morning, followed by the
speaking section in the afternoon, consisting of a bilingual interview in front of a panel of four examiners.
the current version of the exam is the only one that has been mapped on the CeFR from Level A2 to Level C1.

the plurilingual approach to testing in the current format of the exam
the approach to plurilingualism adopted by this exam matches the view of the CeFR (Council of europe 2018),
which goes beyond the idea of balanced bilingualism that is often referred to or implied when most people talkabout bilingualism, where a speaker is exactly as competent in language A as in language B. on the contrary, theCeFR clearly includes partial and developing competences, offering descriptors across all different levels, even forskills such as mediation and translation. According to this perspective, bilingualism is dynamic, and should rather
be described as a continuum (Figure 3), where speakers can place themselves according to their dominant language
in a given context and a given moment, and move along the continuum depending on how their skills develop.
this applies to the context of south tyrol as well, as recent studies have shown how most speakers have a strongly
dominant language (Vettori and Abel (eds) 2017), and often their level in the language of the other ethnolinguistic
group is much lower than one would normally expect in a bilingual region. Based on this idea of bilingualism as acontinuum, Figure 3 also shows how the level of the weaker language determines the level of the exam that acandidate is able to take, as the goal is to overcome the barrier between languages drawing upon the whole
linguistic and cultural resource of the individual, rather than to show balanced mastery of the two languages(Council of europe 2018:157).

Figure 3: Bilingualism as a continuum with corresponding levels available to candidates for the exam of bilingualism used in South Tyrol,
partially adapted from Valdés (2003), as reported in Baker (2006)

the results of a previous reverse engineering exercise (Gandini expected 2020) on the sample papers available
on the website of the south tyrolean office for Bilingualism (www.provincia.bz.it/formazione-lingue/bilinguismo/l-esame-di-bilinguismo.asp) show how the two languages are activated in the different sections of the exam.
While in the listening section there is minimal contact between the two languages, which are simply tested one
after the other, both the reading/writing (Figures 4 and 5) and the speaking tasks (Figures 6 and 7) follow anintegrated approach where the two languages are activated simultaneously, as a part of a plurilingual system,
where each plays its role and gives a contribution to the achievement of the task.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 29

Figure 4: First part of the reading/writing section of the exam, at Level C1. Reading text in Italian, followed by comprehension
questions, language in use and writing task to be completed in German, available online: www.provincia.bz.it/formazionelingue/bilinguismo/downloads/2_schriftl_Arbeit_A_italienisch.pdf

Figure 5: Second part of the reading/writing section of the exam, at Level C1. Reading text in German, followed by comprehension
questions, language in use and writing task to be completed in Italian, available online: www.provincia.bz.it/formazionelingue/bilinguismo/downloads/2_schriftl_Arbeit_A_deutsch.doc.pdf

30 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

As can be seen in the examples, in the reading section candidates are required first to read a text in language A
and answer reading comprehension questions about it formulated in language B. the section proceeds then with alanguage in use task and a writing task (generally on a topic closely connected to the reading task), which are also
to be completed in language B. At first glance, the current format of the exam at Level C1 gives a much clearer
picture of a candidate’s reading and writing skills than the previous version, used until 1998. Also, the addition of acloze task adds some details about the grammatical competence on a range of topics that differs across the levels.
the fact that the code-switching happens immediately after reading the text seems a step forward in terms ofauthenticity, even though the choice of true/false and multiple-choice items and the formulation of the question
stem as a statement rather than a question are significant limits in this respect.
the structure of the exam is very similar across the levels, with only minimal differences to the type ofmultiple-choice items, as can be seen comparing sample tasks across different levels available online:
www.provinz.bz.it/bildung-sprache/zweisprachigkeit/die-zweisprachigkeitspruefung.asp. However, the changes inlength and complexity of the source text are clearly visible, as are those related to the number of items (especiallyin the language in use task), the text type and the expected word count for the written production.
It is not easy, unfortunately, to match the reading comprehension questions and the language in use task to specificdescriptors in the new Companion Volume of the CeFR (Council of europe 2018), as the type of items chosen(multiple-choice questions and gap-fills) rather limits the dimension of translanguaging included in the readingtasks to the identification of specific details listed in language B (in the stem or the options of the multiple-choicequestions) within a text written in language A, and this case is not contemplated in the new descriptors. However,
the more skilled candidates are able to use concepts and ideas from the reading in language A in the writing task
which needs to be completed in language B, as well as to recycle some vocabulary and expressions used in thequestions. However, as the CeFR clearly states in its first edition (Council of europe 2001), categories and examples
are only included as a suggestion, and exam developers should feel free to approach the classification with acritical eye in view of their specific contexts, create further sub-categories, merge some of the ones that are given,and reject others to better reflect the activities that test takers are being asked to perform.
Much closer is the alignment of the oral exam with the scales of plurilingual competence (Council of europe
2018:157–162). this part of the exam consists of two tasks: a brief presentation and a personal opinion on aloosely connected topic. each task is followed by questions in both languages from the four members of thepanel (two German speakers and two Italian speakers), who then assess candidates on the following criteria inboth languages:ltask achievement and communicative abilitylcoherency and fluencyllinguistic expression and vocabulary rangelgrammatical appropriacy and accuracy.

With regard to the assessment procedure of the speaking exam, it is important to highlight how the responsibilityof assessing both languages falls upon all members of the panel and is not split according to ethnolinguistic groups.
Despite not being explicitly mentioned in the assessment criteria as listed on the website of the Regional office for
Bilingualism, plurilingual competence is certainly one of the main features of this exam, especially at higher levels
such as C1 and B2, where candidates are expected to interact as naturally as possible with their interlocutors,
moving back and forth between Italian and German. A good candidate, therefore, is the one that refers back to what
has been previously discussed (in the other language) and co-constructs the interaction in a way that seems very
relevant for a multilingual region such as south tyrol.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 31

the scale of the Companion Volume that seems most relevant to describe the particular skills targeted by thissection of the exam is the one about ‘building on plurilingual repertoire’ defined as the ‘practical functional ability
to exploit plurilingualism’. the descriptors are especially applicable to high-level candidates, who are expected to‘alternate between languages flexibly to facilitate communication in a multilingual context’, summarise
information, gloss, explain and clarify concepts in both Italian and German, responding ‘spontaneously’ to thechange of language initiated by one of the interlocutors, and ‘catering to the needs and linguistic skills of theinterlocutors’ or more simply to their preferences (Council of europe 2018:162, from the descriptors for Levels C1and B2). Achieving this level of flexibility seems to suggest that what candidates do goes beyond code-switching,defined as ‘the act of switching between two languages’ (Coronel-Molina and samuelson 2017) and moves quite
comfortably into the area of translanguaging, where both languages belong to the same linguistic repertoire that aspeaker can creatively exploit by selecting the language or language features that best allow them to take part
effectively in an interaction.
A brief review of some sample tasks across different levels of the CeFR shows how the structure of the exam
remains the same, as exemplified by Figures 6 and 7. Interestingly, the handouts for the speaking exams arecompletely bilingual, which offers a rich input that candidates can use to their advantage in the short preparation
time they have available.
However, the cognitive complexity, target domain and richness of the input varies significantly, from the graph used
as input for the presentation in task 1 at C1 level (see Figure 6), to the simple personal questions used for Level A2
at the other end of the spectrum (see Figure 7). Considering the difference in level between the dominant language
and the other one, candidates at lower levels are not expected to have the same level of flexibility asked ofcandidates at B2 and C1 levels, but merely to respond to the different interlocutors in the same language they
have used to ask their questions.
Outline of the study in progress
the study is now looking at how test takers prepare for the exam and what strategies they adopt during the exam
itself, to analyse whether the particular construct and the multilingual approach are having an impact on teaching
and learning. Data will be collected through short questionnaires and semi-structured interviews with candidates ofdifferent levels and teachers who deliver language courses in preparation for the exam of bilingualism. Due to the
lockdown of schools and public offices and the limits imposed to international travel by the 2020 coronavirus
pandemic, interviews have been postponed and moved online (to be conducted in Italian or German) toaccommodate the respondent’s preference. Anecdotal evidence collected before the lockdown would seem tosuggest that not all candidates have reflected on the distinguishing features of this particular exam, which mightlead to an approach that works exclusively on the weaker language (e.g. Italian for candidates whose native
language is German, and German for candidates whose native language is Italian), instead of practising extensively
all the tasks involving translanguaging and flexible use of all linguistic resources. However, the lockdown and thecancellation of exams and courses have badly affected the recruitment of research participants, and the project
was put on hold until the end of the summer 2020 (when hopefully exams and courses will resume).

32 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Figure 6: Sample task for the speaking component of the exam,
Level C1, available online: www.provinz.bz.it/bildungsprache/zweisprachigkeit/downloads/Beispiel_1_C1_Muendlich
e_Pruefung.pdf

Figure 7: Sample task for the speaking component of the exam,
Level A2, available online: www.provinz.bz.it/bildungsprache/zweisprachigkeit/downloads/Beispiel_1_A2_Muendlich
e_Pruefung.pdf

Conclusion
the multilingual construct behind this exam seems very relevant to the local situation, valuing pluralism andmultilingualism as assets for south tyrol and its people, potentially realising what Dendrinos (2009, 2013)
describes as a ‘glocal’ model. such a model could potentially attract the interest of bilingual test takers outsideof south tyrol, who want to have their skills in Italian and German certified, and also of other multilingual
communities and exam boards, who want to adopt a new approach including tasks that target mediation anduse of plurilingual resources in communicative situations.

References and further reading
Astat (2019) South Tyrol in Figures, Bozen/Bolzano: Autonomous Province of Bozen/Bolzano – south tyrol, Provincial statistics Institute.
Baker, C (2006) Foundations of Bilingual Education and Bilingualism (Fourth edition), Bristol: Multilingual Matters.
Chalhoub-Deville, M B (2019) Multilingual testing constructs: theoretical foundations, Language Assessment Quarterly 16 (4/5), 472–480.
Coronel-Molina, s M and samuelson, B L (2017) Language contact and translingual literacies, Journal of Multilingual and Multicultural
Development 38 (5), 379–389.
Council of europe (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge: Cambridge
University Press.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 33

Council of europe (2009) Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching,
Assessment (CEFR). A Manual, strasbourg: Council of europe.
Council of europe (2018) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume with
New Descriptors, strasbourg: Council of europe.
Dendrinos, B (2009) Rationale and ideology of the KPG exams, eLt News, september 2009, available online: rcel2.enl.uoa.gr/kpg/gr_kpgcorner_
sep2009.htm
Dendrinos, B (2012) Multi and monolingualism in foreign language education in europe, in stickel, G and Carrier, M (eds) Education in Creating a
Multilingual Europe, Frankfurt: Peter Lang, 47–60.
Dendrinos, B (2013) social meanings in global-glocal language proficiency exams, in tsagari, C, Papadima-sophocleous, s and IoannouGeorgiou, s (eds) Language Testing and Assessment around the Globe: Achievements and Experiences, Frankfurt: Peter Lang, 46–67.
egger, K (2001) Sprachlandschaft im Wandel. Südtirol auf dem Weg zur Mehrsprachigkeit, Bozen/Bolzano: Verlagsanstalt Athesia GmbH.
Gandini, e A M (expected 2020) Code-switching and translanguaging in language assessment in south tyrol, in sciriha, L (ed) Comparative
Studies in Bilingualism and Bilingual Education, Cambridge: Cambridge scholars Publishing.
Gorter, D and Cenoz, J (2017) Language education policy and multilingual assessment, Language and Education 31 (3), 231–248.
schissel, J L, Leung, C and Chalhoub-Deville, M B (2019) the construct of multilingualism in language testing, Language Assessment Quarterly 16(4/5), 373–378.
shohamy, e (2011) Assessing multilingual competencies: adopting construct valid assessment policies, The Modern Language Journal 95 (3),418–429.
shohamy, e (2013) expanding the construct of language testing with regards to language varieties and multilingualism, in tsagari, D, Papadimasophocleous, s and loannou-Georgiou, s (eds) International Experience in Language Testing and Assessment: Selected Papers in Memory of
Pavlos Pavlou, Language testing and evaluation series volume 28, Frankfurt am Main, Germany: Peter Lang, 17–32.
stathopoulou, M (2015) Cross-language Mediation in Foreign Language Teaching and Testing, Bristol: Multilingual Matters.
Valdés, G (2003) Expanding Definitions of Giftedness: The Case of Young Interpreters from Immigrant Communities, Mahwah: Lawrence erlbaum.
Vettori, C and Abel, A (eds) (2017) KOLIPSI II. Gli studenti altoatesini a la seconda lingua: indagine linguistica e psicosociale. Die Südtiroler
SchülerInnen und die Zweitesprache: eine linguistische und sozialpsychologische Untersuchung, Bozen/Bolzano: eurac Research.
Von Hartungen, C H (2002) Kurzgefasste Landesgeschichte Südtirols (1918–2002), Bozen: südtiroler Landtag.

Acknowledgement
Figures 1, 2 and 4 to 7 reproduced with kind permission of servizio esami di Bi-e trilinguismo/Dienststelle für die Zwei-und
Dreisprachigkeitsprüfungen.

34 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

Functional use of multilingualism in assessment:
Opportunities and challenges
Fauve De Backer MULtIPLes, Centre for Diversity and Learning, Ghent University, Belgium
Stef Slembrouck MULtIPLes, Linguistics Department, Ghent University, Belgium
Piet Van Avermaet MULtIPLes, Centre for Diversity and Learning, Ghent University, Belgium

Introduction
since the 1990s we have witnessed an increase in mobility. this mobility – among other factors – impacts
linguistic and socio-cultural diversity in schools and universities. Providing equal opportunities for all learners
and raising their language proficiency is a major challenge for many teachers and administrators in educational
institutions. A number of disconnects can be observed between the plurilingual competences of students andcurrent monolingual (language) testing practices. Is it possible to put diversity to effective use when we addressthese disconnects?
In countries like Belgium, students with an immigrant background score lower on all PIsA tests. Compared to
non-immigrant learners, the gap is especially large. When considering the role of the language(s) spoken at home ina context of assessment, it is clear that not speaking the language of assessment at home is an additional barrier toattaining high scores in standardised tests (oeCD 2019). the stakes for these groups of students are high, as they
are often assigned to inappropriate curriculum tracks, have poorer graduation rates, are disproportionately referredto special education trajectories (Counts, Katsiyannis and Whitford 2018, Vantieghem, Van Avermaet, Groenez andLambert 2018) and run a higher risk of drop-out (Garcia, Kleifgen and Falchi 2008, Groenez, Nicaise and De Rick2009). Because testing plays a major role in processes of transition, one of the ways to address these issues is theremoval of construct-irrelevant barriers in testing. this is essential in any kind of testing, and particularly in ensuring
fairness and validity, because when linguistic complexity unnecessarily interferes with pupils’ ability to demonstrate
their knowledge, this poses a serious validity concern (Wolf et al 2008).

Assessment accommodations
our Flemish research project, Multilingual Assessment in Education (MulAe), highlighted the importance ofconsidering plurilingual competences of students regardless of the subject matter of the test (language or othercontent). the central aim of this project was to explore how multilingual children who are not yet proficient in thelanguage of schooling can be assessed in a fair and valid way in content-related areas. this is one way to begin
addressing the question of how diversity can be an asset in contexts of assessment. We empirically investigated theassessment accommodations for multilingual pupils and the stakeholders’ perceptions about them. A testingaccommodation is defined as ‘any change to standardised testing conditions intended to make the test more fair
and accessible for an individual or subgroup that does not change the construct being measured’ (educational
testing service 2009). there are different possible accommodations, e.g. glossaries, dictionaries, bilingual tests,
extra time, linguistic simplification etc. In the MulAe project, three possible accommodations were explored:bilingual tests (see Figure 1), read-aloud accommodations in the language of schooling, and read alouds in thehome language of the students.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 35

Figure 1: Example of a test item in the bilingual test (based on TIMSS (Trends in International Mathematics and Science Study)-released
science items 2011)

the MulAe project concluded that multilingual assessment has potential to be useful and helpful for many pupils,
but that familiarity with the accommodations and multilingual learners’ heterogeneity are two important factors toconsider (De Backer, Van Avermaet and slembrouck 2017). there is no ‘quick and easy fix’ for the issues concernedwith multilingual assessment. there is a strong need for teachers and test designers to differentiate because
populations of multilingual learners are heterogeneous and, while some accommodations may be more beneficial
for some students, this won’t be the case for others. this makes it difficult to draw generalising conclusions on theassignment of particular accommodations for particular user profiles (De Backer, Baele, slembrouck and Van
Avermaet 2019). the MulAe project also indicates that pupils’ multilingual repertoires can be used as a scaffold,also in assessment, although the results equally urge us to be cautious, because for pupils with lower levels of
proficiency in the L1, exploiting their multilingual repertoire in assessment seems not the best way to make theassessment more fair and valid. For pupils with limited proficiency in their L1 and in the language of schooling,
the accommodations explored in the project were unsuccessful in making the assessment more valid and fairer.
For pupils with low L1 proficiency who made limited use of the read-aloud accommodations in the L1, theaccommodations turned out to be more of a distractor than an aid in demonstrating their competences on thecontent (De Backer, slembrouck and Van Avermaet forthcoming) as illustrated in Figure 2. Possible explanations forthis effect are that in some cases, read alouds might work as a distraction to the task at hand or might heighten
confusion among pupils. It would be especially interesting for future research to explore why this punitive effectappears among students who used the read alouds only in a limited way, while for students who used the read
alouds extensively this effect does not appear. When compared to pupils who did not use the read alouds at all or
used them extensively, perhaps these students are more easily distracted. It is perhaps more difficult for them toremain focused for an extended period of time, or they are more easily overwhelmed by large amounts of information.

Figure 2: Illustration of how differential use of read-aloud accommodations in the pupils’ L1 impacts the effectiveness of theaccommodation (©Peam Burapa)

36 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

For each condition in the research project, pupils were interviewed about their testing experience, including theirdecisions (not) to make use of particular accommodations. Results indicate that pupils in general perceived theaccommodations as helpful. the strategies they used depended on personal preferences, language proficiency
and literacy in both languages, the content attended to and the complexity of the test item (De Backer, Baeleet al 2019).
Implementing assessment accommodations without thoroughly considering the heterogeneity of pupilcharacteristics could in some cases come with a new and added risk of widening the achievement gap even further,making the assessment for some pupils less fair instead of increasing fairness for all test takers. therefore,
accommodations must be implemented and monitored carefully and managed well, and the adoption of one
specific learner profile is a decision not to be taken lightly.
However, although assessment accommodations are one way to differentiate between students, the point of
departure in the majority of the cases continues to be that of a standardised test; and in such a sociolinguistic
universe, languages continue to be considered exclusively as separate and separable constructs. We will consider
both points in what follows.

A transition on two axes
the shaping of a learning environment that provides fair and valid assessments for multilingual learners, in our view,
requires a transition on two axes, as represented in Figure 3.

Testing culture

Large-scaletesting

Test with linguistic
accommodations

Monolingual
assessment

Multilingual
assessment
Functional
multilingual
assessment

Learning-oriented
assessment
Assessment
cultureFigure 3: Functional multilingual assessment: sliding on two axes

Towards an assessment cultureon the vertical axis, we suggest a shift from a testing culture towards an assessment culture (Gipps 1994). Here wenote the paradigm shift that has been developing since the 1990s, implying a movement from an exclusive focus on
psychometric properties of testing towards a broader model of educational assessment (Birenbaum 2016, Dierick
and Dochy 2001, Gipps 1994, Ysenbaert, Van Avermaet and Van Houtte 2017). Different movements haveoperationalised this idea, such as the classroom-based assessment movement in the Us, the Assessment Reform©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 37

Group’s promotion of formative assessment or Assessment for Learning in the UK, and the learning-oriented
approach to assessment developed by Cambridge Assessment english (Cambridge english 2019). the main idea isthat we develop a systemic view in which both the macro level (the frame of educational goals and evaluating
outcomes) and the micro level (individual learning interactions) are taken into account. All levels of assessment cancontribute to better learning and a richer interpretation of learning outcomes (Jones and saville 2016).
the purpose of ‘assessment’ is to provide feedback to students and to make instructional decisions (Ysenbaert et al2017). From the interviews with learners reported in the MulAe project we learned how much pupils are in need ofmore feedback (De Backer 2020). they spontaneously point out their desire to learn from their teacher what they
can do to improve in their work. Hattie and timperley (2007) describe, in their widely cited conceptual analysis offeedback, its potentially powerful impact on the learning achievement of pupils. they define feedback as a‘consequence of performance’ (2007:81). Feedback can be information provided by a parent, a peer, a book,a teacher, a learner or any other kind of information. In their synthesis of over 50 meta-analyses, Hattie, Biggs
and Purdie (1996) report the effect sizes of various influences on the achievement of pupils. the average effect sizeof feedback on the 12 meta-analyses that included feedback in classrooms was 0.79, which was in the top five to
10 highest influences. this indicates the potential power which feedback has on the learning of all pupils. Feedback
is equally a very usable strategy when it comes to the assessment of multilingual pupils. By giving feedback to theselearners when we assess them in a language they are not (yet) familiar with, they are more likely to be working intheir zone of proximal development (ZPD). Vygotsky (1978) proposed the use of ZPD to render a more complete
picture of what a pupil can actually do. It is not unthinkable that when we assess pupils in a language they have notyet sufficiently mastered, we require them to perform outside of their ZPD. When we provide some support orscaffolds for multilingual pupils, it is more likely that they can work inside their ZPD. Providing them with thenecessary information on where they are at in their learning process and what the asset of exploiting theirmultilingual repertoire is, is likely to increase the efficiency of their learning but also their motivation.
Translanguaging
For multilingual learners, however, this shift alone is not enough. there needs to be a shift down a second axis:
from assessing a learner in only the school language towards an assessment where a learner can use all of theirmultilingual repertoire to demonstrate their knowledge and competences. It is argued that assessment
accommodations still consider languages as separate(d) constructs, while critical voices such as shohamy (2006)
and Heller (2007) state today’s language labels and the way the category of a specific language is commonly
understood are by and large an invention of the 19th century and a construct in support of a eurocentric,
geo-political ideal of ‘one people, one state, one culture, one language’ (see also Makoni and Pennycook 2007 and
Cook and Wei 2016). scholars such as Canagarajah (2006) call for a more fluid approach to our understanding of
language use, in which meaning and boundary count as constructions, at times negotiated, and in which hybridity
and lack of clear-cut boundaries are recognised. such a recognition is both an aspect of contemporary(metropolitan, urban, migration-affected) sociolinguistic experiences and also part of how we need to rethink thehistories of recognised languages worldwide (slembrouck and Rosiers 2018). In such an approach, boundaries areoften less strictly defined, and attention must be given to the specific dynamics of non-native language varieties,
as well as established uses in contexts of migration which defy a straightforward categorisation in terms ofrecognised languages. Linguistic fluidity – however understood – has not yet been taken into account in researchon assessment accommodations. It is also in this context that the expanding use and promotion of‘translanguaging’ as a concept and as a model of practice has been gaining currency.
the mobilising appeal of the notion of ‘translanguaging’ has its origins in important sociolinguistic, pragmatic
linguistic and language educational observations about the nature of contemporary (migration-affected,
post-colonial, etc.) experiences of bi/multilingualism. However, how might one understand a concept of38 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020‘translanguaging’ in a context of (multilingual) testing and assessment? While the idea(l) of ‘translanguaging’ hasbeen flourishing almost to the point of becoming a panacea solution, researchers have been much less explicit
about the conceptual and empirical limitations of the concept, especially also whether the necessary critique ofbinarism and linguistic discreteness justifies the abandonment of boundary and category altogether. Unless one
succumbs to the atomism of the individual idiolect, it is hard to conceive of language repertoires in terms devoid ofperceptual categories with boundaries and labels – however volatile and situation-specific experiences may be.
We need to move beyond the two points where the critical exercise has taken us so far, i.e. we have noted the
detrimental effects of a one-sidedly vertical understanding of languages as separated entities in standardised
writing, educational institutions and national language use, and we have stressed the need to recognise thehorizontal dimensions of porousness and fluidity in much language use on the ground. In a context of educational
policy and practice, we need to bring together and ‘mix’ the horizontal and vertical understandings, and dare toopen up the question of functionally relevant multilingual repertoires to the complexities of horizontallyexperienced and vertically defined practices, goals and outcomes (Heugh 2015). By going beyond the binaries,
we must grant children the integrity of their specific multilingual repertoires; we should not deny the legitimacy of,and facilitate access to, ‘Dutch C1’ at the end of one of their educational trajectories. At the same time, it is naïve toassume that practices of naming languages and articulating boundaries is not a characteristic of the horizontalsociolinguistic experience. Nor should we restrict the use of the term ‘translanguaging’ to practices understood as‘mixed’ or ‘hybrid’.
Functional multilingual learning
In the remainder of this article, we argue that functional multilingual learning (FML, sierens and Van Avermaet 2014)– a pedagogical approach in which the multilingual repertoires of students are invoked to increase opportunities forlearning in the classroom – holds promise to be applied in assessment for multilingual learners purposely directed
towards higher proficiency in the language of schooling.
As shohamy (2011:427) stated in her proposal for multilingual assessment, when assessment is ‘used for diagnostic
feedback, and learning purposes beyond the narrow view of testing as it is used in large-scale tests, then any
multilingual pedagogical strategy can be used as an assessment procedure’. shohamy’s advocacy is to allow pupils
to mix languages so that they can express the ideas which they are inhibited to express in one language. this is inline with what sierens and Van Avermaet (2014) propose with their concept of FML. they describe this as what lies
between the concept of raising language awareness on one side of the continuum and multilingual education on theother side. It is a pedagogical approach in which the multilingual repertoires of pupils are actively used to raise andenhance opportunities for learning. Linguistic repertoires form part of the capital that pupils bring to the classroom
and use as a scaffold, as didactic capital for learning. this will not only support learners to master the language ofschooling, but also the content of the instruction.
sierens and Van Avermaet (2014) gave an example of a physics lesson in which the teacher gave some objects togroups of three pupils, along with some short texts in the language of schooling with pictures and drawings oflevers, and a web page with online experiments. the pupils’ assignment was to discover the function of the levers.
Pupils in this class were allowed to speak any language or language variety they wanted. there was a group withthree turkish-speaking pupils and one of the team members did not understand all the words in the Dutch text theteacher gave them, so another group member translated some words for him. the teacher checked regularly to see
if the pupils were making progress in their assignment and gave them feedback in Dutch. When pupil A explained
something to pupil B in turkish and then explained it in Dutch to the teacher, this appeared to strengthen theconcepts the pupils gained in both turkish and in Dutch. this was an example of FML because pupils learned about
levers through different linguistic means and the acquisition of academic language proficiency in both turkish andDutch was being supported.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 39

Functional multilingual assessment
While the concept of FML is being explored more and more in recent research (e.g. Ramaut et al 2013, Rosiers 2016,
slembrouck and Rosiers 2018, Van Houtte et al 2016) as a pedagogical strategy, the link with assessment has notbeen made explicit. to address this challenge, let us take the above-discussed group work. suppose that the teacher
wants to assess this group work. there are many possible ways in which such an assessment could be given form.
Here we present one example as a hypothetical sequel to the case described above. When the teacher gives the
instruction for the assignment, they could also tell the pupils that they will be presenting their work to each otherafterwards and that they will be evaluating themselves and their peers. Before starting, the teacher and the pupils
could discuss what would be good criteria on which to base their evaluation of both the group work and theirpresentation. For example, the teachers and the pupils could come to an agreement on five criteria they findimportant, two with respect to working in a group, two with presentation skills, and one with correct content.
After doing their group work as described above, the groups could present their work in front of the class.
suppose that in the presentation, one of the turkish pupils cannot find the right word in Dutch but gets some help
from another pupil. the pupil could then demonstrate to his classmates that he gained knowledge through theassignment on the subject – he knows the working of a lever – but just could not exploit all of the words in thelanguage of schooling. this helps the pupils who are not yet as proficient in the language of schooling to step
forward and dare to speak in front of the class. In this proposed ‘experiment’, enough space is to be left on theevaluation sheet for written feedback. the pupils could be allowed to use any language for this. this would ofcourse mean that the teacher gives the pupils confidence to take their responsibility of giving feedback seriously.

Conclusion
When we see learning and assessment as inseparable, the idea of learning-oriented assessment can beconnected seamlessly to the concept of FML. so, when we shift on both axes – as shown in Figure 3 – we arrive
at functional multilingual assessment. It was argued in this article that functional multilingual assessment
holds potential to overcome some of the critical issues concerning the assessment of multilingual learners.
Research on the assessment of multilingual learners has been focusing mainly on standardised tests with(linguistic) accommodations. the idea of functional multilingual assessment will be further explored in future
research projects.
In order to be able to do so, however, we will need to broaden our scope of teacher assessment literacy.
Assessment is an inherent part of the educational process and a powerful tool for both teachers and pupils to get
more insight into the competences of the students. However, teachers are not always sufficiently aware of thepotential strength of assessment. Moreover, the literacy of the teachers to make valid claims about pupils’competences is threatened when teachers are not sufficiently aware of the potential benefits and pitfalls ofaccommodations. In the explorative research in this project, it appeared that teachers do recognise the language
barrier that multilingual students face but the idea is prevalent that it is not fair to attend to this unless pupils
have a ‘label’. Moreover, the prevailing idea is that differentiation is worth exploring in instruction, but not in
assessment (De Backer, De Cooman, slembrouck and Van Avermaet 2019). We need to address the concerns thatteachers have about feasibility, fairness, comparability of test results and the importance of not lowering the
educational level. We need to work on awareness-raising and enhance self-efficacy by validating research inpractice in order to increase teacher assessment literacy.

40 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

References
Birenbaum, M (2016) Assessment culture versus testing culture: the impact on assessment for learning, in Laveault, D and Allal, L (eds)
Assessment for Learning: Meeting the Challenge of Implementation, Dordrecht: springer, 285–302.
Cambridge english (2019) Learning Oriented Assessment, available online: www.cambridgeenglish.org/research-and-validation/fitness-forpurpose/loa/
Canagarajah, s (2006) Changing communicative needs, revised assessment objectives: testing english as an international language, Language
Assessment Quarterly 3 (3), 229–242.
Cook, V and Wei, L (2016) The Cambridge Handbook of Linguistic Multi-competence, Cambridge: Cambridge University Press.
Counts, J, Katsiyannis, A and Whitford, D K (2018) Culturally and linguistically diverse learners in special education: english learners, NASSP
Bulletin 102 (1), 5–21.
De Backer, F (2020) Bridging the gap between learning and evaluation: Lessons learnt from multilingual pupils, Journal of Applied Linguistics andProfessional Practice 14 (1), 96–118.
De Backer, F, slembrouck, s and Van Avermaet, P (forthcoming) Effectiveness of Multilingual Assessment of Science Competences.
De Backer, F, Van Avermaet, P and slembrouck, s (2017) schools as laboratories for exploring multilingual assessment policies and practices,
Language and Education 31 (3), 217–230.
De Backer, F, Baele, J, slembrouck, s and Van Avermaet, P (2019) Pupils’ perceptions on accommodations in multilingual assessment of science,
Language Assessment Quarterly 16 (4/5), 426–446.
De Backer, F, De Cooman, e, slembrouck, s and Van Avermaet, P (2019) Multilingual assessment: beliefs and practices of geography teachers,in Kirsch, C and Duarte, J (eds) Multilingual Approaches for Teaching and Learning: From Acknowledging to Capitalising on Multilingualism inEuropean Mainstream Education, London: Routledge.
Dierick, s and Dochy, F (2001) New lines in edumetrics: New forms of assessment lead to new assessment criteria, Studies in Educational
Evaluation 27 (4), 307–329.
educational testing service (2009) Guidelines for the Assessment of English Language Learners, available online: www.ets.org/s/about/pdf/
ell_guidelines.pdf
Garcia, o, Kleifgen, J A and Falchi, L (2008) From English Language Learners to Emergent Bilinguals, equity Matters Research Review No. 1,
available online: files.eric.ed.gov/fulltext/eD524002.pdf
Gipps, C (1994) Beyond Testing: Towards a Theory of Educational Assessment, London: Falmer Press.
Groenez, s, Nicaise, I and De Rick, K (2009) De ongelijke weg door het onderwijs, in Vanderleyden, L, Callens, M and Noppe, J (eds) De sociale
staat van Vlaanderen, sint-Niklaas: Drukkerij Room, 33–68.
Hattie, J and timperley, H (2007) the power of feedback, Review of Educational Research 77 (1), 81–112.
Hattie, J, Biggs, J and Purdie, N (1996) effects of learning skills interventions on student learning: A meta-analysis, Review of Educational Research
66 (2), 99–136.
Heller, M (2007) Bilingualism: A Social Approach, Houndsmills/Basingstoke/Hampshire/New York: Palgrave Macmillan.
Heugh, K (2015) epistemologies in multilingual education: translanguaging and genre – companions in conversation with policy and practice,
Language and Education 29 (3), 280–285.
Jones, N and saville, N (2016) Learning Oriented Assessment. A Systemic Approach, studies in Language testing volume 45, Cambridge/UCLes:
Cambridge University Press.
Makoni, s and Pennycook, A (2007) Disinventing and Reconstituting Languages, Clevedon: Multilingual Matters.
oeCD (2019) PISA 2018 Results (Volume II): Where All Students Can Succeed, available online: www.oecd-ilibrary.org/docserver/b5fd1b8fen.pdf?expires=1594202593&id=id&accname=guest&checksum=501eD959BF3F5D43F3CD42FC9662e748
Ramaut, G, sierens, s, Bultynck, K, Van Avermaet, P, Van Gorp, K, slembrouck, s and Verhelst, M (2013) Evaluatieonderzoek van het project'Thuistaal in onderwijs' (2009–2012): Eindrapport (onuitgegeven onderzoeksrapport), available online:
biblio.ugent.be/publication/4301917/file/4302025
Rosiers, K (2016) Translanguaging, Troef of Trend? Een Analyse van Meertalige Interactie in een Gentse en Brusselse Klas, unpublished PhD thesis,
Ghent University.
shohamy, e (2006) Language Policy: Hidden Agendas and New Approaches, New York: Routledge.
shohamy, e (2011) Assessing multilingual competencies: Adopting construct valid assessment policies, Modern Language Journal 95 (3),418–429.
sierens, s and Van Avermaet, P (2014) Language diversity in education: evolving from multilingual education to functional multilingual learning,
Managing Diversity in Education: Languages, Policies, Pedagogies 33, 204–222.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 41

slembrouck, s and Rosiers, K (2018) translanguaging: a matter of sociolinguistics, pedagogics and interaction?, in Van Avermaet, P, slembrouck,s, Van Gorp, K, sierens, s and Maryns, K (eds) Multilingual Edge of Education, London: Palgrave, 165–187.
Van Houtte, M, slembrouck, s, Lambert, P, Van Avermaet, P, van Braak, J, Van De Craen, P and Verhelst, M (2016) SBO Project Application‘Validitv’. Valorising Linguistic Diversity in Multiple Contexts of Primary Education, available online: researchportal.be/en/project/valorisinglinguistic-diversity-multiple-contexts-primary-education
Vantieghem, W, Van Avermaet, P, Groenez, s and Lambert, M (2018) Diversiteitsbarometer Onderwijs, available online: www.unia.be/files/
Documenten/Publicaties_docs/1210_UNIA_Barometer_2017_-_NL_As.pdf
Vygotsky, L (1978) Mind in Society, Cambridge: Cambridge University Press.
Wolf, M K, Herman, J L, Kim, J, Abedi, J, Leon, s, Griffin, N, Bachman, P L, Chang, s M, Farnsworth, t, Jung, H, Nollner, J and shin, H-W (2008)
Providing Validity Evidence to Improve the Assessment of English Language Learners, CResst Report 738, Los Angeles: CResst.
Ysenbaert, J, Van Avermaet, P and Van Houtte, M (2017) Literatuurstudie Evaluatie en diversiteit, Ghent: steunpunt onderwijs onderzoek,
available online: steunpuntsono.be/wp-content/uploads/2018/02/soNo_2017.oL1_.6_2.pdf

42 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

The challenge of assessing plurilingual repertoires:
The EVAL-IC project
Elisa Fiorenza University of Roma tre
Elena Diego-Hernández University of salamanca

Introduction
the promotion of multilingualism has long been a priority of european language policy (see Council of europe2018). However, plurilingualism is making slow inroads across various levels of the educational system and it isalmost completely neglected in the field of assessment.
An innovative solution for plurilingual assessment has been developed in the area of intercomprehension (IC),a pluralist approach that promotes plurilingual and multicultural education as it adopts an integrated language
teaching model, where different languages are considered simultaneously (Candelier et al 2007). the peculiarity of
the IC is linked to the mode of plurilingual communication: those who participate in the communicative event
understand the language of others and express themselves in the language they know. this is a common and often
spontaneous practice for millions of speakers, especially of related languages.1
this practice can be seen as a key to plurilingualism (escudé and Janin 2010:28); however, although IC to date is
infrequently found in schools, it has gathered some institutional recognition in universities (see Bonvino and Jamet(eds) 2016). With regard to the latter, numerous IC courses have been held, for example, at the Language Centre of
the University of Roma tre (Italy), which have seen the participation of hundreds of students from different degree
courses. the University of salamanca (spain) also offers IC courses in two degree courses of the Faculty of Philology,
recording great success among erasmus students too.
In this article, we reflect on the viability of plurilingual assessment. We will start with a brief introduction on thenotion of plurilingual competence and then outline the general guidelines for a plurilingual assessment.
Furthermore, the Évaluation des compétences en intercompréhension (eVAL-IC) project and its innovative testfor plurilingual assessment of IC are presented. Lastly, we will analyse the comments and reflections of a sample
of students who participated in the eVAL-IC test, showing the adherence of this test to the principles of learningoriented assessment (Jones and saville 2016).

Defining and assessing plurilingual competence: the eVAL-IC project
the new Common european Framework of Reference for Languages (CeFR) Companion Volume (Council of europe
2018:28) defines the competence of the plurilingual speaker as ‘the ability to call flexibly upon an inter-related,
uneven, plurilinguistic repertoire’ to accomplish tasks, such as express oneself in one language and understand aperson speaking a different one. From a similar perspective, Coste, Moore and Zarate (2009:5) describe plurilingualand pluricultural competence as a global and multiple competence made up of linguistic, cultural and identity
aspects: they underline the potential links among languages, which cannot be considered separate entities.
1. However, communication is possible also between speakers of languages of different families by using a bridge language, that is a foreign language known to the
speaker and belonging to another family of languages (Meissner, Meissner, Klein and stegman 2004).

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 43

In order to describe plurilingual and pluricultural competence, a series of documents with different objectives
and points of view have been elaborated: the CARAP (Candelier et al 2007)2; the MAGICC project3; the ReFIC4;
the eVAL-IC project (discussed below)5; and the CeFR Companion Volume (Council of europe 2018). these
documents allow the identification of reference levels and descriptors which are the basis of the assessment.
the object of the plurilingual evaluation is a series of ‘know-hows’ that involve the activation of a plurilingual
repertoire, allowing the language user to draw on ‘available knowledge of all kinds from previous language learningin order to understand texts in genetically related languages (intercomprehension, e. g. between Romance, slavic,
Germanic languages)’ (Lenz and Berthele 2010:303). Despite the interest in these skills in our increasingly
multilingual society, many of these do not fall within a traditional framework of assessable skills. the innovative
eVAL-IC project shows that a plurilingual assessment is possible.

the eVAL-IC project and its plurilingual test
the eVAL-IC project was coordinated by ollivier and co-coordinated by Bonvino and Capucho 6. It proposes a
system of assessment of plurilingual competences in IC, is aimed at an audience of university students, and intends
to provide them with a certificate in IC. Within the project, an assessment instrument has been developed andtested with the aim of formally recognising the communicative competences acquired (not necessarily after acourse) by the individuals, in relation to the IC among the five Romance languages of the project (Portuguese,spanish, Italian, French and Romanian)7.
starting from the description and the levels of IC competences, an assessment protocol has been elaborated.
this protocol defines the relationships between the dimensions of plurilingual and pluricultural communication(linguistic, paraverbal and non-verbal, intercultural and pragmatic), the assessment criteria (availability, adequacy,
extent), the observable indicators and the levels of IC. It resulted in the elaboration of an assessment scenario andof evaluation grids. the analysis of the scientific literature8 allowed the authors to distinguish and define three
IC competences and six levels of IC:a) Receptive IC9, which is the competence of comprehension (written and spoken) in several languages. It isessentially based on a transfer of knowledge and skills between languages.
b) Interactive IC10, defined as a communicative process in which speakers express themselves in the language they
prefer and understand the language of their interlocutors.
c) Interproduction11, defined as a communicative process which allows speakers to be understood (in the language
they prefer) by interlocutors who do not know or have little knowledge of the respective language. the speakers
try to reduce possible problems of comprehension by adapting spoken production to the repertoires in contact.
this distinction is aimed at identifying observable competences and should not suggest a fragmented conceptionof competence in IC. on the contrary, in the eVAL-IC project, this competence is considered an integrated,
2. carap.ecml.at
3. unil.ch/magicc/home.html
4. miriadi.net/en/refic
5. www.evalic.eu
6. Christian ollivier (Université de La Réunion), elisabetta Bonvino (Università Roma tre) and Filomena Capucho (Universidade Católica Portuguesa).
7. All products created within the framework of the eVAL-IC project are transferable to other language families.
8. see ollivier, Capucho and Araújo (2019) and Bonvino, Cortés Velásquez and Fiorenza (2018) for further details.
9. see CARAP, MAGICC and ReFIC.
10. see Capucho (2011), Degache (2009) and Araújo, de Carlo and Melo-Pfeifer (2014).
11. see Balboni (2009) and Capucho (2011).

44 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

global, plurilingual and communicative construct (ollivier et al 2019). the various languages taken into
consideration are not evaluated in isolation – rather, the aim is to provide an overall assessment of plurilingual
repertoire management.
the test created by the eVAL-IC team consists of seven tasks (written and oral) 12 that require the activation of
a plurilingual and pluricultural repertoire. the proposed activities are independent from each other, althoughlinked by a scenario-based macro-situation (see Purpura 2016) allowing participants to experience real-life
communicative situations in which they are required to deal with various Romance languages. the scenario
requires the candidates to actively participate in an international conference on sustainable development,
involving some european universities and held in five Romance languages, using the language of their choice(usually their L1). Candidates are evaluated according to the levels, indicators, criteria and evaluation grids
produced within the project.13

Participant reactions to the test
A short questionnaire was designed to ascertain the participants’ evaluation of the test. It was completed by
58 students from the University of Roma tre, all Italian speakers, on paper or on online without time limits.
In the following analysis we will consider the answers given by the students with regard to three items:
1. ‘Briefly explain how you found the eVAL-IC test experience.’
2. ‘Indicate with a score from 0 (none) to 5 (very much) your understanding of the single videos. Write a shortcomment below.’14
3. ‘Indicate with a score from 0 (very easy) to 5 (very difficult) how difficult you found activity 3A 15. Write a shortcomment below.’Firstly, the analysis aims to show the effects of the eVAL-IC test in terms of reflection on (a) the plurilingual test,(b) the aural task and the processes/strategies involved, (c) the languages of the participants’ plurilingual repertoire(including the target languages), and (d) the IC approach. secondly, the analysis aims also to highlight the learningorientation of this test, in line with the three principles of learning-oriented assessment (Carless 2007:59):
1. Assessment tasks as learning tasks.
2. student involvement in assessment as self-evaluators.
3. Feedback as feedforward16.
A qualitative approach was adopted to capture the complexities of the phenomenon. Below is a summary and somesignificant extracts from the students’ comments.
12. For a detailed description of the tasks, visit: evalic.eu/productions/produits.
13. For example, for written IC, the highest level refers to individuals who can: ‘understand the global and detailed sense of a great diversity of types of written text,
varying according to the communicative objective; leverage their personal resources and the potential of the IC to overcome difficulties in understanding;establish interlinguistic and intercultural links in a systematic and appropriate way; mobilise their in-depth knowledge of several (related) languages’(our translation of the original Italian version, available at evalic.eu/productions/produits).
14. this refers to the listening task 3A, where candidates were asked to collect information by taking notes from a series of authentic videos in the five languages ofthe project.
15. see previous note.
16. As stated in Carless (2007:59): ‘for assessment to promote learning, students need to receive appropriate feedback which they can use to ‘feedforward’ into
future work. Feedback in itself may not promote learning, unless students engage with it and act upon it’.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 45

Comments about the plurilingual test
the overall response to the test was very positive. Although tiring for some, taking the test was highly appreciatedand recommended by all participants, mainly because it was a challenge to prove their language proficiency
for themselves.
the scenario-based approach was successful because it was anchored to the students’ reality for various reasons:first of all, it allowed students to enrich their knowledge on the current topic of sustainable development (seeexample 1a); many students considered the test useful for their career as students of languages and/or of didactics
of modern languages, stating their interest for the dynamics of understanding and learning foreign languages (FLs)in a context of multilingualism (see example 1b). Finally, some students appreciated the expendability of acertificate in a professional context. (Please note all comments from students are unedited to maintainauthenticity.)
example 1a: ‘the experience was very interesting ... as it allowed me to test my language
comprehension skills through a series of exercises on a contemporary and important topic’.
example 1b: ‘I found [it] useful and inherent to our course of studies (Languages and CulturalLinguistic Mediation)’.
Comments about oral IC processes and strategies
the results shown in Figure 1 indicate that more than half of the respondents (N = 39) found the test medium todifficult (values 3 and 4).
30

Number of respondents

25
25
20

14

15

11

10
5

4

3

0
1

2

3

4

5

Scale of diffculty
Figure 1: Level of perceived difficulty in completing aural task 3A (1 ‘very easy’ to 5 ‘very difficult’)

A common view amongst candidates was that listening comprehension requires a lot of concentration, even in
one’s own L1 (see example 2a), and that it is more difficult if there is a complete lack of visual aids and previous
knowledge on the subject, and if one cannot manage audio playback (for example, stopping it). Portuguese wasoften perceived as the most opaque (see example 2b).
example 2a: ‘I had to concentrate so that I did not make the mistake of underestimating theimportance of listening ... especially in Italian, which is my mother tongue’.
example 2b: ‘For Portuguese I found a higher level of difficulty than I expected knowing spanish,
but in the end, after playing the video several times, I understood the necessary information’.

46 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

the theme of the importance of being more aware of comprehension processes and strategies recurred throughout
the dataset. students reflected on the cognitive and metacognitive strategies they used to understand unknown
languages, highlighting in particular that they often used inference, context information and transfer(see example 2c). A further and very encouraging element is the fact that many students perceived a greaterawareness of IC strategies (see example 2d), also confirming a gradual automation of sound recognition
processes through exposure (see example 2e).
example 2c: ‘I could see the effectiveness of some strategies applied in a conscious way(text segmentation, expectancy grammar)’.
example 2d: ‘[this test] not only enriches our knowledge but also makes us more aware of the skills
and gaps we have’.
example 2e: ‘After the initial difficulties in understanding (especially for the Portuguese andRomanian videos) I started to perceive more and more information, almost as if my ear was gettingused to them’.
Comments about languages
Five broad themes emerged from the analysis. Firstly, the languages perceived as less distant from Italian are
spanish, followed by French, Portuguese and Romanian (see Figure 2). the second fact that emerged is the positive
aspect of understanding something of new languages, such as Romanian and Portuguese (see example 3a). this isalso connected to an opening towards what is ‘foreign’ (languages, cultures). thirdly, many students have expressed
the desire to deepen their understanding of some languages, in particular Portuguese. the fourth theme concerns
the support deriving from knowledge of other languages, including non-Romance languages (see example 3b).
Finally, participants expressed their awareness of the centrality of vocabulary and listening skills, often associatedwith the desire to deepen the latter.
example 3a: ‘the test was a great opportunity to come into contact with new languages(such as Portuguese and Romanian)’.

Number of respondents

example 3b: ‘My language skills in english and German have been of great help’.

70
60
50

5
4

40
30
20
10
0

3
2
1
PT

ES

FR

RO

Figure 2: Self-assessed comprehension levels of the languages during activity 3A by the Italian candidates (N=57)(PT: Portuguese; ES: Spanish; FR: French; RO: Romanian; 1 ‘none’ to 5 ‘very much’)©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 47

Comments about IC
A variety of perspectives were expressed in relation to the experience of IC. the most recurrent theme was theenthusiasm for having personally participated in a real-life experience that showed them the similarity between
languages in a direct way (see example 4a). Participants were positively surprised to have discovered the richness of
their plurilingual and strategic repertoires, and to have been able to do what was required by reconstructing the
global meaning of the texts even from a partial understanding (see example 4b). Many students expressed their
willingness to deepen the dynamics of IC, highlighting the importance of its spread also outside language classes(see example 4c).
example 4a: ‘I found the activity interesting and ... enlightening as we were given the opportunity toobserve and experience the intercomprehension by actively participating in it’.
example 4b: ‘It was surprising to be confronted with languages never studied before but still be able
to understand the essential notions’.
example 4c: ‘Future teaching of multilingualism ... could be useful not only for language learners but
also for all those who have to work in areas where intercomprehension is strictly necessary, such as
medical, administrative and judicial fields’.

Conclusions
the development and implementation of a plurilingual test is a real innovation in the field of language teaching.
the test proposed in the eVAL-IC project allows the assessment of plurilingual competences through complex
communicative sub-tasks, while working on several languages simultaneously.
the data presented confirm that this type of test is a useful assessment instrument not only for those involvedin language assessment in multilingual contexts, but also for students taking part in the test. In this regard,
the eVAL-IC test emphasises the learning aspects of assessment, embodying the learning-oriented assessment
principles already mentioned above (Carless 2007, Jones and saville 2016) in these terms:
1. Assessment tasks as learning tasks
thanks to the scenario-based approach of the test, participants simulated a real-life situation in which theygathered and organised information from several languages of which they had various degrees of expertise, taking
advantage of the richness of their plurilingual repertoire. We believe that the test ‘mirror[s] real-world applicationsof the subject matter’ (Carless 2007:59) for two reasons that emerged in the students’ comments: first of all,it represents a possible situation for participants, so the learning aspect concerns acquiring information onsustainable development from multiple language sources, as can happen in real life. secondly, most of theparticipants studied foreign language (FL) and/or FL teaching and therefore had an interest in this type of FL-related
activity. For all of them, the test was also a tool to see how much they understood about languages never studied
before and to reflect on their comprehension processes and strategies to ‘set in motion’ and possibly expand theirplurilingual repertoire. In this regard, it is important to note that the test was carried out mainly with students of
Romance languages and studying FL and/or FL teaching17; therefore further research is needed for other language
groups and disciplines.
17. outside Italy, the test has also been tested with students from other faculties, in various locations across europe (for further information, seeevalic.eu/productions/publications).

48 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

2. Student involvement in assessment as self-evaluators
the eVAL-IC test stimulated students’ self-reflection and self-assessment skills on various aspects: their experienceof the plurilingual test, their IC skills in terms of repertoire composition (partial language competences, etc.), theircomprehension and learning processes and strategies (and their evolution during the test), and the characteristicsof the languages (not only the target languages) and their perceived distance.
In this regard, we believe that an IC assessment would also be useful for speakers of non-Romance languages,
especially students with english in their repertoire (as L1 or L2/L3 etc.), for several reasons: first of all because they
could realise the enormous lexical and morphosyntactic closeness of english with French and use english as a bridge
language for a Romance language (Robert 2011:242). Moreover, this test stimulates reflection ‘beyond’ language
families, leading participants to consider not only the specific characteristics of languages (already known or target
languages), but also the functioning of languages as a system. Ultimately, it should be noted that the test structure
itself could be adapted with materials of other language families (Germanic languages, etc.).
3. Feedback as feedforward
students’ self-assessments and the assessments that students will receive from teachers (in progress) are usefulelements to promote work on the development of plurilingual skills (e.g. motivation towards IC practices;willingness to deepen the study of FL) and discussion on FL learning/teaching in multilingual contexts.

References
Araújo e sá, M H, de Carlo, M and Melo-Pfeifer, s (2014) Acteurs et dynamiques de médiation dans une plateforme de formation àl’intercompréhension, Canadian Modern Language Review 70 (2), 133–157.
Balboni, P (2009) Per una glottodidattica dell’intercomunicazione romanza, in Jamet, M-C (ed) Orale e intercomprensione tra lingue romanze,
Venezia: Le Bricole, 197–203.
Bonvino, e and Jamet, M-C (eds) (2016) Intercomprensione: lingue, processi e percorsi, Venezia: edizioni Ca’ Foscari.
Bonvino, e, Cortés Velásquez, D and Fiorenza, e (2018) observing strategies in intercomprehension reading: some clues for assessment inplurilingual settings, Frontiers in Communication 3 (29), 1–15.
Candelier, M, Camilleri-Grima, A, Castellotti, V, de Pietro, J F, Lörinez, I, Meissner, F-J, schröder-sura, A and Noguerol, A (2007) CARAP – Cadre deRéférence pour les Approches Plurielles des Langues et des Cultures, Graz: CeLV-Council of europe.
Capucho, F (2011) IC fra lingue appartenenti a diverse famiglie linguistiche, in De Carlo, M (ed) Intercomprensione e educazione al plurilingüismo,
Porto s. elpidio: Wizarts, 223–241.
Carless, D (2007) Learning-oriented assessment: conceptual bases and practical implications, Innovations in Education and Teaching International
44 (1), 57–66.
Coste, D, Moore, D and Zarate G (2009) Plurilingual and Pluricultural Competence, strasbourg: Council of europe.
Council of europe (2018) The Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Companion Volume with
New Descriptors, strasbourg: Council of europe.
Degache, C (2009) Nouvelles perspectives pour l’intercompréhension (Afrique de l’ouest et Caraïbe) et évolutions du concept, in Araújo e sá,
M H, Hidalgo Downing, R, Melo-Pfeifer, s, séré, A and Vela Delfa, C (eds) A intercompreensão em línguas românicas: conceitos, práticas,
formação, Aveiro: Universidade de Aveiro, 81–102.
escudé, P and Janin, P (2010) L’intercompréhension, clé du plurilinguisme, Paris: CLe International.
Jones, N and saville, N (2016) Learning Oriented Assessment: A Systemic Approach, studies in Language testing volume 45, Cambridge:
UCLes/Cambridge University Press.
Lenz, P and Berthele, R (2010) Prise en compte des compétences plurilingue et interculturelle dans l’évaluation, strasbourg: Council of europe.
Meissner, F J, Meissner, C, Klein, H and stegman, t (2004) EuroComRom – Les septs tamis: lire les langues romanes dès le départ, Aachen: shaker
Verlag.
ollivier, C, Capucho, F and Araújo e sá, H (2019) Defining IC competencies as prerequisites for their assessment, Journal of Applied
Psycholinguistics 19 (2), 3–42.

©UCLES 2020

CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78| 49

Purpura, J (2016) second and foreign language assessment, The Modern Language Journal 100 (1), 190–208.
Robert, J-M (2011) L’inglese: un ponte verso le lingue romanze, in De Carlo, M (ed) Intercomprensione e educazione al plurilingüismo, Porto s.
elpidio: Wizarts, 242–251.

50 |CAMBRIDGe AssessMeNt eNGLIsH – ReseARCH Notes: 78©UCLES 2020

We help people
learn English andprove their skills
to the world
Discover more:cambridgeenglish.org/researchnotes

We are Cambridge Assessment English. Part of the
University of Cambridge, we help millions of people
learn English and prove their skills to the world.
For us, learning English is more than just exams
and grades. It’s about having the confidence tocommunicate and access a lifetime of enrichingexperiences and opportunities.
With the right support, learning a language is an
exhilarating journey. We’re with you every step of the way.

cambridgeenglish.org
Cambridge Assessment English
The Triangle Building
Shaftesbury Road
Cambridge
CB2 8EA
United Kingdom

All details are correct at the time of going to print in September 2020.

/cambridgeenglish/cambridgeenglishtv/cambridgeeng/cambridgeenglish

Copyright©UCLES 2020