Table of Contents

I...... THE KEY COMPETENCES FOR EUROPEAN CITIZENS IN THE KNOWLEDGE SOCIETY 6

1. We have a clear definition of the key competences needed for the 21st century. 6

2. The main challenge now is to assess all key competences. 7

3. Yet too little is done on assessment 9

II..... STRENGHTHENING ASSESSMENT OF ALL KEY COMPETENCES: LESSONS FROM MEMBERS STATES. 11

1. Defining key competences as tangible learning outcomes. 12

1.1 Mother tongue, Foreign languages, Mathematics and Science. 13

1.2 ‘Learning to learn’ 13

1.3 Social and civic competence and creativity. 16

1.4 Digital competence. 17

1.5 Sense of initiative and entrepreneurship. 18

1.6 Cultural awareness and expression. 21

2. Using assessment to measure learning outcomes. 22

2.1 Standardised tests. 23

2.2 Attitudinal questionnaires. 25

2.3 Performance-based assessments. 26

3. Using assessment to encourage the development of key competences. 28

3.1 Peer and self-assessment 31

3.2 Portfolio and e-portfolio assessment 32

3.3 E-assessment 34

4. Mainstreaming assessment of key competences. 39

4.1 Curriculum and consultation. 39

4.2 Textbooks and resources. 41

4.3 Assessment guidance. 42

4.4 Monitoring and evaluation. 43

4.5 Training and development 44

4.6 Professional networks. 47

4.7 School organisation. 48

5. Conclusion: towards a coherent implementation strategy. 51

EXECUTIVE SUMMARY

The “Rethinking Education” Communication aims to help Member States by setting out a range of fields in which the efficiency of education and training systems can be improved.

This Staff Working Document highlights the evidence base for the policy proposals outlined in the “Rethinking Education” Communication on key competences. Its findings are based on analyses, coordinated by the Commission in cooperation with Member States, of best policy practice across the EU and beyond, and on comparative data and current research.

Global competition for skills, technological advances, the impact of the internet and new media on employment, learning and private lives, the growing diversity of our societies – all are forces which are reshaping our education systems and changing the content of and approaches to teaching and learning. To face the current crisis, increase productivity and encourage growth, education systems have a crucial role in building the right 21st century skills and competences.

Already in 2006, The European Union adopted a European Reference Framework of Key Competences for Lifelong Learning to identify the key competences necessary for personal fulfilment, active citizenship, social cohesion and employability in a knowledge society. Its major innovation is to move from a static conception of curricular content to a dynamic combination of knowledge, skills and attitudes appropriate to the many and varied real-life contexts on which people need to use them. It includes not only the key competences that relate to traditional school subjects but also those more cross-curricular (or 'transversal') key competences that enable people to pursue learning throughout their lives, contribute to democratic societies and to succeed in today's and tomorrow's world of work with its demand for high skills combined with creativity and the ability to innovate.

Many Member States have reformed their school curricula and vocational programmes in recent years, to introduce part or all of the approach set out in the Key Competences Recommendation. This involves a major paradigm change from transmitting a static body of pre-defined knowledge to a more dynamic and holistic development of competences.

Yet, the key challenge for education systems in many Member States is the assessment of these competences. Assessment is one of the most powerful influences on teaching and learning but it tends to put too much emphasis on subject knowledge, and less on skills and attitudes, and to neglect altogether the increasingly important cross-curricular competences such as learning to learn or entrepreneurship.

Progress have to be made on assessment approaches to take into account all competences needed for the 21st century. Many promising developments in Member States can be used as best practice examples to support efforts to improve assessment in European countries. These include broadening the scope of assessment of traditional school subjects (mother tongue, foreign languages, mathematics and science) and introducing assessment for those key competences that are less familiar and present a new challenge (digital competence, learning to learn, social and civic competences, sense of initiative and entrepreneurship, and cultural awareness and expression).

Generally, examples in this document show that reforming assessment has no major financial consequences: reforms build on existing expertise and utilize existing structures. However, developing e-assessment requires infrastructure, development work, training and promotion. Ensuring teachers' assessment competences obviously require investment. Nevertheless, in the long run e-assessment will result in savings due to easier access to the material. As regards teachers' competence development, virtual platforms and supporting teachers' professional networks can reduce these costs and make implementation more effective.

This document takes stock of these developments and provides policy makers with an overview of current trends in assessment and examples of how assessment can be extended to support the development of all key competences – not only those that are easy to assess.

The focus of this document is on assessment of individual learners rather than evaluation of programmes, institutions or systems.

How to read this document?

This document is a result of the work of the Thematic Working Group ‘Assessment of Key Competences’ and its predecessor, the Peer Learning Cluster 'Key Competences – Curricular Reform'. It builds on the process of peer learning in the context of the Education and Training 2010 and 2020 work programmes and thus on the expertise, examples and experiences in participating countries.

Section I ‘THE KEY COMPETENCES FOR EUROPEAN CITIZENS IN THE KNOWLEDGE SOCIETY’ provides a summary on what key competences are and which competences are currently being assessed.

Section II ‘STRENGHTHENING ASSESSMENT OF ALL KEY COMPETENCES: LESSONS FROM MEMBERS STATES’ illustrates how Member States have specified key competences in learning outcomes and how they have developed assessment for summative and formative purposes. It continues with implementation of assessment reforms and presents the other policy areas that need to be addressed to make reforms successful.

APPENDIX I describes the key competences set out in the Recommendation of the European Parliament and the Council of 18 December 2006 on key competences for lifelong learning ("The Key Competences for Lifelong Learning - the European Framework").

A literature review on assessment of key competences and extended examples of assessment policies are available at http://ec.europa.eu/education/index_en.htm. This supporting document also includes a glossary. The references e.g. in footnotes of this this document are listed in full in the bibliography which forms part of the literature review. A JRC-IPTS report "Review of Evidence on the Use of ICT for the Assessment of Key Competences" (in press) will be available at: http://ipts.jrc.ec.europa.eu/publications/search.cfm. This website includes also other IPTS publications on e-assessment that contribute to Chapter 3.3. of this document.

I. THE KEY COMPETENCES FOR EUROPEAN CITIZENS IN THE KNOWLEDGE SOCIETY

1. We have a clear definition of the key competences needed for the 21st century

Key competences are a combination of knowledge, skills and attitudes appropriate to a specific context. The European Framework on key competences for lifelong learning defines eight key competences necessary for personal fulfilment, active citizenship, social cohesion and employability in a knowledge society. Critical thinking, creativity, initiative, problem-solving, risk assessment, decision-taking, and constructive management of feelings play a role in all of the key competences. All of the key competences are equally important because each of them can contribute to a successful life in a knowledge society.

Key competences[1]

Since 2006, the Reference Framework has been influential in reforms of school curricula and vocational programmes across the EU.[2] Many Member States have reformed their school curricula and vocational programmes in recent years, to introduce part or all of the approach set out in the Key Competences Recommendation. Austria, Belgium French Community, Czech Republic, Spain, Hungary, Island, Luxembourg, Latvia, the Netherlands, Poland, Romania have used the framework to structure the entire curricula and many countries have used the framework to enhance the role of cross-curricular work. This involves a major paradigm change from transmitting a static body of pre-defined knowledge to a more dynamic and holistic development of competences.[3]

When the EU reviewed[4] the implementation of the Key Competences Recommendation, it confirmed the extent of this change in the teaching and learning requirements of national curricula in the Member States. The traditional barriers between school subjects were breaking down, new demands were being made within these subjects, and wholly new subjects and approaches were being introduced, such as entrepreneurship education and "learning to learn" as a distinct educational objective.

The key challenge, and key weakness, that this review highlighted was the assessment of learning. While curricula and teaching were in the process of profound change, the way the students' outcomes were assessed, evaluated and certified had changed little over decades:

"A large number of countries are introducing reforms that explicitly use the Key Competences framework as a reference point. Good progress has been made in adapting school curricula. But there is still much to be done to support teachers’ competence development, to update assessment methods, and to introduce new ways of organising learning."

2. The main challenge now is to assess all key competences

The purposes of assessment and evaluation vary from gathering data for use in international comparisons of education systems to providing tailored support for individual learning. In this document assessment is taken to relate to judgements on the progression of competences of the individual learner and achievement of the learning goals, and on the effectiveness of institutions or systems, often based on aggregated assessment data.

The table below summarises the different contexts, functions and uses of assessment:

Table 2: Assessment and evaluation

Context || Function || Uses

System || International comparisons; National standards; Accountability. || Policy development, system monitoring

Institution || Monitoring against national standards; Internal quality assurance; Developing internal policies and procedures. || Feed into national data-gathering processes; Feedback to tutors, trainers, teachers, learners, parents.

Learner || Monitor achievement against national standards and curriculum objectives; diagnose strengths and weaknesses; track progress. || Feedback to learner on progress; feedback to tutors and teachers on success of teaching approach.

The National Assessment framework (NA) in Slovenia illustrates the different assessment purposes. Its primary goals are to improve students' competences and improve the quality of teaching.

National Assessment Framework in Slovenia

For students (and parents) and teachers the basic objective of the National Assessment Framework of is to provide students with information about their achievement of curriculum standards. For schools, the analysis of students’ achievements and the comparison with the average performance helps schools in the evaluation of their work. It should also help planning the teaching and learning activities in the school and give valuable information for planning of in-service teacher training. Schools' evaluation reports, in turn, feed into the evaluation of the school system as a whole.

Assessment is crucial for the development of key competences for two principal reasons. Firstly, by focusing on certain learning outcomes, assessment sends a clear signal that these competences are the priority for teaching and learning. Secondly, by providing information about learners’ progress towards these learning outcomes, assessment helps to adapt teaching and learning more effectively to learners' needs.

Moreover, assessment has important consequences for our educational paths. It not only affects how others see us but also how we see ourselves. More specifically, it affects how we think about ourselves, what we feel about ourselves and how we behave in response. Ultimately, it impacts on whether or not we become lifelong learners.[5] Assessment policy – and its implementation in schools – therefore needs to be handled carefully and informed by evidence. This document is for policy makers grappling with this responsibility.

By providing information about learners’ progress towards these learning outcomes, assessment helps to adapt teaching and learning more effectively to learners' needs. Moreover, the feedback given through assessment has an important impact on the learner's motivation, self-esteem and awareness of his/her own learning process.

Assessment also underlies important transitions such as gaining a place at university or an apprenticeship, or getting a job. How learning outcomes are assessed, recorded and certified is crucial. If assessments do not reliably reveal the competences that are needed for success in study or work, if they do not fit the curricula that has been followed, then they distort and impede people's life chances, and they make it harder for employers to select the right people for their workforce or colleges to select the right people for their courses.

Assessment is often controversial. In several Member States, there is public debate about whether tests have become easier compared with a past "gold standard". Employers and universities complain that good grades do not indicate the same level of performance as they once did. At the same time, assessment has to develop to relate to new curricular and learning demands, new skills and new ways of expressing them. Controversies concerning assessment may derive from the fact that its (changing) scope and purposes are not well understood or clear to all stakeholders. It is thus vital to communicate clearly what different types of assessments are supposed to measure or not.

Aware of the importance of assessment to individuals and the quality of teaching and learning, many of the Member States have identified key principles to be taken into account in developing and implementing assessment policies. These can be summarised as follows:

· Assessment should be fair: its methods should be related to the goals and objectives set beforehand and its purpose and consequences should be clear to all concerned. Assessment should take into account students' prior learning and their skills – such as language skills – to assess the progress made properly.

· Assessment should be reliable and valid so that assessors working independently and using the same methods come to same judgements. The better the expected learning outcomes are described, the higher the validity and reliability of the assessment is.

· Assessment should be learner-centred: it should give valuable and forward-looking information for the learners, their parents, teachers and schools on how to improve teaching and learning practices, making it thus an integral part of the teaching-learning process.

Across Member States, there are promising one-off developments to broaden the scope of assessment of the key competences strongly associated with traditional school subjects (such as mathematical competence) and those that present a new challenge – such as learning to learn, social and civic competence and digital competence. These developments are recent and valuable and need to be introduced more widely across EU Member States.

3. Yet too little is done on assessment

Despite the awareness of the impacts of assessment, it still tends to focus only on a narrow part of key competences. The key competences are wide-ranging and holistic. If only some competences are assessed, teachers and learners will tend to focus on these competences, with the result that some learners will not receive a complete education. Furthermore, if only some aspects of each competence are assessed – such as knowledge -, then the development of the other aspects - such as useable skills and attitudes - will be, at best, incidental, and may not happen at all. No matter how broad the intended learning outcomes are, if assessment is too narrowly defined, it will result in narrow teaching and learning.

In many Member States, nationally standardised tests are used to inform decisions about learners’ progression through school and to certify achievement (including selection into different school types), to monitor schools or education systems as a whole, or to identify individual learning needs. In compulsory education, these tests are generally limited to certain key competences. National tests in communication in the mother tongue or mathematics are therefore widespread. In many Member States, there are also national tests in science or foreign languages. However, few organise national tests relating to the other transversal key competences, which gives the impression that these important competences are seen as lower priorities.

National tests of key competences in compulsory education, 2008/09[6]

The figure above focuses on national tests that can be used for summative or formative purposes. It shows strikingly how the key competences that are crucial for lifelong learning, innovation and social and cultural development are not covered by national assessment regimes.

A similar mapping of national tests was done in 2011/2012, but it concentrated on only six of the eight competences, excluding 'learning to learn' and 'cultural awareness and expression'. This mapping reveals that not much has changed in the focus of the national tests in four years: traditional school competences continue to be tested to a greater extent than more transversal competences. The most important change in the testing of transversal competences has been the increase in the number of countries that test social and civic competence (from 4 in 2008 to 11 in 2012).[7] By contrast, digital competence and entrepreneurship continue to be left out.

However, national tests are only part of the picture. The assessment of key competences is sometimes also organised at local level or school level under the responsibility of teachers, particularly when the purpose is to inform day-to-day teaching and learning. Thus, while a major study of the 27 Member States found that national tests were limited to the key competences mentioned above, it also showed that a wider range of key competences was being addressed by teachers and learners themselves. In addition, there are some national initiatives seeking to expand assessment to a wider range of competences.[8] More in detail, the review of the implementation of the Key Competences Recommendation reveals that interdisciplinary projects are used as part of final exams in Austria, Denmark and Germany. In Cyprus, Denmark and Austria ICT-based and web-based assessment methodologies are emerging. Many countries have started to make the best out of standardised test by giving feedback to teachers and learners in schools; this is taking place in Austria, Cyprus, Germany, Denmark, Spain, France, Hungary, Ireland, Lithuania, Norway, Portugal and Slovenia. Formative assessment – "assessment for learning" is well developed in UK-Scotland and UK-England; these countries are strong in linking the development of curricula and teacher education with assessment reforms. "Tools" for formative assessment, such as self-assessment, peer assessment and portfolios are promoted in Austria, Cyprus and Ireland, for instance.

However, it is worth noting that despite these promising initiatives there is a need to harness assessment more systematically to support the development of all key competences. This requires mapping all the objectives and intended outcomes of learning against existing assessment, developing methods to fill the gaps, and making sure that teachers have the competences and working conditions to integrate assessment into the teaching and learning process most effectively.

II. STRENGHTHENING ASSESSMENT OF ALL KEY COMPETENCES: LESSONS FROM MEMBERS STATES

This section focuses on developing assessment for all key competences. While the first section gave an overview on key competences and the importance of assessment, this section gives more detailed guidelines and examples from Members States on how to make it happen.

It develops a comprehensive assessment strategy that should be constructed around four main steps: 1) analysing which of the key competences are covered by national tests and other forms of summative assessment; 2) how key competences can be turned into more detailed learning outcomes that can be accurately and meaningfully assessed; 3) how broadening the scope of assessment can reveal, report and develop more of those competences and, finally, 4) how reforms of assessment can be implemented and what other policy areas are needed to support implementation.

The OECD country report from the Flemish Community of Belgium illustrates the shift of emphasis in assessment and why these steps presented in this chapter are necessary[9]:

The changing view of learning and education which has characterized education policy over the last decade entails a change in how assessment and evaluation are viewed. The shift from the traditional ‘test culture’ towards a ‘broad assessment culture’ has recently gained importance. The broad assessment culture relates to the implementation of ‘active learning methods’. Students are no longer perceived as passive subjects, but they are involved in the assessment of their performances and progress, and may even be involved in the assessment of their classmates. The broad assessment culture evaluation is not just the last part of the learning process, but instead an integrate part of learning. Evaluation not merely serves the goal of judging students, but also (and more importantly) guides students to gain knowledge, skills and attitudes through feedback mechanisms. This means that evaluation is conducted continuously, because both teachers and students need up-to-date information on individual performances in order to create effective learning environments. Evaluation as part of the learning process helps students in their development of holistic competences: next to assessment of cognitive performances, also dynamic-affective, relational and motor aspects are continuously assessed. The broad assessment culture naturally entails different evaluation methods than only the classical tests: observation, portfolios, reflection sheets, peer and self-assessment are only a few examples of the wide range of assessment possibilities. The teacher needs to consider which assessment method contributes most to guiding students to the aspired objectives. Within this culture it is more important to provide information on the progress of a student compared to his previous achievements, than answering the question ‘what grades did this student achieve on a pre-determined scale from one to ten, or compared to his classmates?’ The broad assessment culture shifts from pre-determined or peer- group norms to an individual norm (without losing sight of the pre-determined attainment targets and developmental objectives out of sight)

1. Defining key competences as tangible learning outcomes

As noted in the previous section, the European Reference Framework defines the scope of knowledge, skills and attitudes to be developed in a wide range of everyday contexts by the end of initial education. It does not describe them as specific learning outcomes that are needed to guide day-to-day teaching and learning, however, examples from Member States and evidence from the research literature indicate that it is important to make key competences "assessment ready" by specifying them as concrete and tangible learning outcomes.[10] This specification is to be done by policy makers, teachers and learners within their individual education systems, institutions and programmes of learning.

Policy makers can ensure that these learning outcomes are consistently specified across curricula, syllabi, specifications, standards or similar framework documents. However, teachers have a crucial role to play in interpreting these documents. Importantly, they can identify and plan opportunities for learners to develop their key competences within their specific subjects, across several subjects in collaboration with other teachers or beyond subjects as, for example, in whole school projects. Developing key competences across curricula and programmes may mean teachers working together in ways that are unfamiliar, such as across departments. It is therefore crucial that teachers and school leaders have a shared understanding of the key competences to be developed and the responsibilities and roles of each member of the school community in how key competences are taught, learned and assessed. Effective cooperation and collaboration between teachers and school leaders will enable learners to make connections across their learning and develop more holistic perspectives that prepare them for life and work.

The amount of detail that is specified in framework documents varies according to the purposes of the assessment. For example, high stakes summative assessments need to be highly reliable to ensure that their consequences are fair. To provide a basis for consistent assessment practices, the learning outcomes for assessments such as school-leaving examinations are therefore generally more detailed and are centrally specified. However, defining learning outcomes in excessive detail risks reducing learning to sequences of isolated tasks and restricting learners’ opportunities to demonstrate the range of their competences. A balance therefore needs to be struck between specification in policy documents and scope for interpretation by teachers and learners themselves in practice[11]. However, this interpretation needs to be founded on a shared understanding of key competences. This can be addressed through the training, guidance, networks and evaluation detailed in Chapter 4 of this document.

The Czech Republic provides an example of learning outcomes to be achieved by the end of initial education – developed at national level – and those to be achieved at interim stages developed by teachers with their students. This example focuses on civic competence and shows how it is to be developed in contexts across several subjects, areas or fields.

National and local learning outcomes for key competences in the Czech Republic:

Nationally: For basic education, the Framework Education Programme (FEP) identifies a similar range of key competences at national level as the Reference Framework does at European level. It also identifies educational content in subjects, areas or fields which provide contexts for the application of competences. Each competence is expressed as a set of specific goals, which define the level of competence that learners should develop by the end of compulsory schooling (grade 9). For example, one of the goals for civic competence is that the student ‘considers and judges phenomena, processes, events and issues from diverse perspectives’. This goal implies not only knowledge and skills but also certain attitudes, such as openness and tolerance.

Locally: Every school develops a School Education Programme (SEP) within the structure and content set out in the FEP but according to their local circumstances and their learners’ needs. There is a curriculum handbook providing teachers with examples of how to detail key competences and relate them to content areas at two stages: the end of compulsory education (grade 9); and an interim stage (grade 5). Since the same level of competence can be demonstrated in different ways depending on the educational area and the exact context, these are simply illustrations for teachers and learners to use when reviewing examples of work to assess progress.

The Czech example shows how the task of specifying key competences is divided between the national level actors and local school and teachers. While the national level work provides school guidelines and ensures overall coherence, the work done by schools and teachers ensures ownership and understanding of how key competences can be developed within day-to-day learning contexts.

This chapter now presents how each key competence can be made assessment ready by specifying them as concrete learning outcomes.

1.1 Mother tongue, Foreign languages, Mathematics and Science

The specification of key competences as learning outcomes is, in general, more advanced for those competences that are closer to traditional school subjects. Mother tongue and foreign languages include outcomes such as reading, writing, speaking and listening skills: often the attitudinal aspects such as readiness to communicate are specified as well. Mathematics[12] and science outcomes often include the capability to solve basic mathematical operations but also more transversal skills, such as problem solving, initiative and the ability to come up with creative solutions. Often the real challenge of specifying key competences as learning outcomes is related to the more transversal competences.

1.2 ‘Learning to learn’

Developing a shared understanding of what 'learning to learn' means in practice may be a particular challenge.[13] Yet some Member States have already developed learning outcomes for this key competence. To demonstrate some of these, examples from two Member States are shown here.

The first example is from Lithuania and is significant for three reasons. Firstly, it shows how attitudes can be explicitly incorporated into learning outcomes. Secondly, it demonstrates how the degree to which a competence has been acquired can be assessed with levels indicating each learner’s progress. Thirdly, the learning outcomes are written from the student’s perspective to help them monitor their progress with their teachers.

Learning outcomes and levels for 'learning to learn' in Lithuania:

An EU-funded project in Lithuania identifies four elements within learning to learn competence for learners in the 11-12 and 13-14 age groups:

1. Attitude towards learning and willingness to learn

2. Setting objectives and planning activities

3. Organisation and management of activities

4. Reflection on learning activities and outcomes, and self-assessment.

These elements are described in more detail for each of the age groups at four levels of progression towards development of the competence. The descriptions for learners aged 11-12 are shown here.

First steps || Going in the right direction || Close to the target || Competence acquired

1. Why I learn (Attitude towards learning and willingness to learn)

I learn when others urge, encourage, and persuade me. I feel secure when somebody helps me. When I work together with others, I observe more often how others are doing things. || I learn because I have to, but I eagerly learn only what I am interested in. I am self-confident when I have to complete easily understandable tasks. When I work together with others, I try to express my opinion. || I understand that learning is my duty and I therefore try to learn not only things that I am interested in. I usually feel self-confident when I have to perform both usual and new non-complex tasks, and I am not afraid of making mistakes. When I work together with others, I often express my opinion and I help others whenever I can. || I learn eagerly and responsibly, because I like learning and understand that what I learn now will serve me in the future. I am self-confident, I am not afraid of challenges, and I welcome various new tasks. When working together with others, I not only learn from them but also help them eagerly and confidently.

2. What I learn (Setting objectives and planning activities)

I set learning goals and plan learning activities when others help me. I manage to complete tasks on time when others urge me to. || I consult others to set my learning goals and plan my learning activities. I manage to complete certain tasks on time. || I try to set my learning goals autonomously and plan my learning activities. I usually manage to adhere to the schedule and complete tasks on time. || I set my learning goals autonomously. I successfully plan my learning and am therefore able to complete most tasks and tests on time.

3. How I learn (Organised and targeted learning activities)

I use such learning methods and choose such information sources and learning tools which are offered to me by others. || With the help of others, I decide what learning method, learning tools, and information sources are best to perform a certain task. || I try to use different learning methods and I use various information sources. I try to obtain any required learning tools in advance. || I successfully use various learning methods and choose appropriate information sources. I obtain any required learning tools in advance.

4. I reflect on my progress (Reflection on learning activities and outcomes and self-assessment)

When others help and encourage me, I observe my learning progress and sometimes listen to advice given to me and try to follow the advice. || When others help me, I reflect on how and what I achieved in learning and in what areas and how I should improve. || I autonomously reflect on how and what I achieved in learning; others help me to plan in what areas and how I should improve. || I autonomously reflect on how and what I achieved in learning and plan in what areas and how I can improve.

The second example is from France, where 'learning to learn' and sense of initiative and entrepreneurship are combined in learning outcomes relating to autonomy and initiative. The example is significant for showing how a competence can be divided into fields, items and assessment indicators. These are set out in teaching resources called reference grids for the assessment of competences. These reference grids comprise the assessment requirements for competences in the first, second and third cycles of basic education. The example is an extract from the reference grids for the third cycle (the end of lower secondary education) and is also significant in referring to learners’ development of competences in and beyond school settings. Learners’ development of the competences is recorded in a Personal Competences Booklet, which is detailed in the section on using assessment to develop learners’ key competences

Learning outcomes for the autonomy and initiative competence in France

Competence: Autonomy and initiative Field: Take action in your courses of learning and guidance Item: To be able to self-assess and to write about your interests and the competences you have acquired. Detail of item · Locate and identify the competences you have acquired. · Identify and order your interests and your personal aspirations. · Analyse the consistency between your aims and your present direction. Assessment indicators: Self-assessment here refers to the student’s holistic understanding of themselves and to self-assess in relation to individual achievements. In and beyond school settings, the student knows how to: -use the tools provided in the disciplines to identify strengths and weaknesses; -identify their interests in school activities -anticipate the results of an assessment or task and its impact -identify the activities that can contribute to everyday life. The student can analyse the advantages and disadvantages of their choices. The student appreciates the difficulties and obstacles. The student considers alternative choices.

These two examples show how specific learning outcomes of the 'learning to learn' competence can be formulated. The Lithuanian example also demonstrates how the 'learning to learn' competence can be described as levels that help learners and their teachers to assess progress. The French Personal Competences booklet exemplifies how these learning outcomes can be recorded and communicated so that, for instance, parents can follow and support their children's learning skills.

1.3 Social and civic competence and creativity

Member States have also expressed particular interest in the assessment of social and civic competences, creativity, and problem-solving. The example is from Ireland, where learning outcomes for each key skill are first specified, and then embedded in contexts provided by the syllabus for each subject. Learning outcomes in creativity and critical thinking are therefore embedded in subjects such as politics and society. Importantly, a process of auditing ensures that the key skills are embedded in subjects, assuring that these different documents are consistent and coherent. Extracts of the learning outcomes for the key skill in creativity and critical thinking and the active citizenship topic in the politics and society syllabus are shown below:

Critical and creative thinking and politics and society in Ireland:

Key skill: || Elements: || Learning outcomes:

Critical and creative thinking || Thinking imaginatively, actively seeking out new points of view, problems and/or solutions, being innovative and taking risks. || · recognise that different mind-sets are associated with different forms of thinking · be motivated to seek out alternative perspectives and viewpoints and to reframe a situation · be willing to take risks and to learn from mistakes and failures · be persistent in following through ideas in terms of products and/or actions · develop a strong internal standard in relation to the merits of their own work.

Subject: Politics and Society

Student learns about: || Student should be able to:

Identifying, evaluating and achieving personal and collective goals, including how to develop and evaluate action plans. || · Set realistic personal and collective goals and targets to be achieved within a time frame · Construct action plans to help reach the targets and identify methods for monitoring how well the plans are working (e.g. deadlines, feedback from others) · Co-operate with group members to identify collective goals · Co-operate with group members to identify how different roles can contribute to the overall goals · Communicate ideas and needs within the group · Engage in personal reflection on the process of setting goals or targets.

In Ireland, the approach is therefore to embed key competences in subjects and the example above –from upper secondary level – gives guidelines to teachers and students on what developing these competences and assessing them means in practice. (See accompanying document at [add URL] for a more detailed description of Key Skills in Ireland)

1.4 Digital competence

The next example is from the UK- Northern Ireland, where the cross-curricular skill called 'Using ICT' is divided into: explore, express, exchange, evaluate and exhibit. Each of these elements is described at five levels, set out as ‘can do’ statements. These are applied in subjects and areas across the curriculum. The levels for exchange and evaluate are shown here because exchange emphasises responsible attitudes and evaluate supports learning to learn. The expectation is for pupils to reach Level 2 at age 7 and Level 4 by the end of primary school at age 11. However, it is recognised that learners' progress at different rates and some will be above or below this level. Teachers are encouraged to develop the broad levels into narrower sub-levels in order to monitor and report their pupils’ progress with more precision.

Using ICT in UK- Northern Ireland

Requirements for Using ICT || Level 1 || Level 2 || Level 3 || Level 4 || Level 5

Exchange || Pupils can: || Pupils can: || Pupils can: || Pupils can: || Pupils can:

Communicate safely and responsibly using a range of contemporary digital methods and tools, exchanging, sharing, collaborating and developing ideas || Know what digital methods can be used to communicate || Identify and talk about ways of communicating digitally || Use a contemporary digital method to communicate or contribute to a supervised online activity || Use contemporary digital methods to communicate, exchange and collaborate in supervised online activities || Use a range of contemporary digital methods to communicate, exchange and share their work, collaborating online with peers

Evaluate || Pupils can: || Pupils can: || Pupils can: || Pupils can: || Pupils can:

Talk about, review and make improvements to work, reflecting on the process and outcome and consider the sources and resources used, including safety, reliability and acceptability || Talk about their work || Talk about how to improve their work || Make modifications to improve their work || Use appropriate ICT tools and features to improve work || Use appropriate ICT tools and features to carry out on-going improvements and evaluate process and outcome

1.5 Sense of initiative and entrepreneurship

This next example is from the South East European Centre for Entrepreneurial Learning (SEECEL)[14] specifying the learning outcomes related to sense of initiative and entrepreneurship. SEECEL is a Croatian initiative including the following countries: Albania, Bosnia and Herzegovina, Kosovo, Macedonia, Montenegro, Serbia and Turkey. SEECEL has developed a three-level assessment for entrepreneurship as a key competence at ISCED 2. The levels include measuring the progress of any given school in becoming an entrepreneurial school using 10 indicators, the assessment of teachers and school managers through a questionnaire and the assessment of defined learning outcomes at ISCED 2 level. These learning outcomes (knowledge, skills and attitudes) defined as "Specific Measurable Attainable Relevant and Timely (SMART)" are based on Bloom's taxonomy. They are implemented across the curriculum in existing subjects – hence learning outcomes are assessed using methods normally used to assess those subjects. Alongside these methods, self-evaluation is also used for both students and teachers.

Defining learning outcomes for entrepreneurial learning at the ISCED 2 level, SEECEL in Croatia:

|| KNOWLEDGE || SKILLS || ATTITUDES

Learning outcomes || 1. Define and explain the difference between being an entrepreneur and acting entrepreneurially 2. Explain why scarcity necessitates decision making. 3. Recognise and define what products and services are in supply and demand at local and national level. 4. Define criteria necessary for decision making at individual and group level and explain their impact. 5. Compare benefits with costs. 6. Calculate productivity and identify comparative advantages. 7. Identify different job opportunities. 8. Identify and describe at least two different career paths. 9. Explain the importance of externalities and sustainable planning (think green) for entrepreneurial activities. 10. List the phases of entrepreneurial activity in a classroom or school level example. 11. Identify steps in problem solving and risks and give examples of each. 12. Develop an entrepreneurial idea. 13. Explain public goods and give examples. || 1. Demonstrate ability to work individually and in teams. 2. Demonstrate ability to communicate ideas to others efficiently. 3. Demonstrate ability for creative and innovative thinking. 4. Demonstrate planning, delegation and leading skills during group work. 5. Demonstrate ability to use brainstorming method. 6. Develop decision making grid. 7. Demonstrate ability to recognise and list risks. 8. Identify environmental consequences of their actions. 9. Illustrate ability to solve problems and to make decisions together with others. 10. Demonstrate ability to evaluate results and processes from group work. 11. Demonstrate how to set up a production plan. 12. Present an entrepreneurial idea. 13. Appraise own assets and competences. || 1. Take responsibility for completing tasks, fulfilling obligations and meeting deadlines. 2. Take initiative to network with individuals and groups. 3. Demonstrate ability to work independently. 4. Demonstrate ability to respect others. 5. Demonstrate ability to accept innovation and change. 6. Demonstrate responsibility for public goods.

Teaching and learning methods || Lectures Discussions Group work Peer group presentation Local visits || Lectures Discussions Group work Peer group presentation Local visits || Teachers as facilitators Lectures Discussions Group work Peer group presentation Local visits

Assessment methods || Essays Presentations Project/ group work || Practical assessment Fieldwork Presentation Project work Self-evaluation || Practical assessment Field work Project work

1.6 Cultural awareness and expression

A final example of the specification of learning outcomes relates to the key competence in terms of cultural awareness and expression. As cultural competences are generally defined in learning outcomes in school curricula, the following example from Malta is chosen to show how key competences can be incorporated into national qualifications frameworks and learning outcomes for vocational education and training. The recently developed Maltese national qualifications framework requires mastery of a vocational field to include acquisition of all eight key competences. Mastery at each of three levels is specified as learning outcomes in the framework. As in the European Qualifications Framework, the key competences are described in terms of knowledge, skills and competences. Each key competence is also presented in different strands, so that cultural awareness and expression contains culture and the expressive arts and culture and the workplace. The example shows an extract from the learning outcomes for culture and the expressive arts at Level 2 and culture and the workplace at Level 3 of the national qualifications framework.

Key competences in the national qualifications framework in Malta

Cultural awareness and expression || Knowledge || Skills || Competences

Culture and the expressive arts (extract from Level 2) || Learners will have… − awareness of the interdependence between cultural heritage and natural heritage, and the need to safeguard them || Learners will be able to… − appreciate the relationship between cultural heritage, human rights (including those of minority groups) and the requirements of mutual respect. || Learners will demonstrate… − reflecting on how their response to their environment, their interaction with nature and their history enhances their sense of identity and continuity.

Culture and the workplace (extract from Level 3) || Learners will know… − cross-cultural ethics and etiquette that affect the workplace and can enhance work effectiveness. || Learners will be able to… − identify and rectify areas of cross-cultural misunderstanding that may cause conflict in the workplace. || Learners will demonstrate… − working effectively both as an individual and as a member of a group to bring about significant change within the workplace.

These are some examples of how Member States are specifying knowledge, skills and attitudes in five ‘transversal’ key competences. Specifying intended learning outcomes represents a focus for planning teaching, for reflecting upon learning and for developing assessments that promote the development of key competences.

As teachers are not often prepared to assess the full set of key competences within their initial education, it is crucial to ensure their get support during their careers to put in place effective assessment practices. An example from the Czech Republic shows an option on how to achieve this.

Guidelines for schools to specify competences in Czech Republic

To help the schools, the Research Institute of Education in Czech Republic published documents specifying individual key competences in a set of goals and delivered model examples concerning how to develop and evaluate key competences comprehensively. Practical suggestions as to the development and evaluation of key competences are also published at the www.rvp.cz methodological portal.

2. Using assessment to measure learning outcomes

Reporting learners’ development of key competences is a challenge to existing assessments that focus on knowledge and skills limited to subject contexts. Key competences emphasise knowledge, skills and attitudes required for contexts that extend beyond subject boundaries. The previous section showed how key competences can be specified as learning outcomes within, across or beyond subject contexts. Assessment methods can then be designed to ensure that information about learning in relation to these outcomes is actually gathered and used to report learners’ progress.

This section focuses on summative assessment of key competences, which means assessment for the purpose of reporting learners’ achievements at the end of a period of learning. Assessing the breadth and complexity of key competences is a particular challenge for summative assessments in high stake contexts. This type of assessment typically reports learners’ key competences for selection, certification or qualification. To ensure their consequences are fair, these assessments require carefully controlled conditions. However, these conditions can be an obstacle to assessing students in real-life contexts or authentic representations of them. Innovative methods that bring these contexts into assessments or bring assessments into these contexts are therefore required.

This section presents examples of summative assessments, sometimes in high stakes situations, using one or more of the following methods to assess the development of key competences:

(1) Standardised tests where all questions or ‘items’ address aspects of key competences

(2) Attitudinal questionnaires with these tests or with performance-based assessment

(3) Performance-based assessment, sometimes in conjunction with standardised tests

Examples from Member States and the literature show how these assessment methods can strongly influence teaching and learning practices in favour of the development of key competences. Of course, the reverse is also true: if summative assessment does not incorporate key competences, it can serve to limit the curriculum.[15]

2.1 Standardised tests

Although national standardised tests in Member States typically focus on key competences associated with traditional subjects, there are examples specifically designed to assess a wider range of key competences, notably social and civic competences. Items in these tests often present learners with a problem context and require them to find, communicate and justify a solution or response. As a result, they help to address the creativity and problem-solving themes that cut across the key competences. These test items can also present learners with complex contexts using texts or pictures that are rich in information. Some items may then extend the challenge by requiring learners to draw upon their wider understanding of the context. Learners may therefore be required to work through a chain of reasoning with several steps. As a result, learners can apply different combinations of knowledge and skills whilst relying on supporting attitudes such as persistence.

Tests items can use a range of response formats to assess different elements of learners’ key competences. These include multiple choice (where one choice is deemed correct), complex multiple choice (where more than one choice may be correct), short or extended response formats (to a greater or lesser extent requiring learners to formulate their own responses). In Spain and Poland, standardised tests with a range of response formats assess different aspects of learners’ social and civic competences.

Test items for social and civic competence in Spain

The general diagnostic evaluation provides information on learners’ development of key competences in order to enhance teaching and learning across schools in Spain. Standardised tests assess a representative sample of students in the fourth year of primary education (ages 9-10) and the second year of secondary education (ages 13-14). This example relates to test items for social and civic competence in primary education. These items focus on: the individual; society; social organisation; societies in the past; and, current democratic societies and human rights. To assess different aspects of the competence, several different response formats are used, such as true-false, matching, sentence completion and short response items. Many of the test items are multiple choice, reflecting an emphasis on reliability. Two items are reproduced here, indicating some contents and formats of the test of social and civic competence in primary education. One multiple choice item focussing on civic knowledge is: A) Rules of our towns and cities are written by political parties. B) Are dictated by the different ministries. C) Are approved by the governments of the municipalities. D) Are included in the Spanish Constitution. Other items involve more reflection, description or explanation, such as this short response item: "Give two advantages of working together in a school."

Social studies questions from an upper secondary examination in Poland:

In Poland, the requirements for social and civic competences are defined in the core curriculum and provide the basis for external examinations. Some contrasting test items from the upper secondary social studies external examination are: · List three political rights of the EU citizen. · Choose four sentences that are opinions [from eight sentences] · Can Polish society influence public matters in local communities? Write an essay, express your point of view and describe three forms of social involvement and three forms of political involvement.

Although tests may focus on one particular competence, other competences are often necessarily assessed too. This is likely when assessments model real-life contexts. The OECD’s PISA survey provides some insight into the value of assessing multiple competences simultaneously to reflect the complexity of real-life. Whilst PISA assesses mathematical competence, its incorporation of individual, societal, occupational and scientific contexts in test items means that other competences are also addressed. Thus a mathematics item which asks students to respond to a journalist’s claim that the number of robberies has “hugely increased” gains information on civic competence too. Conversely, another item from the social studies examination in Poland requires some mathematical competence.

Look at the charts and describe the differences between them. Chart one: Do you feel you can influence the decisions taken in your country? Chart two: Do you feel you can influence the decisions taken in your local community?

A further example of the assessment of combinations of competences comes from Germany, where test items are being developed in order to assess, for example, communication competence in biology, chemistry and physics. Similarly, PISA science and mathematics test items often require students to interpret a large amount of information presented as texts or graphics, which consequently means that the communication competence is also assessed. In these examples, learners are required to use combinations of competences, as in many real-life situations. In order to gain a clear picture of each key competence, these combinations can be varied systematically.

Also the development of social and civic competences can be monitored by standardised test, as the following example from Malta shows:

Monitoring social and civic competences through standardised tests in Malta

The nationally standardised Social Studies annual examinations at the end of Year 4, Year 5 and Year 6 are based on the content, skills and attitudes envisaged in the syllabus of each year. Apart from the fact that students have to answer questions that are purely theoretical and knowledge-based, they are also tested on practical situations or case-studies. For example, students are given a number of case-studies where students need to put their theoretical knowledge into action and demonstrate how they would have reacted or behaved in a particular situation. These are called ‘application questions’. A number of questions are directly or indirectly linked to the social and civic competencies envisaged in European Framework of key competences. For example, certain issues that are asked reflect not only the local/national level, but also the wider European and global dimension. In these examinations students need to demonstrate “full respect for human rights including equality as a basis for democracy, appreciation and understanding of differences between value systems of different religious or ethnic groups.” Examinations therefore include a range of issues such as sustainable development, democracy and human rights, respect for the values and privacy of others, community cohesion, justice, equality, citizenship, civil rights, work organisations, gender equality and non-discrimination, society and culture.

2.2 Attitudinal questionnaires

Some assessments combine tests that focus on knowledge and skills (or cognition) with questionnaires that focus on attitudes (or affect) and learning practices (or metacognition). Major international surveys such as PISA, TIMSS and PIRLS offer examples of questionnaires that incorporate attitudes to learning. However, unlike knowledge and skills, these attitudes are not seen as learning outcomes in their own right. The Framework for a European Test to Measure Learning to Learn differed in this respect, using tests to assess learning outcomes relating to cognition and questionnaires to assess learning outcomes relating to affect and metacognition. However, it transpired that these aspects of learning to learn were too closely linked to be assessed through separate tests and questionnaires.

There are also more general problems with the use of questionnaires to assess attitudes as learning outcomes. Firstly, attitudes vary over time and between real-life contexts but questionnaires provide only snapshots in ‘test’ contexts. Secondly, these snapshots are indirect measures, reliant on the accuracy of learners’ self-reports. Thirdly, these self-reports may be influenced by the ‘social desirability’ of particular responses to questionnaire items, particularly if learners’ perceive their answers as having consequences for themselves. Overall, to assess attitudes to learning and learning to learn more generally, the research evidence suggests it is better to use techniques such as observation and dialogue than questionnaires and tests. The contribution of these techniques as performance-based assessments is explored in the next section. However, questionnaires may nonetheless have a specific role to play in assessing one important aspect of learning to learn, namely the accuracy of learners’ self-assessments. Indeed, questionnaire items that ask learners to self-assess their competences are well-established and can be found, for example, in PISA questionnaire items relating to self-efficacy.

Comparing learners’ self-assessment judgements with standardised tests can yield useful information about their learning to learn competence, particularly their metacognition. Furthermore, there is evidence that these judgements play an important role in helping learners to overcome difficulties. Learners have little to gain from inflating their judgements when they know these will be compared with standardised judgements. It is therefore possible to combine the use of questionnaires and tests to assess this specific aspect of learning to learn for either summative or formative purposes. Further research evidence and results from pilot projects should be considered. For example, in Lithuania, an EU-funded project relating to competences in learning to learn, initiative and creativity is producing and piloting an instrument with assessment tasks for students, together with a student questionnaire and a teacher questionnaire.[16]

2.3 Performance-based assessments

Several countries in and beyond Europe have expanded the range of methods they use in summative assessments. This includes assessments in high stakes situations such as school-leaving examinations. The range of methods generally includes standardised tests, reflecting the demand from higher education, employers, students and their parents for highly reliable results. However, the tests are often combined with other methods to gain more and different types of information on which to base inferences about learners’ key competences. These other methods can be characterised as performance-based assessment and they include portfolios, projects, coursework, reflective diaries, presentations, interviews, role plays or group work. Furthermore, these methods are also often used instead of standardised tests, particularly in lower stakes situations.

There are two important reasons for performance-based assessment of key competences. Firstly, since it is predicated on observation, dialogue or sustained tasks, it can increase the range and authenticity of the ‘real-life’ contexts in which learners are assessed. This not only focuses teaching and learning on key competences but also contributes to the development of key competences. Secondly, it can incorporate more direct measures of attitudes in these contexts, such as observations. Two more general reasons are also apparent. On the one hand, performance-based assessment increases the number of assessment ‘events’, providing a fuller picture of performance. This can include information about learners’ progression. On the other hand, it can provide more information not only about the products of learning but also about the learning process itself. Information about the learning process is clearly an important target for assessments of learning to learn. Portfolio assessment is particularly well-suited to gathering information about the learning process and promoting key competences. The next section will focus more on these aspects of assessment of key competences.

The Slovenian example illustrates how performance-based assessment can be incorporated in high-stakes summative assessment.

Including laboratory work in examination in Slovenia

In Slovenia laboratory work in science is assessed as 20% of the final certificate in science subjects in the Matura examination. The laboratory work examination is part of the school-based examination for the final certification. The other 80% is based on external examination. The assessment of laboratory work in the final examination provides a strong incentive for teachers to use more student centred, inquiry driven laboratory work in their teaching, in line with curricular requirements.

Since performance-based assessment is often based on performances during the period of learning, the teacher often acts as the assessor. Whether performance-based assessment is used on its own or in conjunction with standardised tests, measures therefore need to be in place to ensure that it, too, is standardised. This ensures that any variation in the way that learners’ work is assessed is kept to an acceptable minimum. Teacher education, assessment guidelines, work examples, task or item banks, moderation meetings and inter-school networks play an important role in developing consistent assessment practices. When these types of measures are implemented, teacher assessment judgements can meet the minimum levels of reliability required for summative assessment.

Examples from Germany and Austria show how external examiners can review and moderate teachers’ assessment judgements to increase the comparability of results between students, teachers and schools. In Austria, the range of assessment methods for the high stakes school-leaving examination has been broadened in order to address cross-curricular competences valued by higher education and employers. In Germany, the range of assessment methods for the general secondary final examination has also been broadened to address a wider range of key competences. The methods in these examples have the potential to assess the full range of key competences from communication in the mother tongue to sense of initiative and entrepreneurship. The examples are detailed below.

Performance-based assessment for the school-leaving qualification in Austria

In Austria, the qualification for entry to university or tertiary education uses a range of assessment methods to assess competences across the curriculum. These methods include: • A multi-disciplinary paper on a research project • A standardised, competence-based written examination at the end of the final year • An oral examination in front of a representative of the examinations commission after the written examination, when they present and defend their project. These methods assess a wide range of competences, including those relating to communication, mathematics, science, languages, learning to learn, initiative and entrepreneurship, and social and civic competences. Importantly, these competences are assessed in relation to real-life contexts provided by the research project.

To assess a wider range of key competences, Baden-Württemberg has introduced two new elements into the final examination for general education students in Year 10 in the Realschule: 1. The EuroKom (European communication) exam counts for around one half of the final grade in the obligatory foreign language. Students take the 15-minute examination individually or in pairs and it consists of three elements: a) a pre-prepared presentation, followed by questions, on a topic discussed and agreed in advance with the examining teacher b) an oral comprehension exercise c) a role-play based on an authentic situation. 2. In addition to written examinations in the core subjects of German, mathematics and the obligatory foreign language, pupils must also take a cross-curricular competence examination. Like EuroKom, this examination consists of a pre-discussed and pre-prepared presentation, followed by questions from the examiners. However, in this case the examination is taken by students in groups of between three and five, and must cover at least two curricular subjects. The examiners are teachers in these subjects, including an external moderator from a different school.

Performance-based assessment for the school-leaving qualification in Germany

If safeguards for the reliability of assessment judgements are put in place, performance-based assessment offers a great deal of promise for a more complete assessment of key competences, even for summative assessment in high stakes situations. Given the particularly strong influence that high stakes assessment exerts on teaching and learning practices, performance-based assessment can therefore promote the fuller development of key competences. Furthermore, since performance-based assessment can include information about learners’ progression during a period of learning and summarise learners’ achievements at the end of a period of learning, it may serve both formative and summative purposes.[17]

3. Using assessment to encourage the development of key competences

This section focuses on formative assessment, which means assessment for the purpose of helping learners develop their key competences. Formative assessment is a powerful set of practices for encouraging the development of competences, because it:

(1) Involves the learner in clarifying learning outcomes, setting goals, planning learning and assessing progress

(2) Provides timely feedback via teachers and peers or directly to the learner, enabling them to make informed adjustments to plans and goals

(3) Generates information about key competences that is valid for the purpose of informing next steps in teaching and learning, enhancing motivation and achievement

(4) Gathers information about learners’ knowledge, skills and supporting attitudes as they are developed and applied in a variety of contexts over an extended period of time.

More generally, the role of formative assessment in promoting learning in schools was popularised by research on ‘Assessment for Learning’ (AfL). In England, for instance, it is defined as “the process of seeking and interpreting evidence for use by learners and their teachers to decide where the learners are in their learning, where they need to go and how best to get there.”[18] It is used in day-to-day learning and teaching situations, for periodic assessments as a broader view of progress is gained across subject for teacher and learner as well as for formal recognition of pupils’ achievement that is reported to parents/carers and next teacher(s).

AfL has influenced policies and practices in several countries. For example, it provided the basis for the ‘Assessment is for Learning’ initiative in UK- Scotland. This initiative sought to embed formative assessment in teaching and learning by developing a three-way relationship between the curriculum, pedagogy and assessment.

The Assessment is for Learning initiative in Scotland

Assessment is for Learning (AifL) in Scotland supports the innovations taking place under the Curriculum for Excellence reform. It expands the formative assessment concept in terms of a three-way relationship between curriculum, learning and teaching, and assessment: · Assessment of learning is about gathering and interpreting the evidence. It connects the curriculum and assessment. · Assessment for learning is about supporting classroom learning and teaching. It connects assessment and learning/teaching. · Assessment as learning is about learning how to learn. It connects learning/teaching and the curriculum. This initiative resulted in the development of an integrated assessment system where: · formative assessment emphasises self-assessment and teachers use a range of evidence to monitor their pupils’ progress · teachers’ moderate their own summative assessment judgements using external tests and each other’s judgements by working collaboratively · to avoid distorting the curriculum, information from external assessments that monitor standards is based on a sample and is not used for formative or summative purposes. AIFL takes a whole-school approach to assessment, mobilising the support of teachers, parents, managers and pupils and engaging in assessment for learning at every level in the system. Teachers are supported through dedicated training and on-line resources including a self-assessment toolkit, case studies on target-setting, tracking and recording progress, and support for teacher networking and mechanisms for the sharing of good practice and for facilitating teachers’ access to relevant research. An extensive programme of staff training, guidelines, rubrics and resource materials and a web-site with access to on-line tool-kits and materials has been developed to ensure implementation of AifL.

Rather than seeing assessment as an activity distinct from teaching, formative assessment is integral to pedagogy and central to learning. Over the course of a period of learning, teachers can draw upon a wide variety of sources of information about the progress of their learners, enabling them to make informed adjustments to their teaching. The sources may include classroom dialogue with learners, observations of individuals working alone or in groups, examples of students’ work, overall test results, responses to individual test questions or peer and self-assessments. In France, teachers draw upon these sources to record their pupils’ development of competences in a booklet that accompanies them throughout compulsory education, as detailed below. Teachers identify relevant information about each of their pupils’ progress for this booklet using the reference grids that detail the learning outcomes for each competence.

A Personal Competences Booklet for each pupil in France

A Personal Competences Booklet (PCB) has been developed at national level and disseminated to teachers throughout the country. The PCB details each competence defined nationally and its fields and items. Teachers note pupils’ progress in acquiring the competences in the PCB, which follows them as they pass through compulsory education (age 6-16). The booklet serves as a communication tool between the teacher and learner, and between the school and parents. It contributes to a national evaluation at the end of secondary education.

In Malta lower-secondary assessment includes information on pupils' attitudes and personal and social skills.

A personal profile is set for each pupil in lower secondary education. Evaluation criteria used in this context relate to active participation in the classroom and the school policy (the ability to discuss, engage in dialogue, debate, organize and work in groups). Marking is based on observations of pupils’ behaviour in the class and school and pupils' involvement in activities/projects in and outside school. Assessments recorded in the personal profile serves the student for identifying his/her needs to develop learning at upper secondary but it does affect the decision on delivering the lower secondary certificate

Compiling assessment information from a variety of sources can be a challenge. E-assessment can make this task easier as it can integrate information from various sources and analyse at least some of this information and then provide feedback[19]. The ICT-based 'Learning analytics' that are currently being developed would not only allow the compilation of information from a variety of sources, but also the formative assessment of this information. It is important, however, to emphasise that the feedback these systems provide needs to be specific enough to help individual learners – and this is not always the case. A good example of feedback that is targeted is the 'Intelligent tutoring systems' or ITSs for mathematics education. These systems adapt the tasks according to the learners' skill levels and provide them instant feedback and targeted support at least as regards procedural understanding.

Alternatively, learners themselves may integrate the various sources by using peer or self-assessment to select content for their portfolios or e-portfolios. Since peer and self-assessment and portfolio assessment can make particular contributions to the development of key competences, they are the focuses for the next two sections.[20] E-assessment is further presented later in the text.

3.1 Peer and self-assessment

In theory, learner self-assessment is an inevitable result of successful formative assessment. To make sense of teachers’ feedback, learners need to reflect on their own learning. Furthermore, through peer assessment, learners can help each other to reflect on their learning. Peer and self-assessment are linked to many key competences, such as initiative and entrepreneurship, learning to learn and social competence, and cross-cutting themes, such as critical thinking, creativity, initiative, problem-solving, risk assessment, decision-taking, and constructive management of feelings.

In practice, both peer and self-assessment are formative assessment interventions that are successful in improving motivation and attainment, even for learners as young as five-years old.[21] Although peer and self-assessment for the formative purpose is generally honest and trustworthy, there is evidence that learners need to develop this competence gradually over time. Consequently, teacher education needs to include training in facilitating peer and self-assessment. This investment in training is worthwhile because the results of effective peer and self-assessment are committed, effective and independent learners. These forms of assessment can also, in the longer-term, reduce the assessment burden on teachers.

Everyone involved in formative assessment, including learners themselves, needs a good understanding of the intended learning outcomes and the criteria by which success will be judged. Where the key competences represent an unfamiliar approach to learning, developing learners’ understanding of the learning outcomes is particularly important. To develop this understanding, teachers can share and discuss the learning outcomes and criteria with their learners. Learners can then, in turn, peer and self-assess themselves against these outcomes using the criteria, give feedback to one another and consider the next steps. As they do so, they become more competent in working alone and with others, developing various key competences in the process.[22] For instance, peer and self-assessment tools are currently being developed for ICT-based 'learning management systems' such as Moodle to enable coherent formative performance. These systems already provide a means of cooperation and provision of feedback from teachers to learners and between peers.[23]

There are several examples of Member States placing an emphasis on learners’ development of self-assessment competence and this is sometimes explicitly linked to learning to learn. In addition, in the Czech Republic, some schools have piloted the use of a self-assessment booklet to record each learner’s development of key competences.

Learners’ formative use of assessment information is promoted in several Members States:

• Estonia’s cross-curricular ‘learning competence’ is partly about learning ‘to monitor and assess one’s thinking and learning activities’. • In Finland, assessment in basic education aims to encourage pupils to develop their self-assessment skills, by reflecting on their development and helping to set targets in their individual study plan. • In Lithuania, learning to learn includes specific learning outcomes relating to ‘reflection on learning activities and outcomes, and self-assessment’. • The Netherlands’ cross-curricular general objectives for secondary education refer to learning to learn, including ‘assessing information on the grounds of reliability, representativeness and usefulness’. • In Greece, self-assessment is intended to enable pupils to acquire meta-cognitive skills through the control and management of their own learning. • Within the UK, the key skills qualification in ‘improvement of own learning and performance’ offered in England, Wales and Northern Ireland includes an element of self-assessment.

A pilot for a self-assessment booklet in the Czech Republic

A process of learner self-assessment of key competences has been piloted in some schools in the Czech Republic. In these schools, each learner has a self-assessment booklet that specifies their goals and details their progress towards these goals. This provides a focus for discussions that include learners, teachers and parents and has helped to embed formative assessment of the competences in the culture of the schools.

3.2 Portfolio and e-portfolio assessment

Where information is collected during a period of learning, performance-based assessment can help learners to develop their key competences. It can do so indirectly, through feedback into teaching and learning but it can also do so directly, through the types of performance it requires from students. For example, learners giving presentations have an opportunity to develop the communication competence necessary for a particular public-speaking context. Learners compiling a portfolio of evidence about their key competences have an opportunity to reflect on their learning process and develop various competences including learning to learn. When they develop this portfolio in cooperation with their teachers and peers, they also have a particular opportunity to develop their social competence. When an e-portfolio is used, they also have an opportunity to develop their digital competence.

Portfolios provide a method of storing a variety of examples demonstrating learners’ development and application of key competences in a range of contexts during the learning process. The examples selected as entries for portfolios can be drawn from contexts that are difficult to reproduce in standardised tests, such as those involving social interactions. Furthermore, portfolios are particularly valuable for identifying the application of combinations of competences required in real-life contexts. When used in connection with explicit guidelines, portfolios can provide a structure for learners to carefully select entries with the support of their teachers. Reflecting on the learning outcomes and the progress towards them is therefore integral to the assessment. Portfolios can then act as a resource that informs discussions not only between teachers and students about next steps in learning, but also parents.

Perhaps for the reasons outlined here, an international trend towards the use of portfolio assessment is apparent. In several cases, this portfolio assessment uses ICT. Such e-portfolio assessment has been implemented in schools in Austria, Belgium, Portugal, Romania, Turkey and the UK, piloted in Bulgaria, France, Greece and Iceland, and planned in several other countries. E-portfolios expand the range of possible formats for entries, so that audio-visual files and internet links can be included. They also provide a flexible format for sharing the content and gaining the feedback of teachers and peers, such as through a school intranet-based social network. This requires ICT infrastructure and some digital competence on the part of learners and teachers but can reduce the administrative burden on teachers. It is worth noting though that this works best when there are also opportunities for face-to-face interaction, and the wider development of social competence. In some cases, portfolio or e-portfolio entries may be accompanied by learners’ narratives about their reflections on and explanations for their choice of entries. Teachers may then gain further insights into their students’ attitudes to learning and their progress towards the learning outcomes.[24]

Several examples below show how (e-)portfolios are contributing not only to formative assessment but to summative assessment as well by building up evidence of competence during a period of learning.

Portfolio assessment of cross-curricular key competences in Austria:

A project in the Austrian province of Vienna is using portfolios to assess the cross-curricular key competences of students in the cohort aged 15 in lower secondary general education schools. Whilst all eight key competences are assessed, the emphasis is on digital competence, learning to learn, social and civic competences, sense of initiative and entrepreneurship, and cultural awareness and expression. Each key competence is divided into indicators and descriptors, which provide a focus for the evidence of students’ competences chosen for inclusion in portfolios. Teachers assess students’ competences in relation to each of the descriptors at four levels: not observed; partial; expectation met; excellent. The portfolios have a formative purpose and a summative purpose. During the period of learning, students receive feedback on their progress in demonstrating each key competence. At the end of the period of learning, students receive an attachment to the lower secondary leaving certificate complementing the subject specific marks with cross-curricular competences.

The learning achievement portfolio of key competences in Slovenia:

Slovenia’s recent reforms to vocational education have led to the development of a learning achievement portfolio for the assessment of the eight key competences. This development was influenced by the European language portfolio. The purpose of the portfolio is formative: to help students set goals, plan their learning and develop their competences. Students therefore use the portfolio throughout their programme of learning. The portfolio is not scored but teachers are obliged to monitor their students’ development of competences through the portfolio. The portfolio includes worksheets about the student goals, his/ lifelong learning, his/her achievements and provides a basis for the collection of information and documents to monitor the progress of students in developing key competences and specific vocational competences.

The experimental competences booklet in France

In France, an Experimental Competences Booklet is being tested in about 140 schools. This booklet takes the form of a national digital portfolio. It has to be gradually built by the pupils, accompanied by their teachers and associations or professional organisations in close collaboration with their parents. It lists the competences developed inside and outside the school and sums up all the diplomas and the certificates which pupils receive during their schooling.

3.3 E-assessment

As with portfolio assessment, the potential of e-assessment relates to both formative and summative purposes of assessment. In theory, most of the assessment methods presented in previous sections – whether related to standardised tests or peer-assessment – could be replaced by ICT-based assessment. However, using e-assessment just for the sake of it does not guarantee improved assessment of key competences. The key to taking up e-assessment is to use it in ways that bring added value by making previously in-assessable learning outcomes assessable.[25]

E-assessment includes a wide variety of assessment methods ranging from simpler forms, such as computer-based testing, to more complex forms of assessment, such as using augmented reality and immersive games in assessment. Even these simpler forms can provide students with instant feedback, adapt the level of difficulty according to their skills and offer guidance in the process to help students develop appropriate solution strategies. At the same time as reducing test times and test fatigue, such adaptive testing can produce more useful information about learners’ key competences. As described in the previous section, e-portfolios allow students to show their progress and can support self- and peer assessment thus encouraging self-regulated learning. Embedding learning and assessment into computer simulations, virtual laboratories and games takes the learning process even further by enabling students to develop their scientific enquiry, analysis, interpretation and reflexion skills in a real-life context.[26].

There are also new, emerging e-assessment methods, such as learning analytics that could become reality within five years. Learning analytics would allow the interpretation of a vast amount of data produced by students in electronic environments, thus enabling the assessment of progress and tailoring of education for the individual students. Moreover, it could allow embedding assessment in computer simulations, virtual worlds and immersive games. Already widely used in the US are the intelligent tutoring systems for mathematics instruction, which are built around a similar idea as learning analytics but also provide instant feedback and targeted support for students.

Hence, as can be noted from the variety of e-assessment methods, the real significance of e-assessment may be in transforming what is actually assessed. By simulating real-life contexts and repeatedly sampling performance, e-assessment opens up new possibilities for eliciting information on learners’ key competences. Rather than explicitly presenting learners with information, simulations can require learners to make sense of the context for themselves.

Simulations can also create dynamic contexts which interact with learners. This means that every action leads to a reaction, requiring repeated re-evaluation of the task conditions, much as in real life. For example, online games can transpose the social and political context of a real-world problem into a virtual model.

Currently, however, these more complex forms of e-assessment are not widely used in Europe and even though there is an abundance of educational simulations available, many of these do not include or embed assessment. The two most commonly used methods of e-assessment are still computer-based tests and e-portfolios. In the first case, however, the added value of ICT is often unutilised and instead traditional tests have essentially been migrated onto screen format.

There are some initiatives in Europe that illustrate, however, that computer-based testing can be used in innovative ways to enhance the assessment of key competences[27], including adaptive tests and the assessment of the digital competence itself:

Adaptive testing of skills in UK Northern Ireland

In the UK(Northern Ireland), computer based diagnostic tests of literacy and numeracy skills, administered each year between the age of 7 and 11 in all schools provide a tool for teachers to adjust instruction to the needs of their students and to recognise and assess learning difficulties. The tests are centrally designed by the Curriculum Authority and are “adaptive” - students who answer a question correctly are given a harder question, while those who answer incorrectly are given an easier question. These assessments, which last approximately 30 minutes each, are computer graded. The data are intended for use within the schools. There is no central collection of data, although schools are required to report the results to parents. The use of these tests has been supported through a considerable financial effort involving school-based seminars and e-learning for most primary teachers and extensive central ICT support and design of the software over more than 6 years. An important element in their implementation was extensive consultation with teachers and teacher unions on their aims and contents, prior to adoption and during implementation[28].

Testing digital competence among Norwegian schoolchildren

The Norwegian Centre for ICT in Education has been piloting tests for digital literacy in Oslo and Bergen, the two largest cities in Norway.

The test in Oslo has been conducted annually since 2008. Today, learners in the 4th grade and in the 12th grade (upper secondary education) take this test every year. The test assesses the following competencies: basic digital skills, digital communication, digital information processing, digital production and critical thinking. The different elements of the test are weighed differently. In Bergen, a similar test is made for 5th graders.

Norway is also developing a diagnostic test in digital skills. This test will be an optional test, and it is based on a new framework developed by the Ministry of Education and Research and the Norwegian Directorate for Education and Training. The framework defines four “digital areas”: To acquire and process digital information, produce and process digital information, critical thinking and digital communication.

The test has been piloted and it aims map the knowledge level, strengths and weakness of students to help plan an effective early intervention. The 4th grade is chosen for this test, because the learners will, according to the curriculum, start using digital tools in the 2nd grade.

The development and availability of test items is governed by a new framework for basic skills in the Norwegian curriculum being implemented in 2012. The test items cover a spectrum of test tasks, however, according to the informant at the National Centre for ICT in Education, there is a lack of test items that require simulations and test items anchored in real world problems and challenges.

As regards basic skills and tackling low-achievement, ICT-based testing can provide an effective diagnostic assessment tool and provide pupils, teachers and parents with important information on progress:

Diagnostic Assessment in Hungary in Mathematics, Reading and Science

The Centre for Research on Learning and Instruction at the University of Szeged in Hungary is developing an online assessment system with substantial funding from The Social Renewal Operational Programme and the European Social Funds. Through this project, the goal is to lay the foundation for a nationwide diagnostic assessment system for the grades 1 through 6. The project will develop an item bank with nine dimensions (including reading, mathematics and science in three domains as well as a number of other minor domains. According to the project management, the project has a high degree of transferability and can be regarded as close to the framework used in the PISA assessment.

The project currently covers both core subjects such as mathematics, science and reading and is also looking at the feasibility of expanding diagnostic assessment to include other cognitive and affective domains.

ICT-based national assessment of key competences in Czech Republic

In the Czech Republic there is an ICT-based national assessment of key competences in 3 different grades every year. The compiled results of this assessment are public, but with a secured password parents can see the individual result of their children as compared to the average, and schools can also obtain (with a password) the results of their classes. This information should be used by the school for further development. Schools, therefore, through their students' performance, get also information about the performance of the teachers.

The examples below show how ICT-based assessment can also be used in support the more transversal key competences such as entrepreneurship, learning to learn and digital competence:

Computer-based test on entrepreneurship in UK Wales

Gower College Swansea is a further education college in the UK, providing upper secondary and vocational education and training to the local community. It has integrated the completion of an e-survey on entrepreneurship into the induction programme for all new full time students starting courses at the college – around 1200 each year. During their first week at college these new students have to complete a number of computer based skills assessments for numeracy and literacy and the college has added in the Enterprise Catalyst (www.enterprisecatalyst.co.uk) as their final task in this process. The catalyst assesses their current levels of entrepreneurial awareness, skills and interest in starting a business, when they start college and before they undertake any enterprise activities or events. They receive a personalised report showing their 'Enterprise Fuel', with information about their strengths and skill areas which they could consider developing further. At the end of their first year the students then complete the catalyst again to see the distance travelled. The college can then pay for a summary report showing overall measures for all students in the college, and this can provide an insight into the average measures of attitudes and skills as well as intention to start a business. In the future this data can be compared each year. This will be expanded to the new Welsh Enterprise Academy launched in the college in September 2012, and there are also plans to expand this methodology to all vocational colleges in the region through the South West Wales Regional Hub for Entrepreneurship Education.

The added value of e-assessment is evident in performance-based assessment. Here, e-assessment can provide different ways of capturing information. For example, when compiling examples of their work in e-portfolios, learners’ can submit information in a variety of formats, including video, audio, pictures and text. This can contribute to developing and applying digital competence specifically and communication competence more generally. The example from England illustrates how this can be put into practice.

e-Portfolio assessment of cultural and artistic expression in England:

In the eSCAPE project, a six-hour collaborative design workshop replaced school examinations in design and technology for students aged 16 in 11 participating schools across England. Design and technology incorporates aspects of the key competence relating to cultural and artistic expression. Students worked individually, but within a group context, to build their design solution. Students were given a number of staged assessment instructions and information via a PDA (Personal Digital Assistant). The handheld device also acted as a tool to capture assessment evidence – via video, camera, voice, sketchpad and keyboard. During the six hours, each student developed their design prototype and the PDA provided a record of their progress, interactions and self-reflections. At the end of the assessment, evidence was collated in a short multimedia portfolio loaded onto a secure website. The project resulted in 250 e-portfolios and the reliability of the assessment method was reported as very high.

An initiative in UK-Wales shows how web-based assessment can be applied to 'soft skills' such as confidence, self-esteem, motivation, ability to co-operate, self-discipline and wellbeing.

E-assessment of 'soft skills' in UK Wales

The Welsh Assembly has funded the development of a web-based tool which providers can use to record learners' 'soft skills' outcomes in a systematic way[29]. The tool is split into two sections: I: Learners self-assess themselves against a series of statements relating to the following themes at the start, the mid-point and the end of their learning: Attendance, Timekeeping, Relationships, Participation, Self-esteem, Working with others, Beliefs and Responsibilities. II: Once the learner completes each session, they are informed to arrange a planning session with their tutor. The tutor then completes an assessment of the learner against a series of statements relating to the same themes at the same points in learning.

Furthermore, learning software with integrated assessment and feedback loops is starting to be exploited as a means of fostering the development and assessment of key competences, as illustrated in the following case that is developed in Norway and that could give an example for further developments of such software.[30]

The Norwegian Kikora Software for Mathematical Competence

Kikora is both a Norwegian software company as well as a piece of educational software for mathematics. The development of the software was facilitated through a development grant from the Norwegian Ministry of Education and Research in 2006-2007.

The core of the Kikora software is that it contains a feedback function that gives the learner line by line feedback on all calculations. The software also contains a reporting tool that enables the teachers to track the progress and critical learning areas of the students, which in turn facilitates mapping and planning of one-to-one activities based on the individual student profiles. The program also contains an item bank with approximately 3000 tasks for students in lower and upper secondary education. The selection of the items pays attention to the need for items that will support