GUIDANCE

Guidance for Heads of Centre, Heads of Department and teachers on objectivity in grading and ranking Guidance for Heads of Centre, Heads of Department and teachers on objectivity in grading and ranking

Contents

The importance of objectivity ................................................................. 3 Objectivity in grading and ranking decisions ........................................ 3 Unconscious effects on objectivity ......................................................... 3 Information from previous data ............................................................ 4 Reviewing judgements ........................................................................ 4 The importance of objectivity

In these unprecedented circumstances, schools and colleges are best placed to judge the likely performance of their students if teaching and learning, and exams, had continued as planned. Centres know their students well and will have regularly assessed their performance throughout the course of study.

We are conscious that these arrangements have had to be put in place very quickly due to the Coronavirus (COVID-19) pandemic, so it has not been possible to provide national training to school and college staff to standardise these judgements.

We are providing the following additional information on objectivity in grading and ranking to help schools and colleges play their role in ensuring this year’s grading is as fair as possible. This is based on existing research and analysis about how centre-based assessments can be carried out as objectively as possible.

Objectivity in grading and ranking decisions

Each centre assessment grade should be a holistic professional judgement, balancing different sources of evidence and data. It is important that the centre’s grading and ranking judgements are objective; they should only take account of existing records and available evidence of a student’s knowledge, skills and abilities in relation to the subject. This evidence should inform teachers’ professional judgements about each student’s likely performance at the time of the exam. Other factors should not affect this judgement, including characteristics protected under equalities legislation such as a student’s sex, race, religion/belief, disability status, gender reassignment or sexual orientation. Similarly, judgements should not be affected by a student’s behaviour (both good and poor), character, appearance or social background, or the performance of their siblings.

Unconscious effects on objectivity

To avoid unconscious bias, centres are urged to reflect on and question whether they may have any preconceptions about each student’s performance and whether their perception of the evidence might be affected by any irrelevant factors.

Centres should be aware of:

- confirmation bias, for example noticing only evidence about a student that fits with pre-existing views about them
- masking or halo effects, for example a particular view about an aspect of a student that hides, or overly accentuates, their actual knowledge, skills and abilities
- recency effects, for example giving undue weight to the most recent interaction with a student or the most recent piece of work done by a student
- primacy effects, for example giving undue weight to ‘first impressions’ of a student
- selective perceptions, for example giving undue weight to a student’s performance on a particular part of the content of the specification rather than considering performance across all the material • contrast effects, for example over- or under-estimating a student’s likely performance having first considered a large number of students who are all working at a different standard

Information from previous data

The effects described above may not be consistently seen across different centres or individuals. To understand more about possible effects in a particular centre, a centre could look back at previous years’ data, for example, over the past 2 to 5 years, where this is available. Considering data in this way is unlikely to identify all possible effects and may prove inconclusive. Contextual information is likely be important in considering what weight to give any such data. For example, significant personnel changes may mean that effects in previous years may not be assumed to carry forward, or may reduce the benefits of aggregating data between different years.

A centre could use such data to identify whether there may be any indications of systematic under- or over-prediction for different groups of students, for example, those with particular protected characteristics. For example, a centre may find that it has routinely under-estimated predicted A level maths grades to UCAS compared to grades actually achieved for students with particular characteristics; or routinely over-estimated target English GCSE grades compared to grades actually achieved for students with particular characteristics. The centre could use any such findings as it checks whether its proposed centre assessment grades for this summer might have been influenced by preconceptions or irrelevant factors.

In doing any such analysis, centres should be aware of and take into account contextual factors. Awareness of the limitations of data and the context in which it was generated may help centres to consider which data is relevant, which is not, and what conclusions may and may not be supported.

Reviewing judgements

Having considered possible unconscious effects on objectivity and any information from available data from previous years, centres are asked to use this information to reflect carefully on their grading and ranking judgements. Dialogue between heads of departments, teachers and the Head of Centre can support such reflection and review.

Where any possible unconscious effects, or previous systemic under- or over-prediction for particular groups, have been identified, careful consideration would be needed to ensure, for example, that this was not over-compensated for.

Nonetheless, analysing information, reflection and dialogue as outlined above could help a centre to assure itself that it has effectively fulfilled its duties to promote equality and avoid discrimination as set out under the Equality Act 2010, and to assure itself that it has maximised objectivity and fairness in the judgements that it has made.