Education and Training (formerly ‘Learner Responsive’)\
National Success Rates Tables for the academic year 2010/11

Guidance Notes

What is new in the 2010/11 Tables?

- The ‘Headline’ worksheet in the ‘Institutions’ table has been enhanced to include the ‘Length’ dimension. This restores the ability of users to analyse success rates for a provider by ‘Long’ qualification length.
- The Education and Training National Success Rates Tables were revised on 9th August 2012 to correct an error in unknown levels mainly in the ‘Other’ qualification category. Further detail of this revision is available in the revision paper available at the National Success Rates Tables pages of the Data Service website. Success Rates information available elsewhere is not affected. This includes the Statistical First Release, FE Choices and the individual Qualification Success Rate and Minimum Levels of Performance reports that providers receive on their own data.
- The Tables have been renamed the ‘Education and Training’ National Success Rates Tables to reflect the terminology of the Statistical First Release, from which they are a cascade publication.
- Success rates data for 2010/11 have been added to the tables, and those for 2007/08 removed.
- The definitions of A, AS and A2 Level qualifications have been brought into line with those used in the Learner Responsive 2010/11 Qualifications Success Rates (QSR) reports for individual providers. These definitions have been applied to the 2008/09 and 2009/10 data in the tables in order to provide a consistent view of these qualifications.
- The Qualifications tables have been re-organised to arrange qualification types according to the classification used in the Learner Responsive 2010/11 individual provider QSR reports. The first of the reports now shows A, AS, A2, GCSE, NVQ, BTEC/OCR and International Baccalaureate qualifications; the second shows all other Long and Short qualifications (including 14-19 Diplomas). For ease of use, the ‘Other’ qualifications are arranged in 17 worksheets according to their Sector Subject Area (Tier 1) classification.
- Consistent with the Learner Responsive 2010/11 QSR reports for individual providers the tables include data from the University for Industry (Ufi) for the first time.
- The now obsolete ‘Local Office’ dimension has been removed from the Institutions tables. A report showing success, retention and achievement rates by the Local Authority areas in which an institution delivers provision has been added.
- The UKPRN identifier has been added to the Institutions tables.
- The percentiles worksheets for Learner Home Postcode and Delivery Location Postcode have been removed from the ‘Overall’ report.
- To prevent the possible identification of individual learners the reports do not show any qualifications where the number of starters (including transfers) for a year is less than 30. In addition, qualifications where starters exist only at a single institution for a given year are not shown in the Qualification Aims reports. Qualifications that have been removed from reports for these reasons are, for the first time, listed in a supplementary report, ‘Excluded Aims 2010/11’. Counts such as Starters are rounded to the nearest ten. Any value of less than 5 is shown as a dash (‘-’). Values in the range 5 – 9 are rounded to 10.

Various minor improvements have been made to the presentation standards of the tables, with the aim of making them more consistent and intuitive.

Success, Retention and Achievement Rates in Further Education Providers in England

The National Success Rates Tables for Education and Training (formerly ‘Learner Responsive’) provision set out levels of success, retention and achievement in further education (FE) providers in England for the period 2008/09 to 2010/11. The Data Service publishes National Success Rates Tables annually. This release of the Education and Training National Success Rates Tables updates the 2007/08 to 2009/10 Learner Responsive tables published in April 2011. It is consistent with the Learner Responsive Qualification Success Rates reports for 2010/11 published to individual providers in December 2011 and expands upon the information made available in the January 2012 Statistical First Release.

The purpose of the National Success Rates Tables

The publication of Education and Training National Success Rates Tables supports providers in raising the standard of their delivery. The Tables allow providers to assess their performance, and assists the planning of action programmes to improve the success, retention and achievement rates of their learners.

The term National Success Rates Tables, rather than 'Benchmarking Data', is used throughout this guidance. National Success Rates Tables is used to denote a context and reference point for comparison, not a standard of best practice.

What the tables show

The Data Service approach to publishing the Education and Training National Success Rates Tables is to publish a manageable amount of information, drawing on existing statistical measures.

The Education and Training National Success Rates Tables data have been derived from providers' Individualised Learner Record (ILR) returns and furnish a range of national and provider statistics for success, retention and achievement. Information on the qualification success rates methodology is available at the Success Rates pages of the Data Service website.

This year four categories of Education and Training National Success Rates Tables Data are presented:

- Overall data tables showing national averages of success, retention and achievement rates, plus spread of variability, by institution type, level, age, gender, disability, ethnicity, length of aim and geography.
- National data for success rates, retention and achievement by sector subject area.
- Individual learning aim data showing success, retention and achievement by level and qualification type.
- Institution tables showing success, retention and achievement by provider, qualification type, age, gender, disability, ethnicity and subject area. Information contained in the Tables

There is one success rate, one retention rate and one achievement rate shown in this publication. The definitions of the success, retention and achievement rates can be found in the ‘How the Tables are calculated’ section below.

Although it may appear in reports produced by other agencies that consume or receive master data files from the Data Service, a success rate based on the ‘High Grades’ measure is not part of the National Success Rates Tables. The old ‘High Grades’ measure, which no longer had an agreed definition, was removed in the 2009/10 Tables. The ‘High Grades’ definition that was agreed by Data Harmonisation Group in February 2011 was not subsequently included in the 2010/11 Learner Responsive QSRs for individual providers, and therefore does not appear in the 2010/11 Education & Training National Success Rates Tables.

Short learning aims, where the learner expects to complete in fewer than 24 weeks, are distinguished from longer learning aims. The tables also show national success rates for short learning aims of less than five weeks expected duration and short learning aims of between 5 and 24 weeks expected duration.

Exclusions

Key Skills learning aims have not been included in the Education and Training National Success Rates Tables owing to inconsistencies in the way that they were recorded on the ILR. Functional Skills learning aims and aims delivered by the Offender Learning and Skills Service (OLASS) are also not included. This is consistent with the approach taken in Learner Responsive Qualification Success Rates reporting to individual providers.

How to use the Tables

- They allow providers to assess their performance and assist their planning of action to improve the success, retention and achievement rates of their learners.
- They support providers in the process of setting targets as part of their development plans.
- They inform the process of performance review by the Skills Funding Agency and by Local Authorities.

Who uses the Tables

Providers can contextualise their performance by comparing their results with the published Education and Training National Success Rates Tables. This information will support the process of setting targets for 2012/13 and beyond. Individual provider results will also be used by the Skills Funding Agency and by Local Authorities to compare and evaluate the performance of providers.

Where a provider’s performance differs from the Education and Training National Success Rates Tables data, the provider may wish to explore the reasons for this by calculating success, retention and achievement rates for particular parts of its provision and comparing these with statistics for similar provision at national level. To facilitate this comparison, a more detailed breakdown of the Education and Training National Success Rates Tables data by type of qualification and individual learning aim is also available.

In some cases there may be a difference between provider’s own statistics and the Education and Training National Success Rates Tables data because the mix of learning aims at the provider differs significantly from the national mix. This means that for some comparison purposes the National Success Rates Tables data may have to be adjusted. An example of how to adjust the data is shown in the definitions section. Viewing the Tables The data for 2010/11 can be accessed at the Success Rates pages of the Data Service website. The tables are presented as a set of spreadsheets that can be downloaded by users for further analysis.

HOW THE TABLES ARE CALCULATED Summary of calculation method The method takes data for each institution from the following ILR returns to create the Education and Training National Success Rates Tables:

- F05 ILR (2007; 2006/07)
- F05 ILR (2008; 2007/08)
- F05 ILR (2009; 2008/09)
- F05 ILR (2010; 2009/10)
- L05 ILR (2011; 2010/11)

F05 2006/07 is used to provide information on learners starting learning aims that were expected to end in 2008/09 or later, i.e. courses of three years or more duration. For example, learners on a three-year GNVQ programme starting in 2006/07 and expecting to end in 2008/09.

Learners and their learning aims are matched across the five years of ILR returns using the learner reference, course code, start date and expected end date to calculate the number of starters at the beginning of each programme, retention across the whole programme, and success and achievement levels. Only aims that learners expected to complete between 2008/09 and 2010/11 are included in the publication.

The Education and Training National Success Rates Tables data are built from cohort level, a cohort being learners studying a particular learning aim over the same duration and expecting to end in the same academic year.

Presentation standards

- Where the overall number of starters for an aim is less than 30 or the aim is delivered at only one institution, results and percentiles are not shown on the qualification level reports. A list of the aims excluded by these rules is published separately in the ‘Excluded Aims’ report.
- The percentage breakdown of numbers started may not add up to 100% due to rounding.
- On all reports numbers are rounded to the nearest 10. Values less than 5 have been replaced with a ‘-‘, and values in the range 5 – 9 are rounded up to ‘10’.
- Since the data is calculated at qualification level, learners studying more than one qualification will appear once for each qualification.

GCSE grades The variable quality of data in the ‘grade’ field of the ILR means it is not yet possible to accurately differentiate the ranges of GCSE grades. In the Education and Training National Success Rates Tables the number of GCSEs achieved is calculated as the number of GCSEs achieved at grades A\* to G for all providers.

The Effect of Widely Differing Mixes of Qualification Type The example below shows how the National Success Rates Tables can be adjusted to match the mix of learning aims at an individual college, where the mix of aims at the college is significantly different from the national mix. The example relates to adult learners studying at Level 3 in a general further education college. The starting point is the average retention and achievement rates for different level 3 qualifications for all general further education colleges and for our example college.

Table 1. Level 3 qualifications, learners aged 19 and over

| | Average for GFEC / TC | Example College | |----------------------|-----------------------|-----------------| | | Retention Rate % | Achievement Rate % | Breakdown of enrolments % | Breakdown of enrolments % | | All Qualifications | 78 | 72 | - | - | | GCE A2 | 67 | 64 | 18 | 10 | | BTEC | 68 | 77 | 9 | 5 | | NVQ | 82 | 73 | 17 | 50 | | Other long | 83 | 64 | 56 | 35 |

In our example college, the level 3 provision for adults includes a higher proportion of NVQs, and a lower proportion of GCE A2 levels, BTEC/OCRs and Other learning aims than the national mix. In order to check whether the published National Success Rates Tables data for all level 3 aims is an appropriate comparator, the following calculation may be used:

Average GFEC/TC retention rate = 78% GFEC/TC retention rate for mix of aims in example college comprising: 10% GCE A2 retention rate of 67% = 6.7 5% BTEC retention rate of 68% = 3.4 50% NVQ retention rate of 82% = 41 35% Other long retention rate of 83% = 29.05 80.15 rounds to 80%

The same approach can be used for achievement rates:

Average GFEC/TC achievement rate = 72% GFEC/TC achievement rate for mix of aims in example college comprising: 10% GCE A2 achievement rate of 64% = 6.4 5% BTEC achievement rate of 77% = 3.85 50% NVQ achievement rate of 67% = 33.5 35% Other long achievement rate of 75% = 26.25 70.00 rounds to 70%

As can be seen, the adjusted National Success Rates Tables data are similar to the original National Success Rates Tables data, despite the very different mix of aims in the example college. In practice, mix of qualification types is unlikely to be a significant factor for most colleges. This method can be used on success rates and on achievement and retention rates.

Age group A learner’s age group is calculated from their age as at 31 August in the year they started their learning aim. Learners of unknown age are included in the age group 19 and over. Learners under 16 years are included in the 16-18 age group. All tables show the National Success Rates Tables data divided into three age groups: ‘16-18’, ‘19 and over’, and ‘All Ages’.

Short learning aims A learning aim is ‘short’ if it has an expected length of less than 24 weeks. In practice the majority of short aims are of 12 weeks duration or less. Short aims (excluding short Key Skills learning aims) are shown separately, since the success, retention and achievement rates for these aims are significantly different from those of longer aims. In addition, the tables show National Success Rates Tables data for short aims split into two subsets: less than five weeks expected duration and between 5 and 24 weeks duration.

**Learning aim levels**

Learning Aims are grouped according to their qualification level according to the categorisation of each aim on the Data Service’s learning aims database. The levels are:

- **level 1** includes qualifications at level 1 and level ‘E’ (entry level), such as NVQs and other foundation or pre-foundation qualifications
- **level 2** includes level 2 NVQs, intermediate NVQs and precursors (BTEC first certificate or first diploma, City and Guilds Diploma of Vocational Education at intermediate level), GCSEs and other intermediate level qualifications
- **level 3** includes level 3 NVQs, advanced NVQs and precursors (BTEC national certificate or national diploma, City and Guilds Diploma of Vocational Education at national level), advanced VCEs, GCE A, A2 and AS levels and other advanced level qualifications
- **level H** all level 4 and 5 qualifications including HNCs, HNDs, access to HE qualifications, NVQs at levels 4 and 5, and other higher level professional qualifications.
- **Level X** Qualifications with unknown or unspecified level

**Qualification codes**

The qualification codes shown in the qualification level data tables can be mapped to the qualification codes used in ILR L05 (from the Learning Aims Database (LAD)) using the hierarchy table that can be found at: [http://providers.lsc.gov.uk/LAD/downloads/MiscDownloads.asp](http://providers.lsc.gov.uk/LAD/downloads/MiscDownloads.asp) (see table labelled Mapcodes).

In this hierarchy table the Mapcode column corresponds with the qualification code shown in the qualification data tables of the Education and Training National Success Rates Tables.

**Number of starters**

The ‘number of starters’ is the number of enrolments on learning aims where the learner was expecting to complete the aims in that academic year. Details to note on the definition include:

(a) the number started excludes any learner who transferred onto another learning aim at the same institution. The learning aim the learner transfers into will be included as a start on the new aims. (b) learners who start on a learning aim expected to last at least 168 days and withdraw within 42 days of starting, qualify for an SLN payment yet have received no payment, are considered as not funded and are excluded from the count of starters. (c) each learning aim a learner is enrolled on is shown as a separate ‘start’. (d) the ‘number of starters’ for 2008/09 may include some provision not funded by the then Learning & Skills Council (LSC). (e) A cohort of learners on a two year programme who began their studies in October 2008 would appear in the results for 2009/10 as this is the academic year in which they expected to complete their learning aim, even if they withdrew before August 2009.

**Success rates**

Success rates are calculated as the number of learning aims achieved divided by the number of starters who do not transfer out (i.e. to another institution). It can also be derived by multiplying achievement rate by retention rate. Retention rate The retention rate is the number of learning aims completed, divided by the number of learners who started the aim, excluding transfers out. For programmes of study of two years or more, retention is calculated across the whole programme, i.e. from the start to the end of the aim.

Achievement rate The achievement rate is the number of learning aims fully achieved divided by the number of completed aims. This denominator includes those completers recorded with unknown outcomes in the ILR, for example outcome code 4 ‘exam taken but result not known’ or outcome code 5 ‘learning activities are complete but exam has not yet been taken’. Partial achievements are not included as achievements.

Measures of provider variability Measures of provider variability (percentiles) for success, retention and achievement rates enable institutions to compare their results against the range for the sector or particular groups of institutions.

The results in this publication show the rates where:

- providers with a success rate equal to or greater than the 90th percentile are in the top 10% of providers
- providers with a success rate equal to or greater than the 75th percentile are in the top 25% of providers
- providers with a success rate equal to or greater than the 50th percentile are in the top 50% of providers
- providers with a success rate less than or equal to the 25th percentile are in the bottom 25% of providers
- providers with a success rate less than or equal to the 10th percentile are in the bottom 10% of providers

This is illustrated by the following table. A provider with a success rate of 45% would be in the bottom 10% of providers, whereas a provider with a success rate of 82 would be in the top 10% of providers.

| Percentile Position | Overall Success Rate % | |---------------------|------------------------| | 10 | 46.5 | | 25 | 59.4 | | 50 | 68.6 | | 75 | 74.5 | | 90 | 81.3 |

An alternative way of looking at this for provider types is that:

- the top 10% of providers in a provider type category have a success rate equal to or greater than the 90th percentile
- the top 25% of providers in a provider type category have a success rate equal to or greater than the 75th percentile
- the top 50% of providers in a provider type category have a success rate equal to or greater than the 50th percentile • the bottom 25% of providers in a provider type category have a success rate less than or equal to the 25th percentile • the bottom 10% of providers in a provider type category have a success rate less than or equal to the 10th percentile

Measures of variability are calculated at provider level in order to provide information on variation between providers. This is different from the success, retention and achievement rates, which are calculated as the average rate for all the relevant enrolments, weighting each enrolment equally.

The measures of variability weight each provider equally. This means that learners in smaller providers have a greater bearing on results than those from larger providers. The results for small groupings, such as learning aims being studied by adults in sixth forms, will be affected by this weighting more than others.

It should be noted that in some cases the percentile tables may present data based upon a very small number of providers. In these cases the equal weighting of each provider may have a significant bearing on the variability of the measures presented.

The differences between the two methods can be seen by comparing the average or 'mean' rate, with 50th percentile or 'median'. In many cases the difference is slight, whereas in others such as Level 1 learning aims in sixth forms, the difference is greater. In this case the median is higher than the mean due to a number of providers with relatively few learners having high levels of retention and achievement. These providers are given the same weight in the calculation of the median as other sixth forms with a larger number of learners thereby increasing the median provider result. Care should therefore be exercised in the use of this percentile data to support business decisions.

Both the mean success, retention and achievement rate and the measures of variability are valid and useful measures, depending on whether the overall performance of the sector is of interest (mean success, retention and achievement rates) or the variability between providers is the focus (measures of variability).

How do I find out more? If you would like any more information about the Education and Training National Success Rates Tables please contact our Service Desk on 0870 267 0001 or email servicedesk@thedataservice.org.uk.

Please be aware that the Data Service is responsible for producing NSRT reports in accordance with the Learner Responsive Qualifications Success Rates methodology approved by the then Young People's Learning Agency (now the Education Funding Agency) & the Skills Funding Agency. The methodology was produced by the Strategy and Implementation Group of the then Young People's Learning Agency (now EFA) following the principles set out by the Data Harmonisation Group. See: http://www.theia.org.uk/harmonisingsuccessrates/dataharmonisationgroup/ The methodology is available at the Success Rate pages of the Data Service website.

Providers will find answers to most of their queries regarding the Education and Training National Success Rates Tables in the guidance document provided or in the frequently asked questions at the National Success Rates Tables page on the Data Service website. It is strongly recommended that providers refer to these sources before contacting the Data Service. If providers need further help, the Data Service will be pleased to receive queries regarding the Education and Training National Success Rates tables. However, when logging a query with the Data Service please provide a detailed explanation of the query being raised. Please also note that it may take longer to resolve queries in this way as they may need to be reviewed by different members of the Data Harmonisation Group.