Education and Training National Success Rates Tables for the Academic Year 2011/12

Guidance Notes

1. What is new in the 2011/12 Tables?

The changes to the Education & Training National Success Rate Tables from 2010/11 are:

a) Data Changes

Success rates data for 2011/12 have been added to the tables, and data for 2008/09 has been removed.

b) Presentational Changes

i) The names of the provider groups have been changed to use the wording used in the Statistical First Release. The column name has been changed to ‘Institution Type’.

ii) Except on the Institution worksheets the first three columns on all worksheets have been changed to be Institution Type, Year and Age.

iii) In the ‘NSRT Overall’ spreadsheet the ‘Headline Age Level - Short’ worksheets, showing the breakdown of short qualifications, have been removed, and the short qualification breakdown has been included in the ‘Age Length Level’ worksheets.

c) Removal of Columns

A number of columns have been removed from the tables as the data in them is either shown elsewhere or is little used.

i) ‘Headline Age Level SR’ worksheet in the ‘NSRT Overall’ spreadsheet.

Columns ‘A Level’, ‘A2 Level’, ‘AS Level’, ‘BTEC/OCR’, IB, GCSE, NVQ and OTHER have been removed. The values in these columns are present on the ‘Qualification Type SR’ worksheet.

ii) ‘A, AS Levels, GCSEs, IBs’, BTEC OCR’ and ‘NVQ’ worksheets in the ‘NSRT A AS NVQ BTEC IB GCSE’ spreadsheet.

Columns ‘Starts Including Transfers’, ‘No. of Transfers Out’, ‘No. of Continuing’, ‘No. of Completers’, ‘No. of Retained (inc Continuing)’, ‘No. of Achievers’, ‘Known Outcome’, ‘No. of Exams Taken Awaiting Result’, ‘No. of completed aims where exam result is not yet known’ have been removed.

iii) All worksheets in the ‘NSRT Other Qualifications’ spreadsheet. Columns ‘Starts Including Transfers’, ‘No. of Transfers Out’, ‘No. of Completers’, ‘No. of Retained (inc Continuing)’, ‘No. of Achievers’, ‘Known Outcome’, ‘No. of Exams Taken Awaiting Result’, ‘No. of completed aims where exam result is not yet known’ have been removed.

d) Display of Gender, Ethnicity and Learning Difficulty / Disability in the ‘NSRT Overall’ spreadsheet

The information for Gender, Ethnicity and Learning Difficulty / Disability has been split into individual worksheets, to match the method of presentation on the Institutions worksheets.

e) Removal of Duration and Replacement by Length

Where worksheets contained both Duration and Length, the Duration column has been removed and additional aggregations have been added to the Length field. Where worksheets only contained Duration this has been replaced by Length.

f) Suppression Rules

The data suppression rules have been changed.

- Cohorts with fewer than 5 leavers are not output.
- Cohorts where the number of leavers is 5 or more but less than 30 have the number of leavers replaced with a dash (-).

On the percentiles worksheets:

- If there are 20 or more providers the 10th, 25th, median (50th), 75th and 90th percentiles are output.
- If there are between 11 and 19 providers only the median (50th) percentile is output.
- If there are 10 or fewer providers no percentiles are output.

2. The Purpose of the Education and Training National Success Rates Tables

The National Success Rates Tables for Education and Training provision set out levels of success, retention and achievement in further education (FE) providers in England for the period 2009/10 to 2011/12. The Data Service publishes National Success Rates Tables annually. This release of the Education and Training National Success Rates Tables updates the 2008/09 to 2010/11 Education and Training tables published in August 2012. It is consistent with the Classroom Based Learning Qualification Success Rates reports for 2011/12 published to individual providers in January 2012 and expands upon the information made available in the FE Data Library.

The publication of the Education and Training National Success Rates Tables supports providers in raising the standard of their delivery. The Tables allow providers to assess their performance, and assists the planning of action programmes to improve the success, retention and achievement rates of their learners.

The term National Success Rates Tables, rather than 'Benchmarking Data', is used throughout this guidance. National Success Rates Tables is used to denote a context and reference point for comparison, not a standard of best practice. 3) Contents of the Education and Training National Success Rates Tables

The Data Service’s approach to publishing the Education and Training National Success Rates Tables is to publish a manageable amount of information, drawing on existing statistical measures.

The Education and Training National Success Rates Tables data have been derived from providers’ Individualised Learner Record (ILR) returns and furnish a range of national and provider statistics for success, retention and achievement. Information on the qualification success rates methodology is available at the Success Rates pages of the Data Service website.

Since the data is calculated at qualification level, learners studying more than one qualification will appear once for each qualification.

4. Format of the Education and Training National Success Rates Tables

The Education and Training National Success Rate Tables are published as six zip files. Each zip file holds a Microsoft Excel Spreadsheet, which contains a number of worksheets. The contents of each Excel spreadsheet are listed below.

a) Overall Report

The overall report contains worksheets that show headline national success, achievement and retention rates, and national success, achievement and retention rates by qualification level, qualification type, gender, learner difficulty / disability, ethnicity, the learner’s local authority (district) (based on the learner’s home postcode) and the delivery local authority (district) (based on the institution's delivery location postcode).

The overall report also contains worksheets that show percentile information for the headline national success, achievement and retention rates, and for qualification level, qualification type, ethnicity, gender and learner difficulty / disability.

b) Sector Subject Area Report

The sector subject area report contains worksheets that show the national success, achievement and retention rates for qualification type, qualification level and length by sector subject area tier 1.

c) Institution Report

The institution report contains worksheets that show the success, achievement and retention rates for institutions by qualification level, gender, ethnicity, learner difficulty / disability, sector subject area tier 1 and delivery local authority (district).

d) Qualification Type Reports

The qualification type reports contain worksheets that show the national success, achievement and retention rate of individual qualifications by length. For each qualification the qualification code, qualification type, qualification level, qualification title and sector subject area tier 1 are shown. e) Excluded Qualifications Report

Due to the data exclusion rules described in section 6 the full coverage of qualifications delivered is not available in the qualification type spreadsheets. The excluded rows are included in the excluded qualification worksheet to allow the full scope of qualification delivery to be seen.

5. Data Exclusions

The same types of aims are excluded from the Education and Training National Success Rates Tables as are excluded from the Classroom Based Learning Qualification Success Rates reports sent to individual providers. The main exclusions are Key Skills, Functional Skills and Offender Learning and Skills Service (OLASS) provision. The full list of exclusions is described in the Classroom Based Learning business rules, which are available from the Data Service website.

6. Exclusion of Small Cohorts, Data Suppression and Rounding

On all worksheets cohorts with fewer than 5 starts are not shown. Cohorts with between 5 and 29 starts have had the starts value replaced with a dash (-). If the number of starts is 5 or more, but the number of completers is less than 5 the Achievement Rate has been replaced with a dash (-). All numbers have been rounded to the nearest 10. This prevents the potential identification of individual learners or individual learners' success.

Due to data quality issues some figures cannot be shown and have been replaced by an asterisk (\*).

On the percentiles worksheets:

- If there are 20 or more providers the 10th, 25th, median (50th), 75th and 90th percentiles are output.
- If there are between 11 and 19 providers only the median (50th) percentile is output
- If there are 10 or fewer providers no percentiles are output

Where individual qualification success rates are reported, if a qualification is only delivered at a single institution, the qualification is not included in the report. Any rows that are excluded are listed in the “Excluded Qualifications” report, together with the reason why they were excluded.

Because of the exclusion of rows on the qualification worksheets the total of the starters for the individual institution types may be less than the number of starters for the ‘All Institution Type’ row. The excluded institution type rows are show in the Excluded Qualifications spreadsheet, together with the reason for exclusion.

7. Location of the Tables

The data for 2011/12 can be accessed from the Success Rates pages of the Data Service website. 8) Selecting Data on the National Success Rates Tables

The success, achievement and retention rate data is presented as a series of rows in the Excel worksheets. Each row shows the success, achievement and retention rate of a specific cohort of learners. Summary information is shown at many levels, e.g. all ages, all lengths, all institution types. The worksheets contain column filters that are used to select the information required.

The screenshot below shows an example of how the data will appear in the Age, Length and Level success rate worksheet. This illustrates different levels of aggregation.

The column filters are used to select the required level of information. The screen shot below shows the effect of using the filters to select success rate data broken down by age for all lengths and all qualification levels for 2009/10. 9) Worksheet Contents

Information about selected columns in the worksheets is shown below.

a) Institution Type

Each provider is classified into one of 6 institution types. The institution types reported on are:

- General FE and Tertiary College
- Other Public Funded
- Private Sector Public Funded
- Schools
- Sixth Form College
- Specialist College

The Specialist College category include Agriculture & Horticulture colleges and Art, Design & Performing Arts colleges

An ‘All Institution Type’ category is also available on some of the worksheets.

b) Age

The age group of a cohort of learners is calculated from their age as at 31 August in the year they started their learning aim. Learners of unknown age are included in the age group 19 and over. Learners under 16 years are included in the 16-18 age group. All tables show the National Success Rates Tables data divided into three age groups: ‘16-18’, ‘19 and over’, and ‘All Ages’.

c) Length

Each aim is classified into one of three lengths

- Long The expected length is equal to or greater than 168 days (24 weeks)
- Short The expected length is greater than 34 days and less than 168 days (between 5 to 24 weeks)
- Very Short The expected length is less than or equal to 34 days (less than 5 weeks)

An ‘All Short’ category and an ‘All Length’ category are also available

d) Qualification Level

Learning Aims are grouped according to their qualification level recorded in the Data Service’s Learning Aims Reference Application. The levels are:

- level 1- includes qualifications at level 1 and level 'E' (entry level), such as NVQs and other foundation or pre-foundation qualifications
- level 2 - includes level 2 NVQs, intermediate NVQs and precursors (BTEC first certificate or first diploma, City and Guilds Diploma of Vocational Education at intermediate level), GCSEs and other intermediate level qualifications • level 3 - includes level 3 NVQs, advanced NVQs and precursors (BTEC national certificate or national diploma, City and Guilds Diploma of Vocational Education at national level), advanced VCEs, GCE A, A2 and AS levels and other advanced level qualifications • level H - all level 4 and 5 qualifications including HNCs and HNDs • Unknown - Qualifications with unknown or unspecified level

An ‘All Levels’ category is also available

e) Qualification Types

Learning Aims are grouped according to their qualification type. The types are:

• A Level • A2 Level • AS Level • BTEC / OCR • GCSE • International Baccalaureate • NVQ • Other

An ‘All Types’ category is also available

f) Qualification Codes

The qualification codes shown in the qualification level data tables are obtained from Learning Aim Reference Application (LARA) hierarchy file, which can be found on the Data Service website. If a qualification has a Map Code in the LARA hierarchy file this is used on the report, otherwise the Learning Aim Reference is used

g) Number of Starters (Excluding Transfers)

The number of ‘Starters Excluding Transfers’ is the number of enrolments on learning aims where the learner was expecting to complete the aims in that academic year. Details to note on the definition include:

i) The number of starts excludes any learner who transferred onto another learning aim at the same institution. The learning aim the learner transfers onto will be included as a start on the new aim.

ii) Learners who start on a learning aim expected to last at least 168 days and withdraw within 42 days of starting, qualify for an SLN payment yet have received no payment, are considered as not funded and are excluded from the count of starters.

iii) Each learning aim a learner is enrolled on is shown as a separate ‘start’.

iv) A cohort of learners on a two year programme who began their studies in October 2009 would appear in the results for 2010/11 as this is the academic year in which they expected to complete their learning aim, even if they withdrew before August 2010. h) Success Rates

Success rates are calculated as the number of learning aims achieved divided by the number of starters, excluding internal transfers within a provider. It can also be derived by multiplying achievement rate by retention rate.

i) Retention Rate

The retention rate is the number of learning aims completed, divided by the number of learners who started the aim, excluding internal transfers within a provider. For programmes of study of two years or more, retention is calculated across the whole programme, i.e. from the start to the end of the aim.

j) Achievement Rate

The achievement rate is the number of learning aims fully achieved divided by the number of completed aims. This denominator includes those completers recorded with unknown outcomes in the ILR, for example outcome code 4 ‘exam taken but result not known’ or outcome code 5 ‘learning activities are complete but exam has not yet been taken’. Partial achievements are not included as achievements.

10. Percentiles

Percentiles for success, retention and achievement rates enable institutions to compare their results against the range for the sector or particular groups of institutions.

One way of interpreting percentiles is:

- providers with a success, retention and achievement rate equal to or greater than the 90th percentile are in the top 10% of providers
- providers with a success, retention and achievement rate equal to or greater than the 75th percentile are in the top 25% of providers
- providers with a success, retention and achievement rate equal to or greater than the 50th percentile are in the top 50% of providers
- providers with a success, retention and achievement rate less than or equal to the 25th percentile are in the bottom 25% of providers
- providers with a success, retention and achievement rate less than or equal to the 10th percentile are in the bottom 10% of providers

This is illustrated by the following table. A provider with a success rate of 45% would be in the bottom 10% of providers, whereas a provider with a success rate of 82% would be in the top 10% of providers.

| Percentile Position | Overall Success Rate % | |---------------------|------------------------| | 10 | 46.5 | | 25 | 59.4 | | 50 | 68.6 | | 75 | 74.5 | | 90 | 81.3 | An alternative way of looking at this for provider types is that:

- the top 10% of providers in a provider type category have a success, retention or achievement rate equal to or greater than the 90th percentile
- the top 25% of providers in a provider type category have a success, retention or achievement rate equal to or greater than the 75th percentile
- the top 50% of providers in a provider type category have a success, retention or achievement rate equal to or greater than the 50th percentile
- the bottom 25% of providers in a provider type category have a success, retention or achievement rate less than or equal to the 25th percentile
- the bottom 10% of providers in a provider type category have a success, retention or achievement rate less than or equal to the 10th percentile

Percentiles are calculated at provider level in order to provide information on variation between providers. This is different from the success, retention and achievement rates, which are calculated as the average rate for all the relevant enrolments, weighting each enrolment equally.

Percentiles weight each provider equally. This means that learners in smaller providers have a greater bearing on results than those from larger providers. The results for small groupings, such as learning aims being studied by adults in sixth forms, will be affected by this weighting more than others.

Both the mean success, retention and achievement rates and the percentiles are valid and useful measures, depending on whether the overall performance of the sector is of interest (mean success, retention and achievement rates) or the variability between providers is the focus (percentiles). 11) Further Information and Help

If you would like any more information about the Education and Training National Success Rates Tables please contact our Service Desk on 0870 267 0001 or email servicedesk@thedataservice.org.uk.

Please be aware that the Data Service is responsible for producing National Success Rate Tables in accordance with the Classroom Based Provision Qualifications Success Rates business rules approved by the Skills Funding Agency. The business rules and technical specification are available at the Success Rate pages of the Data Service website.

Providers will find answers to most of their queries regarding the Education & Training National Success Rates Tables in this guidance document or in the frequently asked questions at the National Success Rates Tables page on the Data Service website. It is strongly recommended that providers refer to these sources before contacting the Data Service.

If providers need further help, the Data Service will be pleased to receive queries regarding the Education & Training National Success Rates tables. However, when logging a query with the Data Service please provide a detailed explanation of the query being raised.

Published by: The Data Service Version: 1 Date: 25 April 2013