Apprenticeships national success rates tables for the 2012/2013 academic year

Guidance notes

Date of issue 16 April 2014

Publication intent NOT PROTECTIVELY MARKED

Contents Section 1 – Introduction ........................................................................................................... 2 Purpose of this document ........................................................................................................ 2 Purpose of the Apprenticeships national success rates tables ........................................... 2 Location of national success rates tables for 2012/2013 .................................................. 2 Changes to the Apprenticeships national success rates tables ........................................... 2 Section 2 – Apprenticeships national success rates tables worksheets .......................... 3 Contents of the tables ............................................................................................................. 3 Format of the Apprenticeships national success rates tables ............................................. 3 Overall report .................................................................................................................... 3 Sector subject area and framework report ....................................................................... 4 Institution report for overall measure and timely measure ............................................. 4 Excluded framework report for overall and timely measure ........................................... 4 Section 3 – Apprenticeships national success rates tables open data files ..................... 5 Format of Apprenticeships national success rates tables open data files ....................... 5 Overall report .................................................................................................................... 5 Sector subject and framework report .............................................................................. 6 Institution report .............................................................................................................. 6 Excluded frameworks report ............................................................................................ 6 Section 4 – Contents of national success rates tables ....................................................... 7 Report columns .................................................................................................................... 7 Institution type .................................................................................................................. 7 Age group .......................................................................................................................... 7 Apprenticeship level and type ............................................................................................ 7 Framework name and framework code ............................................................................ 7 Sector subject area tier 1 and tier 2 .................................................................................. 8 Overall leavers and timely leavers .................................................................................... 8 Report measures .................................................................................................................. 8 Overall success rate .......................................................................................................... 8 Timely success rate ............................................................................................................ 8 Success rates percentiles .................................................................................................. 9 Section 5 – Exclusions from the national success rates tables ........................................ 10 Section 6 – Choosing information on the national success rates tables ........................ 11 Further information and help ............................................................................................... 12 Section 1 – Introduction

Purpose of this document

1. This document provides information about the Apprenticeships national success rates tables for the 2012/2013 academic year.

Purpose of the Apprenticeships national success rates tables

2. The national success rates tables for Apprenticeships set out levels of success in Apprenticeships provision in England for the 2010/2011, 2011/2012 and 2012/2013 academic years. We publish national success rates tables every year.

3. The information in the tables is consistent with the Apprenticeships qualification success rates reports for 2012/2013 published in January 2013. It expands upon the success-rate information made available in the January statistical first release (see www.gov.uk/government/publications/learner-participation-outcomes-and-level-of-highest-qualification-held).

4. Apprenticeships national success rates tables help providers to raise the standard of their work. It allows providers to assess their performance, and helps them plan action programmes to improve their apprentices’ success rates.

5. Throughout this guidance we refer to ‘national success rates tables’ rather than ‘benchmarking data’. National success rates tables allow for comparison, and are not a standard of best practice.

Location of national success rates tables for 2012/2013

6. The national success rates tables for 2012/2013 are on the data.gov.uk website at [enter URL for website].

Changes to the Apprenticeships national success rates tables

7. The changes made to the Apprenticeships national success rates tables for 2011/2012 are as follows.

- The information from the 2012/2013 Apprenticeships qualification success rates reports has been added to the national success rates tables for 2012/2013.
- The information from the 2009/2010 Apprenticeships qualification success rates reports has been removed from the national success rates tables for 2012/2013.
- For the first time we are publishing qualification success rates reports for providers, to help them measure their success.
- Only funded framework aims are included in the success rates tables. Section 2 – Apprenticeships national success rates tables worksheets

Contents of the tables

08. Our aim in publishing the Apprenticeships national success rates tables is to provide a useful but manageable amount of information, drawing on existing statistics.

09. The information for the Apprenticeships national success rates tables has been created from providers’ Individualised Learner Record (ILR) returns, and they provide a range of national-level and provider-level statistics for success. The method of calculating success rates is available on the gov.uk website at www.gov.uk/government/publications/apprenticeships-qualification-success-rates-2012-to-2013.

10. The Apprenticeships national success rates tables contain information on success rates for three academic years (2010/2011, 2011/2012 and 2012/2013). The information for each academic year comes from the success rates dataset produced for that year, calculated using the method that applied to that year.

11. The Apprenticeships national success rates tables are calculated using the ‘overall’ success rate and ‘timely’ success rate. The timely success rate is only shown on the provider-level report.

Format of the Apprenticeships national success rates tables

12. The Apprenticeships national success rates tables are published as four compressed files. Each compressed file holds a Microsoft Excel spreadsheet that contains a number of worksheets. The separate spreadsheets are: • Overall report; • Sector subject area and framework report; • Institution report for overall measure and timely measure; and • Excluded framework report for overall and timely measure.

The contents of the spreadsheets are shown below.

Overall report

13. The overall report contains separate worksheets that show national success rates at headline level, which is an overview of all national success rates. This report also presents the success rates by: • sex; • learning difficulty or disability; • ethnic background; • apprentices’ local authority (based on the apprentices’ home postcodes); and • the local authority for where the learning is delivered (based on the postcode of the providers’ delivery locations).
14. The overall report also shows success rates percentiles on a separate worksheet. For a further explanation of success rates percentiles and how they are calculated, see paragraphs 47 to 52. The report shows percentiles at headline level and by:

- ethnic background;
- sex; and
- learning difficulty or disability.

15. Percentile sheets for apprentices’ and delivery locations’ local authorities have not been included in the overall report as the dataset contains many small groupings (cohorts), which would have a significant effect on the rates presented.

**Sector subject area and framework report**

16. The sector subject area and framework report contains worksheets that show the national success rates by:

- sector subject area tier 1 and 2; and
- framework.

**Institution report for overall measure and timely measure**

17. The institution report contains worksheets that show the success rates of each institution at headline level, which is a high-level overview of the overall success rates for each provider, by:

- sex;
- ethnic background;
- learning difficulty or disability;
- sector subject area; and
- framework.

18. The hybrid end date (which is the actual end date or the planned end date, whichever is later) is shown for the overall success rate and the planned end date is shown for the timely success rate.

19. The sector subject area and framework reports could be split by year over separate worksheets if you go over the row limit of 65,536 in Excel 2003.

20. A timely success rate report is published for the first time in 2012/2013. This shows timely national success rates for providers in the same worksheet format as the institutions’ overall national success rates reports.

**Excluded framework report for overall and timely measure**

21. Due to the rules described in section 5, the full coverage of frameworks delivered nationally, and at institution level, is not available in the national- and institution-level framework spreadsheets for the overall success rates.

22. The excluded frameworks are published in the excluded framework worksheets to allow the full scope of frameworks delivery to be seen. The report indicates the reasons for the particular frameworks being excluded. It is split into four worksheets:

- Excluded Apprenticeship frameworks at national level for 2010/2011, 2011/2012 and 2012/2013;
- 2010/2011 overall excluded frameworks for institutions;
- 2011/2012 overall excluded frameworks for institutions; and

23. As the timely success rate is only being published for the institution report, excluded frameworks at institution level will be shown in the timely excluded frameworks report along with the reasons for the exclusions. The report is split into three worksheets:

- 2010/2011 timely excluded frameworks for institutions;
- 2011/2012 timely excluded frameworks for institutions; and
- 2012/2013 timely excluded frameworks for institutions.

24. The excluded frameworks for institutions are displayed by year on separate worksheets as the volume of data goes over the Excel 2003 row limit of 65,536.

Section 3 – Apprenticeships national success rates tables open data files

Format of Apprenticeships national success rates tables open data files

25. The Apprenticeships national success rates tables are also published in a non-proprietary format to make sure they meet the 'three-star open data standard'. The comma-separated variable (CSV) file is our preferred non-proprietary format. For more information on open data, see the published white paper written by the Cabinet Office. This is on the website at www.gov.uk/government/publications/open-data-white-paper-unleashing-the-potential.

26. The Apprenticeships national success rates tables open data files are published as 27 separate compressed files. The contents of each file, and where the information comes from, are listed below.

Overall report

27. There are 10 files containing data from the overall report worksheets explained in paragraphs 13 and 14.

- NSRT201 – headline success rates
- NSRT202 – headline percentiles
- NSRT203 – success rates by gender
- NSRT204 – percentiles by gender
- NSRT205 – success rates by ethnic background
- NSRT206 – percentiles by ethnic background
- NSRT207 – success rates by learning difficulties or disabilities
- NSRT208 – percentiles by learning difficulties or disabilities
- NSRT209 – success rates by apprentices' local authority
- NSRT210 – success rates by delivery locations' local authority. Sector subject and framework report

28. The sector subject area and framework report worksheets explained in paragraph 16 is published as two files listed below.

- NSRT211 – sector subject area and framework success rate
- NSRT212 – sector subject area success rates. (All the sector subject area worksheets are in this file.)

Institution report 29. There are 12 files containing the institution report worksheets explained in paragraphs 1 to 20.

- NSRT213 – overall headline success rates for the institution
- NSRT214 – overall success rates for the institution by gender
- NSRT215 – overall success rates for the institution by ethnic background
- NSRT216 – overall success rates for the institution by learning difficulties or disabilities
- NSRT217 – overall sector subject area success rates for the institution
- NSRT218 – overall framework success rates for the institution
- NSRT222 – timely headline success rates for the institution
- NSRT223 – timely success rates for the institution by gender
- NSRT224 – timely success rates for the institution by ethnic background
- NSRT225 – timely success rates for the institution by learning difficulties or disabilities
- NSRT226 – timely sector subject area success rates for the institution
- NSRT227 – timely framework success rates for the institution

Excluded frameworks report 30. There are three files containing excluded frameworks report worksheets as explained in paragraphs 21 and 24.

- NSRT219 – overall excluded national frameworks
- NSRT220 – overall excluded institution frameworks
- NSRT221 – timely excluded institution frameworks

31. All the excluded institution framework worksheets are in a single file. Section 4 – Contents of national success rates tables

32. This section explains the columns in the Apprenticeships national success rates tables worksheets.

Report columns

Institution type

33. Providers are classified into one of six institution types. The institution types reported on are as follows.

- General further-education or tertiary college
- Other publicly funded institution
- Private-sector publicly funded institution
- School
- Sixth-form college
- Specialist college

34. The ‘specialist college’ category includes agriculture and horticulture colleges, and art, design and performing arts colleges.

An ‘all institution type’ category is also available on some of the worksheets.

Age group

35. The age group of a cohort is based on an apprentice’s age on the start date of the framework aim. Apprentices of unknown age are included in the 25 and over age group. Apprentices under 16 are included in the 16 to 18 age group. All tables show information divided into the following age groups.

- 16 to 18
- 19 to 24
- 25 and over
- All ages

Apprenticeship level and type

36. Apprenticeships are grouped according to their level and type. The Apprenticeship levels and types reported on are as follows.

- Level 2 – intermediate Apprenticeship
- Level 3 – advanced-level Apprenticeship
- level 4 + – higher-level Apprenticeships (this includes level-4 and level-5 Apprenticeships)
- All levels and types

Framework name and framework code

37. The framework name and framework code of an Apprenticeship is in the learning aim reference application (LARA). Sector subject area tier 1 and tier 2
38. The sector subject area tier 1 and sector subject area tier 2 for an Apprenticeship framework are in the data lookup used in the October 2013 statistical first release and qualification success rate reports.

Overall leavers and timely leavers 39. The ‘overall leavers’ column shows the number of Apprenticeships that are planned to be completed, or have actually completed, within a given academic year.

40. The ‘timely leavers’ column shows the number of Apprenticeships that are expected to be completed by the planned end date, or within 90 days of the planned end date, within a given academic year.

41. A ‘leaver’ is defined as an apprentice who:

- has not transferred to a different programme with the same provider;
- has not transferred to another provider due to intervention by us or the Education Funding Agency;
- is not on a planned break; and
- has not withdrawn within the first six weeks of study.

42. The number of leavers in a sub-cohort (a distinct group of apprentices within a cohort) may not add up to the total number of apprentices in the cohort because:

- the number of leavers is rounded to the nearest 10 at all levels; and
- small cohorts of leavers (fewer than five) are not included in the published tables.

Report measures 43. The overall and timely success rates are calculated in line with our Apprenticeship qualifications success rates business rules and methods. These are on the gov.uk website at www.gov.uk/government/publications/apprenticeships-qualification-success-rates-2012-to-2013.

Overall success rate 44. The overall success rate measures the percentage of Apprenticeship frameworks achieved against the overall number of Apprenticeship frameworks completed. The overall success rate is based on the hybrid end year. The hybrid end year is the actual learning end year or the planned learning end year, whichever is later.

45. The definition of ‘leaver’ in paragraph 41 is also used to exclude Apprenticeship frameworks from the calculation of the overall success rate.

Timely success rate 46. For the first time, the Apprenticeships national success rates tables for 2012/2013 provide a limited set of information for providers using the timely success rate. The timely success rate measures the percentage of Apprenticeship frameworks that were achieved by their planned end date, or within 90 days after their planned end date, against the number of frameworks that were expected to be completed. The exclusions for the overall success rate also apply to the timely success rate. Success rates percentiles

47. Success rates percentiles allow providers to compare their results against the range for their sector or particular types of provider. One way of interpreting percentiles is as follows.

- Providers with a success rate on or above the 90th percentile are in the top 10% of providers.
- Providers with a success rate on or above the 75th percentile are in the top 25% of providers.
- Providers with a success rate on or above the 50th percentile are in the top 50% of providers.
- Providers with a success rate below or on the 25th percentile are in the bottom 25% of providers.
- Providers with a success rate below or on the 10th percentile are in the bottom 10% of providers.

48. This is illustrated by the following table. A provider with a success rate of 45% would be in the bottom 10% of providers, whereas a provider with a success rate of 76% would be in the top 25% of providers.

| Percentile | Example overall success rate | |------------|-----------------------------| | 10 | 46.5% | | 25 | 59.4% | | 50 | 68.6% | | 75 | 74.5% | | 90 | 81.3% |

49. An alternative way of looking at this for provider types is that:

- the top 10% of providers in a particular category have a success, retention or achievement rate on or above the 90th percentile;
- the top 25% of providers in a particular category have a success, retention or achievement rate on or above the 75th percentile;
- the top 50% of providers in a particular category have a success, retention or achievement rate on or above the 50th percentile;
- the bottom 25% of providers in a particular category have a success, retention or achievement rate below or on the 25th percentile; and
- the bottom 10% of providers in a particular category have a success, retention or achievement rate below or on the 10th percentile.

50. Percentiles are calculated at provider level in order to display the differences between providers. This is different from the mean success rates, which are calculated as the mean rate for all the relevant leavers, averaging each leaver equally.

51. When calculating percentiles, the success rate for each provider is used to create an average. This means that apprentices with smaller providers have a greater bearing on results than those from larger providers. The results for small groupings, such as Apprenticeships being studied in sixth-form colleges, will be affected by the average.

52. Both the mean success rate and the percentiles are valid and useful measures, depending on whether you are interested in the overall performance of the sector (consider mean success rates) or the differences between providers (consider percentiles).

Section 5 – Exclusions from the national success rates tables

53. The rules on excluding small cohorts, suppressing figures and rounding figures in the success rates worksheets are listed below.

- Cohorts with fewer than five ‘overall leavers’ are not shown on any worksheets.
- Cohorts with five or more ‘overall leavers’ but fewer than 30 have the number of leavers replaced with a dash (–).
- The number of overall leavers is rounded to the nearest 10. This prevents the possibility of individual apprentices being identified.

54. The rules on exclusions, suppressions and rounded figures on the percentiles worksheet are listed below.

- If there are 20 or more providers, the 10th, 25th, 50th, 75th and 90th percentiles are shown on the worksheet.
- If there are between 11 and 19 providers, only the 50th percentile is shown on the worksheet.
- If there are 10 or fewer providers, no percentiles are shown on the worksheet.

55. Where individual framework success rates are reported, if a framework is only delivered at a single institution, the framework is not included in the report. This affects the overall report (framework success rate worksheet) and the institution report (success rates for institutions by framework) worksheet for both the overall success rates and timely success rates. Any frameworks that are excluded are listed in the excluded frameworks report worksheets, together with the reasons why they are excluded.

56. As rows on the framework spreadsheet are excluded, the total of the leavers for the individual institution types may be less than the number of leavers for the ‘all institution type’ row. The excluded institution types are shown in the excluded frameworks report spreadsheets, together with the reason for them being excluded. Section 6 – Choosing information on the national success rates tables

57. The success rate data is presented as a series of rows in Excel worksheets. Each row shows the success rate of a specific cohort of apprentices. Summary information is shown at many levels (for example, all ages, all Apprenticeship levels, all institution types). All the columns in the worksheets contain drop-down menus that allow you to choose the information you want.

58. The screenshot below shows an example of how the information will appear in the headline success rate worksheet of the overall report.

59. The drop-down menus are used to choose the level of information. The screenshot below shows the effect of using the drop-down menus to choose success rates broken down by age for all Apprenticeship levels for 2012/2013. Further information and help

60. If you would like any more information about the Apprenticeship national success rates tables, phone our service desk on 0870 267 0001 or email servicedesk@sfa.bis.gov.uk.

61. We produce Apprenticeships national success rates tables in line with our Apprenticeship qualifications success rates business rules. The business rules and technical specifications are on the success rate pages of the gov.uk website at www.gov.uk/government/publications/apprenticeships-qualification-success-rates-2012-to-2013.

62. If you need more help, give our service desk a detailed explanation of your query.

Published by: Skills Funding Agency Version: 1.0 Date: 16 April 2014