Annex E: Analysis of service performance data

1.1 This annex summarises data on the disruption experienced by passengers in the weeks following the May 2018 timetable change. It demonstrates the severity of disruption on specific routes using different metrics: the number of trains removed from the published timetable, the number of minutes by which trains were delayed, the percentage of planned trains that did not run, and the volume of passenger complaints received by operators. The annex shows that there was a disproportionate level of disruption on GTR and Northern services, using regulatory data that is gathered and monitored by ORR. It also shows that there were material ‘knock-on’ effects on the performance of TPE. The annex concludes with a description of the potential safety consequences of the failure to manage disruption.

1.2 There were warnings of potential issues immediately prior to the introduction of the new timetable. Passengers were told that anticipated improvements to performance, capacity and service provision would not be immediately apparent.

1.3 On 16 May GTR warned that on some services ‘it will take a few weeks until we reach normal operation in the off peak’, but that nevertheless ‘we are keeping the changes [to the published timetable] to an absolute minimum, at quieter times of the day wherever possible’. On the Thameslink services passengers would experience ‘a limited short term implication at night, Monday to Friday, when a small number of services will not run’.¹

1.4 A bulletin from Northern on the 18 May stated that ‘some routes will see a deterioration in timetable services [and that] opportunities for resolving last-minute train scheduling problems have not been available’.²

1.5 These warnings did not give passengers an accurate impression of the scale of the disruption that was to follow. The weeks following the timetable change saw a threefold increase in delay minutes, a doubling of the percentage of trains cancelled, widespread confusion about train dispatch and stopping patterns, and a corresponding surge in the number of complaints. The evidence that passengers have submitted to ORR shows that there was a widespread and severe disruption to peoples’ work, social lives, and finances. This impact on passengers is set out in detail in the next chapter.

1.6 Disruption was not distributed evenly across the UK rail network. Some routes and operators were largely unaffected, while there was a significant level of disruption on the Northern route and on GTR's Thameslink and Great Northern routes.

¹ GTR stakeholder communication, 16 May 2018 ² Connections Express – The regular email bulletin for Northern Stakeholders, 18 May 2018. 1.8 The map below shows relative punctuality over time. It compares the ‘on-time’ performance of passenger services calling at stations across the national network in the weeks between the 20 May and 3 June (when a revised timetable was introduced), compared to the weeks immediately prior to the timetable change. The disproportionate levels of disruption experienced by passengers on the Northern route and GTR services are clearly shown.³

Diagram A: Relative on-time⁴ performance, showing 20 May – 3 June 2018, compared to 1 April – 19 May 2018

1.9 The ‘on time’ measure is calculated against the ‘plan of the day’, a daily schedule of services which is agreed by the operator and Network Rail at 22:00 the night before. However, these parties can agree to move to an emergency timetable (typically in response to adverse weather conditions).

³ Source: Network Rail performance data, submission to ORR.

⁴ On Time measures the proportion of recorded station stops that trains arrive at within one minute of the scheduled arrival time with no distinction made for long distance services. This covers 80% of all station stops which are equipped with the necessary recording equipment. In this case, the cancelled services will be excluded from the recording of punctuality and reliability measures ‘on the day’ including ‘on time’, the Passenger Performance Measure (PPM) and Cancellations and Significant Lateness(^5) (CaSL).

1.10 In order to present a full picture of cancellations, including those that were removed from the timetable prior to the ‘plan of the day’, figure B illustrates all cancellations across the national rail network. It shows the proportion of all trains(^6) that were cancelled before the plan of the day was agreed(^7), and partial(^8) and full cancellations made against the plan of the day.

Figure A: Proportion of trains cancelled pre-plan, in full, in part. (Source: ORR monitoring data)

![National - Proportion of cancelled trains](image)

Notes: Note: Monday 07 May and Monday 28 May were bank holiday Mondays, when services were affected by engineering work. Weekends are also likely to have a higher proportion of pre-plan cancellations caused by engineering work. Data is for GB franchised operators, excluding Merseyrail, owing to concerns about data quality.

1.11 This shows an increase in full cancellations in the two weeks after the timetable change, followed by an increase in pre-plan cancellations in the subsequent two weeks. This increase in pre-plan

______________________________________________________________________

(^5) Significant Lateness refers to trains that called at all scheduled station stops, but were between 30 and 119 minutes late at their destination.

(^6) Applicable trains within TRUST (Network Rail’s computer system used for monitoring the progress of trains and tracking delays on the GB rail network.)

(^7) Cancellations may have been replaced with a revised schedule of the same train (i.e. for instance a Sheffield to Hull train being cancelled and a revised Doncaster to Hull train being introduced). Such cases will feature as 2 trains in the TRUST applicable train count.

(^8) Part cancellations are trains on the day that either failed to complete their journey, did not stop at all scheduled stops, or were more than 120 minutes late at their destination. Full cancellations are trains on the day that failed to run any part of the journey. cancellations corresponds with the revised timetables introduced on 4th June by Northern, and 15 July by GTR, in an effort to provide passengers with a more reliable service.

1.12 The disproportionate extent of disruption suffered by passengers on the GTR and Northern routes is illustrated by the percentage of trains that were cancelled on these services, compared to elsewhere on the network.

1.13 Figure B shows the percentage of trains cancelled from the daily plan on the network as a whole, excluding those services run by GTR and Northern. Throughout the period in question, the percentage of trains ran by all other operators never fell below 96%.

Figure B: Trains planned / ran, excluding GTR and Northern. (Source: ORR monitoring data)

1.14 In comparison, the corresponding data for Northern and GTR is significantly worse. The following sections illustrate the level of cancellations, delay minutes and complaints for these two operators.

______________________________________________________________________

9 “Trains Planned” refers to the number of trains that are scheduled to run on the timetabled ‘plan of the day’, produced each day for the following day’s services. Trains Ran refers to how many trains that were scheduled to run actually did so. Data for these sources is collected by ORR as part of our regular performance monitoring function. GTR performance data

1.15 This section considers the performance experienced by passengers on GTR services. It shows data for the period prior to the introduction of the 20 May timetable and during the disruption which followed.

Train service cancellations and delays

1.16 Figure C presents data for the proportion of pre-plan and plan of the day GTR trains that were cancelled, either fully or partially. This illustrates an increase in full cancellations in the two weeks following the timetable change and a steady increase in pre-plan cancellations in the subsequent two weeks, as GTR worked to implement a reliable timetable.

Figure C: GTR cancellations, pre-plan and post-plan. (Source: ORR monitoring data)

Note: GTR services on Wednesday 09 May, Thursday 24 May, Saturday 26 May and Friday 19 June, were affected by industrial action.

1.17 Figure D below shows that the percentage of Trains Ran for GTR decreased by 7.5 percentage points for the two weeks after the timetable change (compared with the two weeks before the timetable change), before recovering somewhat in the following three weeks. The percentage of Trains Ran increased to above 90% following the introduction of a revised timetable, though this is still well below the 95% achieved in the two weeks before the timetable change. 1.18 With GTR planning to run on average 3,880 trains on a weekday, approximately 470 scheduled trains, more than one in ten did not run each day.

1.19 For those trains that did run, passengers experienced a sharp decline in punctuality. The graph in Figure E below shows that the number of minutes by which GTR's services were delayed (which can be attributed to GTR itself) more than trebled in week one of the timetable and had not returned to pre-20 May levels by late June.(^\\text{10})

______________________________________________________________________

(^{10}) Delay incidents producing three or more minutes of delay on Britain's railways are attributed to either Network Rail or a train operator. As well as infrastructure and operational delays such as signal failures and overrunning engineering works, delays caused by external factors such as severe weather, vandalism, cable theft and trespass are also attributed to Network Rail. This is because they are considered best placed to mitigate for such incidents. Figure E: GTR delay minutes (Source: ORR monitoring data) Passenger complaints and compensation claims

1.20 Provisional complaints information from GTR shows that it dealt with nearly 7,300 passenger complaints in period three (27 May to 23 June) 2018/19, a considerable rise compared to the corresponding period the year before, and a marked increase over the period immediately prior to the introduction of the new timetable. As the graph below demonstrates, passengers were most dissatisfied about timetabling and connection issues as well as train service performance, both of which showed a sharp escalation in rail period three.

Figure F: Provisional complaints data, GTR. (Source: ORR monitoring data)

1.21 Based on provisional delay compensation data from GTR, claims for delay compensation to GTR increased by 45% between periods one and two to 84,292 claims, and by 191% between periods two and three to 244,921 claims. Northern

1.22 This section considers the performance experienced by passengers on Northern services. It shows data for the period prior to the introduction of the 20 May timetable and during the disruption which followed.

Train service cancellations and delays

1.23 Figure G presents data for the proportion of pre-plan of the day and plan of the day Northern trains that were cancelled, either fully or partially. It illustrates an increase in full cancellations in the two weeks following the timetable change, and an increase in pre-plan cancellations in the subsequent two weeks which correspond with the interim timetables put in place on 4th June. The increase in pre-plan cancellations also corresponded with a decrease in part and full post-plan cancellations.

Figure G: Northern cancellations, pre-plan and post-plan (Source: ORR monitoring data)

1.24 The following chart (Figure H) shows that the percentage of ‘Trains Ran’ (as a proportion of services in the ‘plan of the day’) for Northern decreased by an average of 5.8 percentage points for the two weeks after the timetable change (compared with the two weeks before the timetable change), before recovering to a higher level in the following three weeks, following the introduction of a revised timetable on the 4th of June\\textsuperscript{11}. This revised timetable was intended to provide a greater degree of reliability and stability for passengers.

Figure H: Northern trains planned / ran (Source: ORR monitoring data)

1.25 With Northern planning to run on average 2,810 trains on a weekday, approximately 310 scheduled trains, more than one in ten each day did not run.

1.26 For those trains that did run, figure I below shows that the number of delay minutes on Northern Rail’s services more than doubled in week one and two of the new timetable, before recovering to pre-20 May levels following the introduction of a revised timetable.

\\textsuperscript{11} Figures for Trains Planned/Trains Ran have been adjusted to account for strike days on 9 May, 24 May, 26 May, and 19 June that are not accounted for under business as usual. This is to allow a more accurate comparison across the time period. Note: Monday 07 May and Monday 28 May were bank holiday Mondays, when GTR services were affected by engineering work. Data for these sources is collected by ORR as part of our regular performance monitoring function. Northern passenger complaints and claims for compensation

1.27 The third measure of performance, passenger complaints and compensation claims, reflects the dip in performance shown by the other two metrics. Based on provisional data for periods one - three (1 April to 23 June) the level of passenger complaints to Northern relating to train service performance (reliability and punctuality) increased sharply in rail periods two and three (29 April to 23 June) following the introduction of the new timetable, as shown in Figure J. This can be seen most clearly in the level of complaints from passengers that relate to train service performance, whereas the number of complaints that relate to other categories remains broadly consistent. 1.28 The number of claims submitted by passengers for delay compensation also increased accordingly. Based on provisional data on delay compensation supplied to ORR from Northern Rail, claims increased 146% between period one and two to 22,006 claims, and by a further 68% to 36,968 in period three (see Figure K, below).

Figure K: Provisional compensation claim figures, Northern (Source: ORR monitoring data) Disruption to other operators

1.29 The disruption to GTR and Northern services also had a knock-on effect on some services provided by, in particular, TransPennine Express (TPE). East Midlands Trains (EMT), and Virgin Trains East Coast (now London North Eastern Railway (LNER)) were also affected.

1.30 Whilst some of this impact was negative, there was evidence of train companies rising to the challenge to help passengers. For example as one of the regular GTR cancellations was a train used by LNER staff to get into work, LNER operated a service instead for staff which they extended for use by passengers. LNER also added an additional stop at Huntingdon on the early morning service from Kings Cross for school children, and more of their trains called at Stevenage(^\\text{12}).

1.31 DfT also responded to the issues through the establishment of a group involving affected train operators to discuss where the problems and issues were and how they were being addressed.

Figure L below shows that the number of delay minutes on TPE’s services (that can be attributed to other train operators) more than trebled in week one and remained high, and did not recover until week five.

Figure L: TPE Weekly delay minutes

1.32 Whilst the number of minutes that TPE passengers were delayed was considerably less than passengers using GTR and Northern Rail services, we are aware of the problems experienced by passengers using TPE’s services. In particular, congestion in the Central Manchester corridor quickly affected services, with trains being terminated short rather than running to the airport. At

(^{12}) LNER interview by ORR 25 July 2018. the other extremities of the TPE network trains were being terminated short to maintain times on the return journey. This led to passengers being left at stations such as Malton, York, and Stalybridge, some of which only have an hourly TPE service, or relied on Northern Rail connections which may not have been running. Many TPE services also serve Leeds and were affected by the station working issues at that station.\\textsuperscript{13}

**Safety**

1.33 Under the Health and Safety at Work Act 1974, railway operators have a duty to ensure, so far as reasonably practicable, the safety of passengers. It is clear that good planning and communication are key to making the best of abnormal arrangements. A lack of capacity, in terms of both infrastructure and rolling stock, combined with difficulties with passenger information, can potentially lead to increased safety risk. The most serious potential risks can include, for instance, people falling off crowded platforms.

1.34 Following the introduction of the timetable on 20 May, there were instances of very short notice cancellations (including after the train should have arrived at a station). To compound the situation, the information to staff and customers was inadequate resulting in platform crowding and train dispatch issues, and passengers having to rush between platforms at short notice with resulting risks to passengers. Some of these issues were particularly apparent at Harpenden station. Senior ORR staff met with senior representatives of GTR to discuss ORR’s concerns in relation to train dispatch and crowding and also visited the GTR control in order to identify the issues with wrong or missing information on Customer Information System screens, and GTR’s proposals to quickly resolve the situation. This included seeking assurance on the arrangements for the introduction of the 15 July interim timetable.

\\textsuperscript{13} TPE interview by ORR 18 July 2018.