REPORT: STAKEHOLDER ENGAGEMENT

# REVIEW OF RAIL PERFORMANCE INCENTIVES

## REPORT: STAKEHOLDER ENGAGEMENT

### IDENTIFICATION TABLE

| Client/Project owner | ORR | |----------------------|-----| | Project | Review of Rail Performance Incentives | | Title of Document | Report: Stakeholder Engagement | | Type of document | Final Report | | Date | 19/10/2020 | | File name | 20201019 Performance Incentives Review – Full Report | | Reference number | ORR/CT/19-57 109599 | | Number of pages | 56 |

### APPROVAL

| Version | Name | Position | Date | |---------|------|----------|------------| | Draft | Authors | Paul Le Masurier Evelyn Robertson Emma Salter | Projects Director Associate Senior Consultant | 21/07/2020 | | | Checked by | Chris Pownall | Director | 22/07/2020 | | | Approved by | Chris Pownall | Director | 23/07/2020 | | Draft Final | Author | Evelyn Robertson | Associate | 12/08/2020 | | | Checked by | Chris Pownall | Director | 13/08/2020 | | | Approved by | Chris Pownall | Director | 13/08/2020 | | Final | Author | Evelyn Robertson | Associate | 19/10/2020 | | | Checked by | Chris Pownall | Director | 19/10/2020 | | | Approved by | Chris Pownall | Director | 19/10/2020 |

# TABLE OF CONTENTS

**EXECUTIVE SUMMARY**

1. **INTRODUCTION** 1.1 Research Background 1.2 Research Objectives 1.3 Research Context 1.4 Approach 1.5 Report Structure

2. **INFLUENCES ON TRAIN PERFORMANCE AT A STRATEGIC LEVEL** 2.1 Introduction 2.2 Influences on Strategic Decision Making 2.3 Customer Satisfaction 2.4 The franchise bidding process and franchise agreements including Schedule 7.1 Influences 2.5 Individual motivators 2.6 Media and political influences 2.7 ORR monitoring and Network Rail Scorecards 2.8 Schedule 8 2.9 Secondary influences on train service performance decision-making 2.10 Collaborative working

3. **INFLUENCES ON TRAIN PERFORMANCE AT AN OPERATIONAL LEVEL** 3.1 Introduction 3.2 Day-to-day decision making 3.3 Sense of pride 3.4 Company policy 3.5 Data and experience-led decisions 3.6 Secondary influences on train service performance decision-making 3.7 Collaborative working

4. **IMPACT OF COVID-19 ON RAIL INDUSTRY** 4.1 Introduction 4.2 Key factors behind the performance improvement 4.3 Short, medium and longer terms risks to performance

5. **SUMMARY OF FINDINGS** 5.1 Key Themes 5.2 Summary EXECUTIVE SUMMARY

Introduction

ORR commissioned SYSTRA to undertake research into rail performance incentives, to include formal incentive mechanisms (extrinsic influencers) and other behavioural motivators (intrinsic motivators). The overall aim of the research is to build an understanding of what, in practice, motivates people and organisations to improve rail performance. ORR is seeking to use the findings to optimise performance incentives which are under ORR’s control, so as to encourage better outcomes for users of the railway.

This report summarises findings from a series of interviews and an online survey, both taking place between February and July 2020, with over 50 representatives from:

- Train Operating Companies (TOCs) and Owning Groups;
- Freight Operating Companies (FOCs);
- Infrastructure Managers (primarily Network Rail);
- Central government and local government; and
- Trade associations, representative bodies and consumer groups.

A range of strategic and operational representatives were spoken to, as were different TOC types (comprising regional, long distance, London/South East, and Open Access) to ensure a range of viewpoints were captured.

Key Themes

The research has uncovered key themes, when considering rail performance incentives. Primarily, these are:

- A demonstration that the way some formal incentive mechanisms are intended to work in principle, is not necessarily how they play out in real-life situations;
- In many cases, clear distinctions between decision-making at strategic and operational level can be drawn, revealing that different people across the industry face different motivations, and place greater weight on different incentives, depending on the focus of their role;
- Personal job satisfaction and customer satisfaction derived from running trains as planned, on time are key motivators for delivering a high performance service across all stakeholders, however other considerations come into play, such as a capacity/performance trade-off and the need for some formal mechanisms to provide protection to franchised TOCs;
- Prioritising increased capacity, an ambitious timetable, and theoretically efficient use of resources, as some TOCs believe is required to be successful in the franchising process, can be to the cost of performance on the network;
- There are very different views on the effectiveness of performance targets, particularly those in franchise agreements even within the same stakeholder types, and this may reflect different experiences due to differences in the effectiveness of the calibration;
- Increased collaborative working was viewed across the industry as beneficial for improving performance but some incentive mechanisms were perceived as discouraging this way of working; and
- Theoretically optimal incentive mechanisms must be supported by high quality data, shared across different parties, to facilitate good performance outcomes.

A more detailed summary of the findings can be found in Chapter 5.

1. INTRODUCTION

1.1 Research background

1.1.1 Reliability and punctuality are the fundamental building-blocks of a high-performance railway network, as recognised in the Williams Rail Review(^1), and confirmed by Transport Focus(^2), Network Rail(^3), The Office of Rail and Road (ORR), and users and operators of passenger and freight train services across Britain. Despite the high priority given to reliability and punctuality in the industry’s processes since privatisation, evidence from recent years shows that from 2000 to a high-point in 2011/12, there has been a medium to long-term decline in performance from 2015 to 2020. In the year ending 2019-20 Q2, 65.1% of services arrived at station stops on time and 2.8% of services were cancelled(^4). Data from 2019-20 Period 10 shows 4.5% of services were cancelled or significantly late(^5).

1.1.2 There are a number of organisations responsible for the delivery of train service performance across Britain’s railway network (GB Rail). The Infrastructure Manager, primarily Network Rail, owns and manages the track and supporting signalling systems, and manages the timetable process to regulate traffic. Passenger Train Operating Companies (TOCs) and Freight Operating Companies (FOCs) also play an important role in the delivery of service performance, through their provision of trains and staff to run the timetabled services, whilst public sector funders such as the Department for Transport (DfT) and Transport Scotland and other funders and specifiers of train services, and rail regulator the Office of Rail and Road (ORR) also play key roles in facilitating and monitoring performance, including agreeing contractual and ad-hoc performance incentives that seek to improve the decisions that affect the performance of rail services, particularly those decisions made by Infrastructure Managers and operating companies.

1.1.3 There are a number of mechanisms in the rail industry that are intended to create incentives for train operators, Network Rail and other infrastructure managers to take decisions that positively affect train performance. These incentives include, but are not limited to, financial incentives, and may work at either a corporate or individual level. Incentive mechanisms and frameworks include regulatory mechanisms, commercial and franchise mechanisms, and reputational incentive mechanisms.

1.1.4 A risk with any performance incentive framework involving multiple organisations is that it may encourage investment in shifting responsibility for delays, rather than investment in actually reducing delays. This would be likely to lead to undesirable outcomes for users of the railway. As such, ORR wish to update their evidence base describing how important and effective performance mechanisms are in shaping decisions and outcomes.

______________________________________________________________________

(^1) Williams Rail Review update: [https://www.gov.uk/government/speeches/williams-rail-review-an-update-on-progress](https://www.gov.uk/government/speeches/williams-rail-review-an-update-on-progress)

(^2) Transport Focus Rail Punctuality Comment: [https://www.transportfocus.org.uk/news-events-media/news/train-punctuality-measured-minute/](https://www.transportfocus.org.uk/news-events-media/news/train-punctuality-measured-minute/)

(^3) Network Rail Railway Performance: [https://www.networkrail.co.uk/who-we-are/how-we-work/performance/railway-performance/](https://www.networkrail.co.uk/who-we-are/how-we-work/performance/railway-performance/)

(^4) Passenger Rail Performance 2019-20 Q2 Statistical Release: [https://dataportal.orr.gov.uk/media/1629/passenger-performance-2019-20-q2.pdf](https://dataportal.orr.gov.uk/media/1629/passenger-performance-2019-20-q2.pdf)

(^5) ORR, CaSL data by TOC: [https://dataportal.orr.gov.uk/statistics/performance/passenger-rail-performance/casl-by-toc-and-sector-table-37/](https://dataportal.orr.gov.uk/statistics/performance/passenger-rail-performance/casl-by-toc-and-sector-table-37/) 1.2 Research Objectives

1.2.1 As part of their aims to drive for a safer, high performing and efficient railway, ORR wishes to build an understanding of what, in practice, motivates people and organisations to improve rail performance. The research objectives are to:

- Identify the mechanisms which exist to influence people and organisations’ decisions related to train performance;
- Understand the motivations for improving train performance, including the importance and influence of extrinsic and intrinsic incentives (both financial and non-financial), and how they interact;
- Understand the consequences of industry mechanisms in relation to performance outcomes, including any impacts on collaboration between different individuals and organisations in the event of performance related incidents; and
- Ascertain how incentives and behaviour vary within and between different organisations.

1.3 Research Context

1.3.1 ORR commissioned SYSTRA to undertake a review into rail performance incentives, to include both formal incentive mechanisms (extrinsic influencers) and other behavioural factors (intrinsic motivators). The research study was designed in three parts:

- Wave 1 interviews: in-depth interviews with industry representatives, identifying key influencing factors on behaviours affecting rail performance;
- Online survey: quantitative and qualitative insights from a larger number of respondents across the industry; and
- Wave 2 interviews: in-depth interviews with industry representatives, refining and challenging the themes emerging from the Wave 1 interviews.

1.3.2 Wave 1 interviews took place in February and early March 2020.

1.3.3 Due to the global Covid-19 pandemic, as of 23rd March 2020, government suspended rail franchise agreements for at least six months. All revenue and costs risks have been transferred to government, whilst operators continue with day-to-day services for a management fee. A significantly reduced rail service was initially run, and demand substantially reduced as a result of government guidance regarding social distancing. From May 2020, service levels have been increased, but not yet to pre-pandemic levels, and demand has only slowly started to recover.

1.3.4 Due to the impact of Covid-19 on the rail industry, further primary data collection was temporarily put on hold. Wave 2 interviews and the online survey subsequently took place in June and July 2020, after consultation with the industry, to ensure this would not place an unnecessary burden on respondents.

1.4 Approach

1.4.1 ORR provided SYSTRA with an initial list of rail industry contacts covering the following key stakeholder groups, and this was supplemented with SYSTRA’s own industry contacts:

- Train Operating Companies (TOCs) and Owning Groups;
- Freight Operating Companies (FOCs);
- Infrastructure Managers (primarily Network Rail);
- Central government and local government; and 1.4.2 Representatives were identified in each group either for interview themselves, or to facilitate interviews with appropriate respondents to ensure a structured coverage of role, background and experience within the target groups.

1.4.3 A range of strategic and operational representatives were targeted, as were different TOC types (comprising regional, long distance, London/South East, and Open Access) to ensure a range of viewpoints were captured.

1.4.4 The full methodology can be found in Appendix A.

Qualitative Research

1.4.5 Interviews were undertaken through a mixture of face-to-face (in February and early March) and remote methods by core members of the SYSTRA research team (and report authors). A discussion guide was used to ensure key topics were covered, whilst allowing the respondent freedom to introduce any other topics they considered relevant to rail performance.

1.4.6 Many techniques were used to encourage open and transparent answers and reduce the potential for biased responses. The independent nature of the research was emphasised, with great weight placed upon respondent anonymity.

1.4.7 With consent, discussions were voice-recorded, and a write-up of the interview produced. Verbatim quotes have been provided throughout this report, alongside stakeholder group identifiers.

1.4.8 As with all qualitative research, it should be noted that:

- While the views of a range of rail industry representatives are reflected in the research, the sample selected for this study is not statistically representative; and
- The views and opinions reported are based on statements made by participants, and are not necessarily factually correct.

1.4.9 If differences between different participant types are not highlighted, it can be assumed that the views expressed did not vary significantly by these characteristics.

1.4.10 In total, over 50 industry representatives were consulted with across 38 interviews. A summary of the interview breakdown by stakeholder type is:

| STAKEHOLDER TYPE | NUMBER OF INTERVIEWS | |-------------------------------------------------------|----------------------| | TOC Owning Group and TOCs | 20 | | Track and infrastructure | 6 | | FOCs | 5 | | Government and local government | 4 | | Trade associations, consumer groups and representative bodies | 3 | 1.4.11 It should be noted that the majority of interviews were completed with TOCs and Owning Groups. Whilst the first wave of interviews was more evenly split between TOCs and Infrastructure Managers, it proved difficult to secure interviews with the latter group in the second wave of interviews.

Quantitative Research

1.4.12 An online survey was distributed to representatives undertaking interviews and those unable to take part in interviews, with requests for the survey to be distributed more widely amongst teams.

1.4.13 The survey included a mix of open and closed questions. Responses to the open questions have been thematically analysed with the interview findings. Due to the small number of responses, closed questions have been reported as totals, as appropriate, throughout the report. Full data tables are available in Appendix C.

1.4.14 In total, 32 survey responses were received:

| STAKEHOLDER TYPE | NUMBER OF RESPONSES | |-------------------------------------------------------|---------------------| | TOC Owning Group and TOCs | 23 | | Track and infrastructure | 1 | | FOCs | 4 | | Government and local government | 3 | | Trade associations, consumer groups and representative bodies | 1 |

1.4.15 It should be noted that 7 responses were received prior to the temporary pause of the research, after which questions were added on the impacts of Covid-19, therefore some question response sizes vary.

1.5 Report Structure

1.5.1 This report is structured as follows:

- Chapter 2 summarises influence on performance at strategic level;
- Chapter 3 summarises influences on performance at operational level;
- Chapter 4 summarises the impact on performance of Covid-19; and
- Chapter 5 summarises the findings from the research to date.

1.5.2 The study found that stakeholders that held strategic decision making capacity tended to reflect different views to those making decisions in an operational capacity. As such, the report reflects these views in different chapters.

1.5.3 The report appendices are:

- Appendix A: Research Methodology
- Appendix B: Response to Delay Scenarios
- Appendix C: Survey Results

2. INFLUENCES ON TRAIN PERFORMANCE AT A STRATEGIC LEVEL

2.1 Introduction

2.1.1 This chapter provides an overview of the reported influences on performance-related decisions at the strategic level amongst the stakeholders sampled.

2.1.2 Strategic level stakeholders engaged with as part of this review, included:

- TOC Managing Directors/COOs;
- Performance and Planning Directors;
- Heads of Rail;
- Heads of Performance;
- Heads of Operations;
- Heads of Strategy/Policy/Regulation;
- Heads of Joint Performance Team between Network Rail and TOC;
- Client Services Directors; and
- Regulatory Managers.

2.2 Influences on Strategic Decision Making

2.2.1 At a strategic level, decision-making is focused on running the business, with the day-to-day running of rail services and activities being delegated to front-line staff, whose views and experiences are reported in Chapter 3. Train performance does, however, feature prominently in many strategic decisions at a planning and scheduling level. These include:

- Managing the impacts of new/updated timetables;
- Overseeing the operation, maintenance and renewal of the fixed assets along a route;
- Developing long-term plans for improving resilience of the network infrastructure;
- Overseeing the operation, maintenance and upgrade of services e.g. fleet and station improvements;
- Managing delay attribution disputes; and
- Defining service offerings at franchise bidding stage.

2.2.2 Train performance also features prominently in many strategic decision-makers’ perceptions of ‘a good day at the office’, ranking alongside new ways of doing things, but after safety considerations.

“[A good day is] when I feel like we’ve moved the business forward from future thinking perspective and we’re doing something innovative and new.” (TOC)

“After not killing anybody, it [train performance] is the second most important thing we do, because our business is built entirely on the needs of passengers, the ones we’ve got currently and the ones we’d like to have in the future, so operational performance is the number one influencer on passenger behaviour, satisfaction, repeat business, willingness to pay...” (TOC)

“[A good day is when] everyone is safe and our trains run on time.” [TOC]

“Safety trumps everything.” (TOC) 2.2.3 These spontaneous responses were to the interviewer’s initial prompt, and prior to specific questions on train performance per se. The responses suggested that, at a strategic level, train performance features prominently as their means of gauging their own company performance (TOCs/FOCs) or route/industry performance (Infrastructure Managers/Government).

2.2.4 Responsibility for strategic decision-making that directly affects the day-to-day outcomes of train performance rests with TOCs, FOCs, DfT and Network Rail. There are several important influences on strategic-level decisions, including:

- Safety;
- The customer experience;
- Contractual obligations;
- Train plans, including capacity;
- Revenue; and
- Risk.

2.2.5 The importance of each will depend on the specific circumstance but, in general, the main influence varied by type of organisation.

“There’s almost a three-way tension between customer impact (which is capacity); revenue (I also mean negative revenue as in refunds); and performance as well. And performance isn’t just of that train, it’s of all trains. They all pull in different directions.” (TOC)

“Sometimes decisions have to be made to upset a small number of people but to benefit a large number of people... when there are decisions to be made, we are protecting the really heavy loaded trains.” (TOC)

2.2.6 The remainder of this chapter looks at each of the key influences on train performance in detail, in descending order of impact on decisions that primarily relate to performance, as perceived by strategic decision-makers. Priority in ordering is given to internal company decision making, followed by perceptions on other parties’ decision making.

2.3 Customer satisfaction

2.3.1 For TOCs, a key influence on performance decisions is passengers’ views of their travel experience. There was a clear sentiment from some strategic decision-makers across TOCs that train performance is crucial for the success of their business, as company revenue and reputation is inextricably linked to customer satisfaction which, itself, is underpinned by passengers’ perceptions of service performance. There was an emphasis that sometimes passengers can, and do, vote with their feet. This was particularly considered the case by open access operators, who risk going out of business if they do not attract and retain sufficient numbers of passengers. Franchised operators may have some protection from this risk through revenue risk-sharing mechanisms (e.g. cap and collar), and for a number of operators, a significant proportion of their income will come from subsidy rather than passenger revenue.

“We are working in a market-place where passengers have choice, so our reputation as an organisation that delivers for passengers is enormously important.” (TOC)

______________________________________________________________________

6 ORR interpret this quote to be highlighting possible trade-offs between quality of service (in terms of reliability, punctuality and crowding), revenue, refunds and performance payments under Schedule 8. “If you stood at a meet the manager meeting, and had a passenger who was really, really upset and shouting at you because you're so badly affecting their life, that's probably enough incentive to improve performance... you can tell if performance has been good in the last 6 weeks because almost no one will come and talk to you, if performance has been poor, you may need to take the British Transport Police with you.” (TOC)

“There have been occasions where we've taken the decision in the short term to do right by customers but it has had longer term implications... we might put customers in taxis to get them home, which has a huge cost to us on the short term but what it does mean is that the trains might be in place for the next morning. So it's a short-term impact for the evening but the issue is resolved for the longer term impacts and customer satisfaction. You know it was on our Twitter last week, somebody had missed their connection at [station] and we put them in a taxi and they went on Twitter on said 'that's great'.” (TOC)

“And in financial terms we do make the wrong decision sometimes by trying to do the right thing. The ‘Beast from the East’ two years ago, we ran a full train service and everybody came out and cleared all the platforms and everything but nobody travelled because everyone else has said ‘don’t travel, this is a nightmare’. So the financial implications for us were huge, but [we think] we did the right thing by our customers. And we would make those same decisions again possibly, because its 90% of the time a customer driven piece. And from my point of view if you do the right thing by the customers, that is, in the long-term a more sustainable financial play as well.” (TOC)

“Our main interest is purely what delivers for the passengers... delivering a reliable, simple and sustainable railway... it’s all about how do we please our customers, how do we keep them coming back, and the fact that they pay us money to do it is great because we are a commercial business... we want to meet our franchise commitments and meet or beat our financial commitments and the revenue comes with it.” (TOC Owning Group)

“Despite the fact you have all these commercial aspects, nothing focuses people more than having a screaming customer in their face, or you know people threatening to take them to court, or writing horrific letters to more senior people in the company, so there is seeing the white of people eyes, and people not wanting the hassle, and wanting to deliver themselves.” (TOC)

“Our open access contract, we are incredibly customer focused there. It matters hugely how we are performing because that is our bond, that is our promise to our customers, and we don’t have the typical protections that a contracted operator might have... if we don’t perform, our customers go somewhere else and we can’t survive.” (TOC Owning Group)

2.3.2 In addition to this, Delay Repay (or equivalents) means compensating customers for delays, regardless of the cause.

2.3.3 Delivering a service that meets passenger needs is a clear focus of TOC representatives, however there were varying opinions on what this should look like in reality. In general terms, doing the best for customers was described as delivering services to time, overcoming disruption with minimal impacts and providing good customer information. For some, doing the best for customers was described as doing the best for a large proportion of customers, with disruption inevitably caused to a smaller number of customers. 2.3.4 There was disagreement over what constitutes the best way to meet passenger needs, including which metrics best meet passenger needs; there was disagreement between TOCs and consumer advocates. These views included:

- The ‘on-time’ metric is thought to be perverse as it incentivises prioritisation of stopping services(^7), rather than services that may be carrying more passengers with higher time sensitivity requirements;
- Good delivery for passengers means arrival within five minutes at destination, as this is the time period that could affect ongoing journeys or plans; and
- Punctuality should only be seen as one facet of customer satisfaction, as 90% PPM for a commuter will still mean delays on at least one journey per week.

“We’re looking to ensure we run a reliable and punctual service. It’s really all about customers... because we’re a commercial business... [we try] to provide the biggest number of people with a punctual and reliable train service.” (TOC)

2.3.5 A need to maintain and improve customer satisfaction influences decisions in relation to service recovery and investments in enhanced capacity and reliability.

2.3.6 Amongst strategic decision-makers at infrastructure organisations, the focus was also on train performance but, seemingly, more from a process-driven perspective – reflecting that their contribution affected their rail-industry customers, rather than the end-customer. Their strategic ambitions were set more on developing better ways of monitoring and understanding issues affecting train service performance with the aim, ultimately, of improving it.

“Most of my work is around attending meetings, looking at strategies, programme plans, sign-off of reports ... it could quite conceivably include performance on some days but not on others.” (Infrastructure Manager)

“We try and influence the design of Schedule 8 and to try and make sure that that drives the right incentives for performance improvement and good behaviours with regards to performance.” (Infrastructure Manager)

“Signalling has to be at the centre of performance management, and at the moment it’s not even part of performance management.” (Infrastructure Manager)

2.3.7 TOC representatives expressed some concern that, overall, Network Rail does not have the same passenger focus, despite some examples at a regional level that it does. Reasons for this include:

- Disparity in TOC and Network Rail targets); specifically concern that Network Rail’s performance target is perceived to be easier to achieve than similar targets for TOCs. It was also noted that Delay Repay (or equivalent) means the TOC compensates the customer, regardless of the cause of delay. As a result, Network Rail is thought by some TOCs to have little incentive to deliver high performance. A possible solution to this suggested by TOC and Owning Group representatives

______________________________________________________________________

(^7) The on-time metric records whether or not a train was on time at each station stop. Therefore a stopping train will be recorded more frequently than a fast services. The metric is not weighted according to demand or service type. was thought to be some form of end-customer performance metric for Network Rail; and

- Network Rail is potentially not perceived to be responsible for poor performance by the general public, as they do not have a contractual and financial relationship with passengers like TOCs do. This can mean that, in some TOCs’ views, Network Rail does not take responsibility for meeting the needs of the end-passenger, despite the fact that many delays experienced are their direct responsibility.

“In CP6, Network Rail’s targets are very achievable, they are not just achievable, they are smashable! Effectively, the Regulator allowed Network Rail to set its own targets and Network Rail invested a huge amount of time and resources in manipulating the setting of those regulatory targets and making them very, very achievable... Network Rail are effectively being rewarded for below par performance which puts financial pressures on TOCs and we get fined in three different ways for this: one, we end up paying Schedule 8 for performance we do not consider acceptable; secondly we have to pay our customers for delay repay; and the third way is through Schedule 7.1... Network Rail pay once.” (TOC Owning Group)

“I think that everyone at Network Rail should have some kind of end customer KPI.” (TOC)

“We, as the operator, are always going to be the person they buy the ticket from and the person they have a contractual relationship and emotional relationship with, and we only cause, directly, 30-40% of their delays but we have to front up and take the responsibility for all of it. It would be easier, sometimes, if the Network Rail corporate PR machine was a bit more willing to stand up and be counted.” (TOC Owning Group)

2.3.8 In the rail freight industry, customer satisfaction is an extremely important influence on train service performance decisions, inasmuch that FOCs wish to provide high quality services that encourage ongoing custom, in order to maintain their bottom-line. This is reflected in FOC KPIs on service delivery, and also the FOC’s service offer, which is typically tailored to different freight customers recognising that sensitivity to delays varies by customer and commodity type.

“We have a company KPI on the delivery of our [redacted] services to the customers’ location. That is on the same basis as the ‘Arrival to 15’ freight measure.” (FOC)

“Our customers are with us because we don’t stop... many of our customers are supermarket chains, it’s next day delivery stuff, if it doesn’t arrive the supermarket shops are shut... it’s customer satisfaction.” (FOC)

“We’re commercial businesses, so what drives a commercial business? The bottom line... For our customers, they have genuine mode choices that they can make, they can either deliver by rail or they can deliver by road... For the customer, you want to know that the railway is consistently reliable, that your goods arrive when you want them to arrive, day in, day out. Precisely how you define that reliability... will be unique by customer... if you are one of our customers who move construction materials by rail, you will have a terminal in the city centre which will be supporting local builders... the thing that you hate the most is a cancellation... you have a building site that is then

______________________________________________________________________

8 Stakeholders who put forward this suggestion did not elaborate on what it might look like. 9 ORR has noted to us that Network Rail was required to work with TOCs in setting performance trajectories for CP6. 2.3.9 For strategic decision-makers at ROSCOs, the focus is similarly on keeping their customers happy to ensure follow-on business. There are no contractual incentives to ensure a certain level of reliability, but maintaining high levels means they are more likely to secure future business by contributing to their customers’ (i.e. TOC’s) reliability. Hence, at a strategic level, ROSCOs must work collaboratively with TOCs to support them and ensure their rolling stock does not adversely affect TOCs’ train performance.

“What matters to us is having trains with a good or high standard of reliability so people will lease them again in the future... there’s nothing that obliges us to certain levels of reliability.” (ROSCO)

2.3.10 Strategic decision-makers in government share the same view as TOCs, that train performance lies at the heart of public perception of the service provided by the rail industry; and if the public are happy, then so are the politicians. The view was expressed that government is paying for a huge amount of the railway system, and so needs to ensure that the money is spent in a way that maximises service performance within the financial settlement available.

“I think there is a very, very clear public narrative that performance and punctuality is what matters to passengers and what matters to passengers tends to matter very, very much to politicians.” (Government/Local Government)

2.3.11 The perspective of the consumer advocates is similar to that of government – very much believing that train performance is the key influence on public perception of the industry, and that even a minor delay (e.g. 1 minute) has repercussions on customer satisfaction.

2.4 The franchise bidding process and franchise agreements including Schedule 7.1 influences

2.4.1 On the GB Rail network, most passenger services are delivered by franchises. Owning groups compete to operate franchises specified and awarded by DfT. The commitments expected of the winning bidder are specified in a Franchise Agreement. Schedule 7.1 of the standard Franchise Agreement sets performance benchmarks with regards to cancellations, TOC minute delays and short formations, with a rewards and penalties regime based on actual performance.(^\\text{10})

2.4.2 There was a feeling amongst some strategic decision-makers at TOCs that the industry should focus on ‘brilliant basics’ rather than ‘over-promising and under-delivering’. They blame TOCs’ over-optimistic performance proposals on the nature of the franchise tendering process, and the basis on which DfT evaluates the bids and makes their award. The result being that some TOCs feel they are under intense pressure to increase the number of services run in order to generate higher ridership and increased revenue, in a context where Network Rail is obliged to accommodate such requests ‘where feasible’ -

(^{10}) See for example [https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/664303/sf-itt-attachment-b-franchise-agreement.pdf](https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/664303/sf-itt-attachment-b-franchise-agreement.pdf). Pages 275-315 set out the payment regimes, Page 316 onwards sets out performance benchmarks. thereby putting further pressures on an already capacity-constrained system. Moreover, there is unlikely to be much contingency in the successful franchise bidder’s assumed resources to run the services, thereby leading to a cost-efficient, but in practice increasingly non-resilient, rail service.

“[Part of my focus in on …] ensuring the forward-looking timetables are fit to deliver performance, as per our franchise contract.” (TOC)

“We have to increase our trains, increase the capacity on trains. We’ve got a huge new fleet introduction across all our TOCs because we have to provide the extra seats for the extra capacity [we need]. You’ve got to balance out the costs with the revenue… in an ideal world you put a longer train on because you’re paying for the path, the electric, the diesel… if you can do it with one train it’s better. If you do it with two trains, you’re paying the additional access costs, additional resource costs and therefore you’ve got to earn more so you have to balance them.” (TOC)

“The franchise bidding process encourages operators to run as low cost a service as they can. They are almost not allowed to have spare train crew and spare rolling stock as that would be a cost and if they put that into their franchise bid then they wouldn’t have won the franchise because another operator would have said, ‘we can do it without that!’… there is no resilience. And if they do put that resilience in after, then it’s eating away at their thin margins and it’s not consistent with their bid… it’s not funded.” (Infrastructure Manager)

“In some of the bids, the enhancements we put in are above the contractual requirements… it’s a sales pitch… It’s supposed to work but you could argue that when you put your franchise bid in, you don’t know what is around the corner. A franchise lasts 7 years, that goes across a control period, I don’t know what is going to be available in the next control period.” (TOC Owning Group)

2.4.3 The TOC’s reported view that Network Rail were obliged to accommodate TOC requests for additional services at franchise agreement stage was, however, confirmed by one infrastructure manager but disputed by another.

“When we come to producing the joint timetable, the operators will, understandably, be wanting to make sure that they’ve got a timetable that runs the number of services they have been told they must run by DfT or Transport Scotland through their franchise agreement. We just have to allow the services to run if they physically can… we can highlight performance concerns but, ultimately, we have to allow that service to run. What that means is that there is not really much consideration about what having additional trains on the network will do to performance. There has been no analysis to work out - is it better to have that one extra train on the network and let performance fall a bit, or is it better to not run that train and have a better performing service? A recent example is West Midlands Trains.” (Infrastructure Manager)

“We have to apply the Network Code which is a set of rules about writing a timetable. So, at the minute, we can reject applications for track access rights if we think the performance impact of those services would be too adverse. That’s quite key to how we work. We want to have a high performance timetable as well as balancing that how with running as many trains as we can – it’s the capacity vs. performance trade-off.” (Infrastructure Manager) A note on the Franchise Process

At a strategic decision-making level, many TOCs indicated that a key source to many of the day-to-day problems running the railways was over-optimism of franchisees at the bidding stage. The suggestion is that the franchise tendering process encourages bidders to have over-optimistic business plans, including delivering additional services on a network where capacity is close to being fully utilised, for as low a cost as possible. A resilient train plan, developed at bidding stage, was considered important to delivering good performance, with lower resilience correlated to poorer performance.

Strategic level staff at TOCs see this as a fault of DfT and Network Rail who outline service specifications that are not deliverable but are required to be bid for in order to be compliant. Therefore, some train performance problems are perceived to be caused ‘at source’ i.e. through over-commitments in timetabling (e.g. May 2018 timetable disaster). Additionally, TOCs note it can be difficult to develop franchise bids for franchise agreements which go over 7 years, including changes in Network Rail control periods, as they cannot anticipate future use of rail or changes in the industry. Some non-franchised TOCs and Network Rail note that this means that some TOCs are under intense pressure to increase the number trains they run in order to meet increased revenue targets. Schedules 4 and 8 may be perceived as ‘insurance policies’ to fund revenue shortfalls, with some TOCs effectively finding that their business plan relies on Network Rail providing poor performance on their routes to meet their revenue targets.

Strategic level staff at Network Rail report that their input into the timetabling process at bidding stage has risen compared to the past, and hope to have better aligned targets in the future. This does not resolve current issues, however, and the scope for future trade-off between capacity and performance is not clear. Network Rail also does not see the current franchise process as incentivising investment in performance initiatives such as staff/driver training.

The Williams Rail Review is likely to see changes in franchising process and a number of new models which could better incentivise performance were suggested by stakeholders, including a concession model and a more devolved model.

2.4.4 Local and national government departments also felt that franchise agreements are influential on on-going challenges for TOCs to deliver reliable train performance.

Schedule 7.1 Influences

2.4.5 Breaches in franchise agreement obligations (such as Schedule 7.1) were noted as potentially far-reaching for a TOC, potentially resulting in:

- A loss in revenue;
- A need to make public statements which can be reputationally damaging;
- The franchises being terminated early; and
- An impact on their future franchise bids.

2.4.6 Mixed views were reported on to the benefits of performance targets set within franchise agreements:

- Some TOC representatives suggested that the obligations create strong financial incentives to deliver and thus highly influential on decision making at strategic level and carried through to operational delivery; this could lead to decisions being taken that prioritise meeting targets (see quote below for example) rather than prioritising customer needs. Others suggested that obligations tend to take care of themselves and therefore are not themselves the key driver of decision-making:

- A local government representative argued the financial incentive would override other influences on decision making, while other respondents from local government and non-franchised TOCs reported a view that targets are not ambitious enough with regards to performance;

- One TOC Owning Group suggested that franchise agreement targets are arbitrary, and therefore potentially de-incentivising.

2.4.7 The dissatisfaction with service specifications, defined in the franchise bidding process, was reflected in the survey results, as the most frequently selected intervention to change, to achieve a more positive impact on train performance.

“We’ve got a franchise, we’ve got a contract, that contract was signed off by the DfT and awarded to us so we want to adhere to that… whatever is in our franchise obligations, we have to do… if you don’t, you lose the franchise, a la Northern.” (TOC Owning Group)

“The main influence is about delivering franchise targets, and then if that doesn’t happen, there’s also the aspect of how much not delivering is having a financial impact on your company.” (TOC)

“There’s some key metrics in our franchise commitment… for delay minutes and cancellations. If we’re continuously not performing very well, our annual moving average may well breach the threshold agreed with DfT. You may well get a slap on the wrist to say don’t do it again, you may well get a hefty fine, and worst case scenario, if you are operating badly, is you have the keys taken off you and the franchise is taken off you… when we have been close… we have to go in and meet with the DfT to talk about what we’re doing about it. In those extreme cases, we may ask the control team to do something different. [If we are approaching a breach in the cancellation benchmark] it may then involve if a train is running late, run through to destination, so it’s not counted as a cancellation… it’s not ideal for customers but we have to keep the interests of the business [in mind].” (TOC)

“Not only does [Schedule 7.1] have quite sharp financial teeth, it’s reputationally damaging for me, or any of the metrics, to fail. I’ve had to give evidence to the transport select committee to apologise and explain, that certainly does drive behaviour. In addition to that, if you fail more than once, and the government decides that it is not happy with the remuneration that you put in, it can terminate your contract… it is a pretty strong motivator. In addition to that, every time I breach a contract, when I want to bid for another one, I have to report that I have breached and send the DfT a letter explaining what happened and why and they reserve the right to take that into account when they award future contracts. Those are pretty powerful motivators for not wanting to breach [performance] targets.” (TOC Owning Group)

“What they come to work to do is to deliver that Franchise Agreement… it measures very different things than what the rest of the industry is measuring… it measures

______________________________________________________________________

11 In considering these different views it is worth bearing in mind that each TOC’s performance targets are separately calibrated. The significantly differing views expressed may be a reflection of the effectiveness of the calibration process from TOC to TOC. cancellations but they have to be dire before anyone is doing anything. It measure short-forming in a way that I haven’t quite got my head around... even if they operated every train according to that plan, it’s still not good enough for customers. It doesn’t incentivise good enough performance because the benchmarks are too low... they are there to show what really bad looks like, not what really good looks like.” (Local Government)

2.4.8 There were concerns raised by strategic representatives at TOCs, Owning Groups, local government, and Network Rail representatives over the target setting process at franchise bidding stage. These include:

- Bid evaluation scoring focuses on finances (cost saved and value added), without an operational understanding of how they could be delivered;
- Punctuality targets are not aligned between Network Rail and TOCs (e.g. PPM vs right-time vs average delay-minutes), nor are the terms in which they are set (control periods vs franchise agreement terms);
- TOCs are encouraged to develop their bid response in isolation and they are therefore not aligned across operators, including where parts of the network are shared. This can result in different views on what level of delay minutes are acceptable even on the same part of the network.

2.4.9 Other concerns raised, primarily by TOCs and Owning Groups, over targets developed at the franchise bidding stage include:

- Targets, both for performance and more generally, are set for the entirety of the franchise agreement, which could extend to ten years and, whilst achievable at the beginning of the term, changes to the network, changes in other TOC’s agreements, and unpredicted events make them harder to achieve over time; and
- The bidding process involves a high degree of commercial confidentiality that can hinder best practice development during the bid development period, such as learnings specific to the network area being bid for.

“The root of all evil is there... people are saying the franchising system is broken. The biggest reason why it’s broken is because of the nature of how those competitions have been run. You’re incentivising the operator to optimise the revenue line... to squeeze timetables, optimise resources, run more trains, flood the network... DfT are quite happy to take that because from a Treasury point of view, it’s fantastic... from a network output perspective, it doesn’t achieve the right thing.” (TOC Owning Group)

“We just all need the same ultimate goal or incentive... as a train operating company, we are incentivised by the department on reducing delays and cancellations and train length. Network Rail aren’t incentivised on train length, they are not incentivised on our level of cancellations. If the industry could all agree on one set of objectives for performance, it would be a lot better... you sometimes feel that you come to work in the morning, and you’re trying to achieve different things.” (TOC)

“When people bid for a franchise, they often commit to do really crazy things, to win the franchise, knowing that some of them are quite unachievable.” (TOC)

“Franchises and targets are all set in isolation [sequenced rather than coordinated] and it’s not until you put all TOCs together, and all different TOC contracts together, that you realise it’s never achievable.” (TOC) Timetabling Influences

2.4.10 Strategic representatives at TOCs and Owning Groups noted high capacity utilisation of key sections of the network and the mixing of commuter with long-distance services can compromise performance delivery. A number of suggestions emerged from the interviews with strategic staff at TOCs, Network Rail and industry/passenger representative groups, to make high-capacity train plans more resilient. These include prioritising infrastructure investments to improve system capacity so that future plans can be more reliable; having resources in key strategic locations; building realistic terms for train drivers with recognition of actual obstacles to reliable train planning (e.g. toilet breaks); and including non-standard dwell times, dependent on time of day.

“The train plan for railways is of enormous importance in terms of delivering performance outcomes. The train plan is an exam paper and that is the exam that you take every day, and you get marked on it. You set your own paper so it’s not as though it comes as a surprise... if you put the thought in at the front-end, you can make it intrinsically easier to deliver and more robust and better for your customers.” (TOC)

“As an industry, we continue to over-commit and under-deliver, with an apparent inability to recognise risks early enough to take mitigating actions. There are more timetables being re-wound to a previous stable state than there are being moved forward in improvement positions.” (TOC)

2.4.11 TOC and local government representatives highlighted perceived tensions in the timetabling process at bidding stage; that the specifications set by DfT in the Train Service Specification can be misaligned with Network Rail’s ability to deliver, and that Network Rail may not scrutinise and challenge TOC’s overpromises sufficiently to provide confidence that they are deliverable. Local government representatives also raised the difficulty for Network Rail in balancing competing requirements from TOC specifications and DfT requirements, suggesting that TOC requirements may not take into account the performance of the rail network as a whole 12.

“Network Rail try to facilitate and agree timetables that they wouldn’t necessarily agree to if they were aware of the performance impact. They quite often do it to be helpful to meet the requirements of the specifiers and train operators. There’s a tension there between doing things because the operators are specified and obligated to deliver, and the reality of actually being able to do it. I don’t think there’s enough performance modelling done on the effects from specification right through to Network Rail going through and admitting those into the timetable... it’s quite an antiquated approach, and quite manual and labour intensive. There is not enough modelling... I think it’s a failing that Network Rail don’t have a standard way of easily modelling the performance impact of things.” (Government/Local Government)

“Timetable changes should be more thoroughly evaluated to understand the impact they will have on the network and industry. This should include proposed stock and crew diagrams contained in the Franchise Bid to ensure that they are realistic and deliverable. Nobody will ever win a franchise bid unless they promise to do more, run more trains and make their fleet work harder to deliver greater efficiency. But therein lies the

12 SYSTRA was unable to speak to Network Rail representatives on timetabling in the second wave of research interviews to obtain their response to this viewpoint. 2.4.12 Despite the expressed view from a Network Rail representative that timetable pressures do not arise from franchise agreements, they are, nevertheless, becoming more proactive in working with DfT and bidders on future specifications to ensure feasibility. Senior representatives at Network Rail suggest that there is a desire to improve the industry process for creating new timetables, not least in response to the significant negative media and ministerial interest. This is especially in the context of the problems caused by the May 2018 timetable change; this led to the development of the Timetable Assurance Steering Group aiming to review infrastructure, rolling stock and route/operator readiness for future timetable change.

“I’m well aware that performance has reduced quite considerably in recent years and there’s a massive desire from everybody in the industry to try and recover performance. That’s informing lots of discussions and work streams at the moment. It gets commented on by ministers, it gets in the newspapers, it’s quite a big impact on the reputation of the industry. Take the issues in the north of England, we know train service delivery in and around Manchester, in particular, is not up to scratch. Back in 2018 there was a catastrophic failure of the timetable which resulted in horrendous performance. It was causing real severe distress to the industry in terms of reputation... it’s certainly having an impact at the top. The Timetable Assurance Steering Group we put in place oversees risks and issues of a future timetable.” (Infrastructure Manager)

“Over the last couple of years, Network Rail is increasingly working with the regional colleagues, to give input to the DfT on re-franchising competitions, what we think of those and if we think the train service specification actually works... that’s something that we’ll do more of.” (Infrastructure Manager)

2.4.13 It was observed by one strategic TOC representative that, in contrast to a TOC’s narrow remit of its own route and customers, Network Rail has a national remit and has the opportunity to take a more holistic approach to solving known disruption hot-spots on the network. In some TOCs’ view, it is Network Rail’s failure to find local and regional solutions to known network problems, despite having the necessary resources (as the TOCs perceive), that leads to the majority of day-to-day delays to services. An example highlighted was the ‘known’ timetable congestion at York that regularly leads to certain Northern trains always being delayed, with nothing that the TOC can do about it.

“The inability of Network Rail to address repeated hotspots for reactionary delay, my trains get caught up in that.” (TOC)

2.4.14 The industry regulations affecting timetabling, and the local processes for regulating out-of-course trains at key locations, were also seen to have direct adverse performance outcomes for freight services (see Error! Reference source not found.) and local/smaller services, as well as indirect reactionary delays on passenger services.

“Most of the time, if there is a timetable clash on the network then a freight train gets put into a loop and the passenger train goes. Passenger gets priority because they have passengers on them. But, actually, if you think about the dynamics of stopping a freight train; the fuel demand; and the loop which it goes into might have points which require you to come out of them at 15mph - there are some performance considerations across the network! If you actually stop the passenger train and let the freight train go, you might have had a better net capacity and performance - but the regime doesn’t let anybody look at that. I think it’s about timetabling... there is an absence of strategic capacity development, we tend to just do the same thing.” (Consumer/Trade Association)

2.5 Individual motivators

Personal Pride

2.5.1 Representatives at TOCs, FOCs and Network Rail highlighted that feeling pride in their work and operational performance was a key motivator for staff at all levels, and indicated that this drives the delivery of a high performing train service. Developing a good personal and organisational reputation for performance, which could be reflected in league tables, was considered very important and linked to customer satisfaction (and therefore revenue) as well as franchise outcomes for TOCs.

“We’re very proud of our punctuality and reputation. We are top three in the country for performance. That’s something people get a great deal of pride from.” (TOC)

“We pride ourselves on having that delivery of performance... there is a professional gratification knowing that we have outperformed everyone else... our performance and our ability to deliver for our customers has been at the forefront of the expansion of the business.” (FOC)

Performance related pay

2.5.2 There is limited evidence from the interviews suggesting that performance related pay or bonuses impact decision making at Infrastructure Managers or TOCs. Often, performance is only one element of a bonus scheme and there is a perception that little can be done at an individual level to influence it. Occasionally it may be used as reminder in an attempt to help motivate employees to continue to do a good job, but it was not clear from the comments received that this has a material effect on actual train service performance. One FOC representative did believe that the performance related bonus scheme was influential in decision making.

“At group level, there are 5 KPIs that we monitor the business on at a strategic level, one of which is operational performance, and our group chief executive has a proportion of his pay that is derived from how we achieve those KPIs... individual executives are then set targets as part of their objectives... what drives them isn’t the piece of paper or financial flow, which is relatively small, it’s the reputation – everybody wants to be, or work for, the best performing rail operator... that’s what drives a railway career... it’s a bit like joining the NHS, you do it because it’s a very tangible public good... I think it’s what’s driven most of the improvements over the years, not any of the synthetic incentive regimes.” (TOC Owning Group)

“Our performance bonus scheme includes an element which relates to train performance. It focuses everyone on working with [Infrastructure Partner] to deliver, but the frustration though is we are so far removed from it that we don’t make decisions on how to deliver it.” (Infrastructure Manager) “We do have part of the company’s bonus regime paid dependent on how well we perform on [our KPI for delivering services to customers within 15 minutes of schedule]... this is probably about 0.1% of 1% for me. It is of no significance whatsoever. You can’t influence it in any case because the performance statistics that are produced are generated by Network Rail and they are audited independently before those bonuses are paid... It’ll be what it will be, it certainly wouldn’t influence any behaviours across the business.” (FOC)

“Network Rail, as a company has a corporate level scorecard, and the different functions and regions have got different scorecards and performance features quite heavily on those scorecards... That will affect people’s achievement of performance targets, as well as driving directly into bonus payments. I don’t think I come to work each day thinking, ‘oh, if performance is a bit worse, my bonus is going to go down by £X as a result of that.’ But, from time to time, when we review the scorecard you do take stock if the number is going up or down and what’s driving it... you are aware that that is going to have an effect on your bonus at the end of the year and it is something, as a senior manager at Network Rail, I make a point of for the team.” (Infrastructure Manager)

“Our organisation has a bonus schedule, depending on reaching the certain targets. So if we’re performing poorly... we will not be paid bonuses... they continually influence performance.” (FOC)

2.6 Media and political influences

2.6.1 Strategic TOC representatives noted that whilst maintaining good relationships with media, MPs, and consumer advocacy groups was important, it was likely that actions were already being taken to resolve performance issues before they were raised by any of these groups. A TOC representative also noted that action taken to appease stakeholders, may not be the best decisions to make for overall performance outcomes.

2.6.2 A TOC representative noted the importance of maintaining good relationships with unions, such as working in line with rest days, overtime and strike allowances, all of which impact performance. However, it is not always possible to agree to union requests, as this could have negative performance, safety and cost outcomes.

2.6.3 TOC Owning Groups in particular emphasised a need to keep their internal stakeholders (e.g. governance boards), as well as influential third parties, including the media and politicians, on-side.

“You don’t want a high profile stakeholder saying unpleasant things about you. Sometimes, as much as it grates me to say, we do dance to the tune of one or two individuals who are high profile. Sometimes that is at the expense of large volumes of the travelling public.” (TOC)

“You’re in a goldfish bowl with the media, you’re in a goldfish bowl with the politicians. It’s almost like they smell blood! If they think there is an issue, they will go for you and it takes a long time to get off their radar. It’s nice to be under the radar screen. We’ve been all over the media in the [Region] because our performance was awful. We accept that, we apologise... Does it focus us on improving? Yes it does, but our own internal professionalism focuses us a lot more.” (TOC Owning Group) 2.6.4 TOC representatives also noted that they were more likely to receive poor media attention for e.g. timetabling issues, than other parties such as Network Rail.

2.7 ORR monitoring and Network Rail Scorecards

2.7.1 Strategic stakeholders at Network Rail and in local government perceive ORR’s Network Rail Monitor, including the use of Scorecards, as having some influence on performance decisions within Network Rail. However, for others in Network Rail, Scorecard targets are secondary compared with individual pride in their work.

“I think the fact that the ORR comment on [Network Rail] performance, and we’ve got a performance order from the ORR, certainly does have an impact. It’s not the strongest issue but it’s not trivial.” (Infrastructure Manager)

“There are two ways of approaching it, you either come to work to say ‘I will do my best regardless, whether I’m doing well or badly against my target’ or you say ‘I’m motivated by target’. As long as everyone is trying their best, the target is just there in the background.” (Infrastructure Manager)

2.7.2 Strategic stakeholders at FOCs suggested that Scorecards have little impact on Network Rail performance as Network Rail was believed to focus their performance related decision-making primarily on senior level staff’s priorities.

2.7.3 Strategic stakeholders at TOCs were split in their views, some felt that Network Rail Scorecards have some importance to Network Rail decision-making and others argued that the Scorecards did not incentivise good performance for Network Rail. There was also some desire from TOCs, and passenger/trade representatives, that the Scorecard metrics should be strengthened, including:

- Ensuring that the metrics are not able to be manipulated by Network Rail, with the assumption that this will affect Network Rail employee bonuses;
- Encouraging better maintenance of assets; and
- Introducing a ‘right-time’ metric.

2.7.4 Some TOC representatives did not feel that ORR’s monitoring of Network Rail was strong enough, and that they did not see the benefits of their efforts in agreeing the Scorecard KPIs and targets. This was also reflected in the survey results (primarily TOC respondents), where ‘ORR’s monitoring and enforcement on Network Rail’ was most frequently selected as having a negligible outcome on performance.

“I’ve seen some very odd behaviours driven from the Scorecard. Back in February/March last year, there was a massive drive to remove lots of Temporary Speed Restrictions (TSRs) because they were over target, and it was a Scorecard measure to try and keep them under control. We had 10 periods where these TSRs were over – and, as it got closer and closer to Scorecard, and [because] it was directly related to bonus - suddenly there was a lot of attention on TSRs. A whole swath of TSRs were removed. And on the 10th April they started to come back on again in the same places! There is a degree of game playing... we have a lot of supplementary customer measures, which are not directly related to a bonus, and they were not paying the same attention to those. On one level that does prove that Scorecards and personal financial incentives work, but you’ve just got to make sure that you’re measuring the right things and that the determination on whether the bonus is paid or not is done over a balanced rate on an entire period.” (TOC Owning Group)

“We would like [ORR] to be a bit more abrasive than they are - a bit more demanding of Network Rail, and perhaps a bit less trusting that they’ve got it all under control - particularly around the asset condition, maintenance regime.” (Consumer/Trade Association)

“We have pushed for the ‘All Stations, Right-Time’ metric to be a key part of the Scorecards. We want ORR to dictate, and it was ORR’s view that it was for Network Rail’s customers to decide what’s important for them.” (Consumer/Trade Association)

“One of the weaknesses of the last few years is that Network Rail regulation has been quite weak... there is limited evidence that Network Rail is busting a gut to drive performance improvements.” (TOC Owning Group)

“I don’t see Network Rail being influenced by what’s agreed on their Scorecard. There is no real incentive... they need to be better regulated to how they are currently and held to account.” (TOC)

“I find with Network Rail, I certainly don’t see anything that actually says, right now we’ve agreed to this figure, we’re going to do all this to actually deliver this figure [...] I do not see at the delivery level it actually making any convenience, I see it as a hellish inconvenience when you’re actually trying to set up realistic and grown-up and challenging targets, so that’s the scorecard for me.” (TOC)

2.7.5 Strategic stakeholders in the rail freight industry tended to feel that Network Rail and ORR do not have much control, or say, over rail freight performance. Reasons for this included:

- It is reported that scorecards only have one measure related to freight performance and two measures are required to fail in order for an action to be taken; and
- The rail freight industry is purely commercial and so is not regulated by ORR. ORR could, however, publish total freight performance data; this could provide more scrutiny for the industry, and better-incentivise good performance decisions.

“I’m not sure that [ORR] have much influence over Network Rail. ... Any interjections of a failure, according to the Scorecards, can only come about after failure of more than one performance criteria - and find me two freight performance criteria – there isn’t two on the Scorecard! How can the ORR take action against Network Rail in relation to freight performance?” (FOC)

“ORR don’t report on total freight performance. Freight businesses are not going to allow themselves to be regulated on total performance because they are commercial businesses. In my view, it probably should be published - it wouldn’t do any harm, it’ll just be a little bit more scrutiny.” (Consumer/Trade Association)

2.8 Schedule 8

2.8.1 Schedule 8 of the track access contract (the ‘performance regime’) outlines the rules for payments in relation to delays due to unplanned disruption. It is intended to provide a financial incentive to address issues affecting performance; and provide insurance to TOCs when making revenue commitments when bidding for a franchise. 2.8.2 There was recognition across all stakeholder representatives that, in principle, financial compensation when performance fails due to another party was necessary. The delay attribution process, through which responsibility is assigned for delay incidents, helps to support Schedule 8 but is also seen as important in its own right. There was a desire for accurate recording of delay incidents, including the ability to ascribe a cost per minute of delay, and the capability to investigate the data for trends so that performance related investment decisions are correctly made. Examples were provided whereby Schedule 8 data was used to justify internal business cases for fleet improvements, and reference to its use in external business cases to apply for funding e.g. the Performance Innovation Fund) through NR.

“Where I do see Schedule 8 has a purpose, is when I am bidding for a contract and I’m being asked to take revenue risk. The biggest single driver of revenue performance is operational performance and I see Schedule 8 effectively as an insurance policy that Network Rail will be held to account to deliver their performance targets.” (TOC Owning Group)

“I think Schedule 8 is an extremely clever regime. When you’re bidding, Schedule 8 is the only protection you’ve got against your predictions for Network Rail performance not being there.” (Consumer/Trade Association)

“I’ve heard people say we should abolish it and I find myself thinking, really?! Are you really saying that we shouldn’t compensate for loss if Network Rail fail to perform?” (Consumer/Trade Association).

“Say you, for example, you have a problem with a train, some kind of defect and in order to improve that defect you need to switch out a part on it but that part isn’t due for renewal any time soon, so you have to make a case to switch it out sooner... let’s say that part costs £200 per train and you’ve got 55 trains, and that part has only cause you a 6 minute delay, you’re way off the cost benefit... whereas, if switching out that part saves you 10 minutes’ delay every week, it’s simple maths, it’s a no brainer.” (TOC)

2.8.3 However, there were wide-ranging concerns that Schedule 8 fails to deliver on its intended purpose, resulting in perverse incentives relating to performance decisions, and is used for purposes beyond its intended effect. There were also frustrations expressed at the time and resources dedicated to allocating delays, which could otherwise be spent resolving the causes of delays.

“As an industry, we spend too much time worrying about whether this train here delayed this train here, and this train here. We employ armies of people to go to the minutiae. We should be focusing more on root cause. Schedule 8 does create an unnecessary conflict and unnecessary workload that the industry could do better without.” (TOC)

2.8.4 Views expressed by Network Rail, non-franchised TOCs, government and trade/passenger associations suggest that some TOCs use Schedule 8 to maximise their financial gain, resulting in poorer performance outcomes. TOCs are perceived by Network Rail as actively dis-incentivised to assist in the resolution of delays because, as the delays accrue, so do the associated payments.

13 In June 2019 ORR started a review of delay attribution: https://orr.gov.uk/rail/publications/economic-regulation-publications/delay-attribution-review “A DfT franchise - you make all sorts of contractual agreements but they do not have huge financial consequences whether they are delivered or not. You bid, based on a level of subsidy from government, if that proves to be insufficient you lose a lot of money. So, it’s ensuring you get enough subsidy out of the government to produce profit and - if you’ve got shortfall on the calculation of the subsidy you receive - you then look at everything you can to make up the shortfall. The best place you can go is Network Rail, and you’ve got to hope they fail continually - so it’s perverse, not directly in the customers’ interests. Over the last year, Network Rail paid something like £4 million(^{14}) to the train operators as a compensatory Schedule 8.” (non-franchised TOC referring to franchised TOC)

“There’s a debate about whether it [Schedule 8] acts as compensation or an incentive regime.” (Infrastructure Manager)

2.8.5 Strategic representatives at franchised TOCs strongly rebuffed this assertion, stating that passenger satisfaction was an absolute priority, and that, in their view, revenue loss from dissatisfied passengers, due to delays, far outweighed any financial gain to be had through Schedule 8 payments. They also noted that those making decisions during delay incidents were unaware of any financial consequences to their actions.

2.8.6 TOC representatives suggested that Schedule 8 generates decisions within Network Rail that differentiate between operators, and make other prioritisation decisions, due to differing payment penalties(^{15}). TOC representatives also noted that Network Rail was seemingly more proactive in service recovery when the delay is their fault under Schedule 8; meaning it does influence their decision making. However, there was desire for more holistic regime which incentivises action from all parties, regardless of cause.

“We drum it into a control team ‘don’t worry about the financial sides of decision making’. We tell them, ‘do what’s best for the customer’.” (TOC)

“Schedule 8 is tacked on to delay attribution and does affect the veracity of delay attribution. The fact that a single party effectively controls that truth is difficult, particularly a party that has an incentive to optimise that position for themselves.” (TOC Owning Group)

“We’re seeing this on a wider scale, where there are decisions being made in Network Rail to protect the performance of long-distance operators to the detriment of local operators... just because of the value their compensation attracts in Schedule 8... there’s a legacy thing in there too... It’s not just one or two isolated incidents... our customers are just as important as everybody else’s are.” (TOC)

2.8.7 The delay attribution process from which payments are calculated also causes a great deal of dissatisfaction for Network Rail, TOCs and FOCs. TOCs and FOCs argue that Network Rail does not attribute delays with enough accuracy, and have to invest significant resources demonstrating that past delays have been incorrectly attributed.

(^{14}) As with all verbatim quotes used in this report, the accuracy of this statement has not been specifically checked and it may not be factually correct

(^{15}) SYSTRA was unable to speak to Network Rail signallers as part of this research 2.8.8 Network Rail representatives argue that it is paying out significant amounts in Schedule 8 compensation which overstate the actual long-term revenue impacts experienced by the operator.

“Our performance team will look at our reactive elements of our train performance, what happened yesterday, which incidents have been attributed to [FOC], and are those attributions correct? We will dispute any instance which we believe is not in accordance with the laid down procedure - to try and recover the subsequent penalties... In addition to that, from a proactive perspective, the team will look at the outputs from those reactive events and attempt to identify trends; do some analysis to identify root cause; and look to improve train performance - maybe making our customers, suppliers and own people aware of the underlying issues of the delay and cost to the business. We will identify any locomotives or rolling stock that has caused a succession of issues; look at potential underlying defects; look at our working plans and identifying where there is insufficient time to undertake train preparation; safety checks; making sure everything is in order; changing arrival at work times... looking at train paths to ensure they are robust.” (FOC)

“Schedule 8 is designed to compensate for long-term revenue losses as a result of disruption... it doesn’t actually provide any compensation for Delay Repay... we should try and get the rates right and maybe include the passenger compensation elements so we can be really clear about what Schedule 8 is compensating for, rather than having this murky world of it being designed to compensate for revenue losses but it doesn’t actually.” (Infrastructure Manager)

2.8.9 An industry representative said that, under delay attribution rules, the external delay on an inbound journey could delay a freight service when it begins its outbound return leg. The external delay on the inbound journey would not be recognised as the cause of the delay for the outbound journey under Schedule 8, and the FOC would be financially liable for the outbound Schedule 8 payment.

2.8.10 On one route discussed, it was suggested that FOCs do not risk operating as the payment rates to TOCs are so high (e.g. £700 per minute of delay to the TOC’s service(^\\text{16})), compared to FOCs’ operating margins.

2.8.11 Schedule 8 penalties were a clear driver of FOC decision making, dependent on the situation and examples were provided to demonstrate this. If the network is quiet, then decisions can be made with the focus on customer needs; however, in particularly congested areas, Schedule 8 will be the focus of decision making to mitigate the risk of large penalties in the event of a FOC-caused delay. The significance of Schedule 8 payments has meant the set-up of delay attribution teams to monitor attributions and dispute those which are believed to be incorrect (including setting targets for disputes), and introduction of Tisa(^\\text{17}) and CCTV systems on services. An independent way of monitoring and attributing delays was considered preferable to this being part of Network Rail’s role.

“If we have a failure or something on the network, then the priority for them is to get that train moved out the way so that they’re not causing too much delay, because every

(^{16}) ORR has noted that this high sum would be intended to reflect the TOC’s expected loss in revenues.

(^{17}) Real-time monitoring software, used to track progress of trains against the agreed train plan. minute of delay that we cause to another operator is quite hurtful, we get quite a lot of cost associated with that.” (FOC)

“For the performance team, we’ll set them a percentage review rate of how many of the delay minutes reviewed, that were caused by [name of FOC]. And we’ll also set them a percentage target of how many they’ll actually dispute. Which I think is healthy because obviously its related to the job and if you’ve got a target to achieve and it just gives them that motivation.” (FOC)

2.8.12 The payment rates themselves are also a source of contention across the industry. For example, delays caused by extreme weather conditions are assigned under Schedule 8 to Network Rail to compensate operators for revenue loss, but long-term revenue impact from these events is disputed by some.

2.8.13 Different payment rates across different TOCs mean some TOCs argue that the trains with the highest associated payment rates are those prioritised by Network Rail employees during delay incidents, irrespective of volume of passengers and any likely time-sensitivities of journeys, and therefore at a potentially wider cost to overall passenger satisfaction.

“With external delay events, Schedule 8 forces this blame to be assigned(^\\text{18}). In the vast majority of cases - to Network Rail! And, at the moment, all of reactionary delay is deemed to be Network Rail’s responsibility to fix whereas, in reality, operators can do a lot to fix that. There is also evidence that, for these extreme weather events, there isn’t actually any revenue impact. The Passenger Demand Forecasting Handbook stated that weather events have no impact whatsoever... it was a one off event.” (Infrastructure Manager)

“Our issue is that reactionary delays are a whole industry issue, we can’t fix it on our own, we need TOCs’ support in how they mitigate that; what they do with their train crew, what they do with their rolling stock, decisions about whether or not they carry on with service or whether they cancel it. At the moment, [TOCs] cannot make a business case to improve those because they’ll get no Schedule 8 benefit whatsoever if they help improve the reactionary delay.” (Infrastructure Manager)

2.8.14 The other UK infrastructure managers and non-franchise TOCs have Schedule 8-type schemes (see quotes); some of these are more along the lines of existing concession agreements, where revenue risk is held by a third party. These were seen to have differing outcomes in relation to performance incentivisation when compared to the standard Schedule 8 performance regime. On one hand, it was suggested that as Schedule 8 payments do not affect the TOC’s financial position, there is no incentive for it to improve performance. Moreover, spending restrictions mean that any Schedule 8 payments that are made are not necessarily available for reinvestment into the network to address causes of delays. On the other hand, it was suggested that the absence of Schedule 8 payments help to encourage collaboration by channelling funds to a common goal

“[TOC] is on a gross cost contract so we don’t take risk on passenger volumes or revenue. We actually don’t get any receipts from Schedule 8 either, it’s treated as a

(^{18}) ORR has noted that delay attribution is required by legislation, and that delay attribution rules are agreed within the industry. Attribution of external delay events is one issue within ORR’s delay attribution review commenced in January 2019. pass through. What that’s done is created quite a different relationship between the DfT, Network Rail and the train operator - and I don’t think it’s necessarily been fruitful for improving operational performance. The DfT has found themselves in a position that, in one calendar year, they made a windfall of about £45m in Schedule 8 payments that they didn’t know what do with. As soon as it hits the DfT, it becomes public money and effectively they can’t spend it - it ends up lost to the customer.” (TOC Owning Group)

“What happens is the money goes in to a performance improvement fund pot, which is there to be spent on performance improvement schemes - small schemes to improve performance on the route. That could be Network Rail things, it could be things on the [TOC] fleet. It could be all sort of things but that money goes into that. And we’ve got in place a pretty simple approval process. So it’s good - it’s very sensible and it’s all been agreed and put in place with the DfT. So that kind of changes the commercial focus a little bit. To me, it’s great because it’s completely sensible. You’re kind of using that money and recycling it back into the railway. You know the common perception is that in the press the money gets siphoned off by ‘greedy shareholders’.” (TOC)

2.8.15 Across the industry, the purpose, process of allocation and payments of Schedule 8 are disputed. There is a desire for a system that better-encourages collaborative working and tackles the existing blame culture within the industry. There is a perception that Schedule 8 does not encourage the collaborative working required to tackle the cause of some delays, which would have a positive impact on performance.

2.8.16 Whilst not universal, there is some conviction by non-franchised TOCs and Network Rail that Schedule 8 adversely affects performance decisions made by TOCs; and conviction from TOCs that Schedule 8 negatively influences performance decisions made by Network Rail. However, it is less clear that Schedule 8 is manipulated in such ways for financial gain, at a cost to performance.

2.9 Secondary influences on train service performance decision-making

Schedule 4

2.9.1 Schedule 4 is intended to incentivise Network Rail to plan and schedule engineering works in a timely manner, with enough notice for TOCs and FOCs to make alternative arrangements, and to financially compensate operators for losses in revenue during Network Rail possessions. Schedule 4 was given much less weight on influence of performance related decisions in discussions, but there was a general consensus that overall, it operates as intended. This was also reflected in the survey results, whereby Schedule 4 was the most frequently selected mechanism for having a positive outcome on performance, but was also frequently selected as having a negligible outcome on performance.

2.9.2 Occasionally, however, possession conflicts were reported as causing issues for the delivery of scheduled engineering works. It was reported by TOC, Network Rail and industry representatives that Schedule 4 can lead to delayed decision making, particularly in poor weather conditions, where neither wants to take action and have the delays categorised and theirs.

2.9.3 A FOC representative noted that the payment associated with Schedule 4 did not always cover the cost to their business, and TOC and Owning Group representatives noted the expense of, for example, enhancements delivered over multiple weekends. “Schedule 4 is to get Network Rail to manage their possessions and restrictions of use as effectively and efficiently as possible... it’s always frustrating when you get a possession because possessions stop you running the railway you’re wanting to run and you have to put alternative modes in, but, we all know the railway needs fixing so it’s a necessary evil. My only comment would be that I wish Network Rail could do even more in the space of time that they have available.” (TOC Owning Group)

“It’s probably less of an issue... sometimes operators will say that ‘we don’t want to give you access’ because sometimes the Schedule 4 compensation in that instance wouldn’t be enough.” (Infrastructure Manager)

“[The Infrastructure Manager] can have an impact on our trains. They closed a piece of infrastructure because of works required and we ended up having to go to a half hourly interval because the infrastructure was shut. [But] they could have re-routed us via another junction and we could have carried on working to our normal permit. [This is what they did for another TOC] ... managed by the same people as us but we are not in the same department [like the other TOC is] and there is a competitive tension there, which is intentional.” (TOC)

2.10 Collaborative working

2.10.1 The importance of collaborative working on decisions made relating to performance, and the ways in which some incentive mechanisms can encourage or discourage collaborative working emerged as a key theme from the research.

2.10.2 Strategic level stakeholders across the rail sector highlight the importance of working collaboratively with other industry partners to deliver high levels of.

2.10.3 There are identified barriers to collaborative working to deliver higher performance outcomes:

- The perceived commercial focus for TOCs and FOCs lends itself to focus on short-term individual performance and financial outcomes; this may lead to decision-making that is not necessarily aligned with what would work best from a long-term system-wide performance perspective;
- The perceived commercial focus also lends itself a lack of willingness to share information and data which could lead to better performance outcomes;
- The Schedule 8 process is seen as propagating a “blame culture”, in which investment in the reallocation of costs is more productive than investment in identification and collaborative resolution of the root-cause of delays; and
- The targets, incentive mechanisms and review periods can be misaligned, with differing goal – TOCs said to be following Franchise Agreement obligations, and Network Rail monitored using separate metrics by the ORR (although this was not universally agreed as a barrier); collaboration across different TOCs can be difficult through different targets (e.g. PPM or right-time) even on shared areas of the network meaning there are different priorities when it comes to resolving delays.

“Do we know what we’re chasing? People get together and there are performance boards and performance task forces but what are the top ten actual things that we are doing as an industry to increase performance?” (Consumer/Trade Association) “[Collaboration] has got a lot better in recent times but the fact that the industry has different incentives on TOCs and Network Rail means you have a misalignment... that does adversely impact performance... I think the targets that are set in the franchise, versus the targets that are set by the ORR on us is a big one... there needs to be agreements between the ORR and the DfT about target setting.” (Infrastructure Manager)

2.10.4 Local government representatives highlighted the need for collaboration between Network Rail and TOCs when planning infrastructure enhancement. They perceived that Network Rail’s long-term infrastructure improvement programme, developed with DfT, is focused on capacity (maximising the number of services that can be run), rather than the reliability and resilience of the system. Local government and TOC representatives argued that the focus on enhancing capacity is undertaken without consideration of what infrastructure enhancements would be required to maintain reliability.

A note on infrastructure enhancements

Infrastructure enhancements are recognised as important for the long-term resilience of the railway. Infrastructure enhancement decisions require joined up thinking in order to identify enhancement needs and to build business cases. However, there is currently a lack of collaboration around this, meaning it can be difficult to determine when major investment is needed, what investment is required, and what the short and long-term impacts will be.

2.10.5 Network Rail, TOCs and local government stakeholders each made suggestions for actions that could be taken in order to enhance collaboration and performance across the industry:

- DfT and ORR should coordinate target setting across the industry, with Network Rail given some form of end-customer performance metric and, more widely a whole system approach should be taken, which aligns objectives and regulation periods so all are working together toward the same outcomes;
- TOCs should take lessons from one another and should develop joint route performance strategies with Network Rail. A few TOCs did have examples of joint Network Rail strategies currently in place, and provided examples of actions made to improve performance, such as having Emergency Incident Response teams at key stations or seeking to streamline performance targets and metrics, as well as analysis and reporting of performance data. These strategies are currently not route-wide, however, meaning they do not include all TOCs on the same route; other TOCs felt they are disadvantaged in these instances;
- TOCs should better-engage with frontline staff within the business in order to take learnings on how strategies work, or do not work, in practice;
- Data sharing across the industry should be mandated with an Open Data system;
- Network Rail should strengthen its inputs to the franchise process; and
- Local government stakeholders suggested that long-term strategy development should be devolved to local transport authorities, meaning investment decisions can be made in the interests of all passengers.

“It’s quite extraordinary to reflect that until last Autumn, while every individual operator had a one-on-one bilateral performance plan with Network Rail, there was no forum to talk about performance on the network on the main line. It did not exist. There still is not, as we stand today, a route performance strategy. Every operator has their own. “We’ve been much better in [region] at embracing collaboration... it does mean that we do some really interesting stuff and we do stuff differently and it does lead to improved performance... a really good example is that Network Rail have established Emergency Incident Response teams and we have fitters with them so we can attend the incidents and we’ve put paramedics at [station]. If you can bring the two organisations together, people have so many different ideas about what works, what doesn’t and what you can’t do – you can get things in quicker and much more joined up.” (TOC Owning Group)

“We are going to be improving our relationship with Network Rail. Doing things like getting signallers out with drivers – so signallers can see how decisions at the signal box affect trains real-time. ... The people [at Network Rail] that we work with on a day-to-day basis want to understand our business, and we work very well with them. [In contrast], what used to be Infrastructure Projects are more remote and do not seem to consider the end customer as much.” (TOC)

“At the moment we’re doing a big piece of work with Network Rail in terms of forecasting and planning our improvement for the next 12 to 24 months... it’s very much [TOC] do your bit, Network Rail do your bit and we’ll kind of put them together and see what we get. Which is all well and good but you end up with [TOC] does things in one way or talks about delay minutes or impacts on PPM or something and Network Rail will come along and talk about interventions and incidents and delays per incident. So even to start with you’ve got this disconnect in the planning process, so we’re trying to make that a bit more integrated... going forward, one of the things we want to do is kind of bring the two performance teams together and make them a bit more integrated and a bit more joined up. And I know saying that out loud sounds quite easy on paper, but it’s actually a little bit harder to do than you think. But that’s what we want to do, to streamline the processes, the data management and the reporting and really get people to focus on the analysis of things which could improve performance, because that’s what we’re trying to do.” (TOC)

“Network Rail’s long term planning process is informed by the DfT’s requirements. That is focused mainly on capacity and connectivity... there is not a particular emphasis on performance or reliability.” (Government/Local Government)

2.10.6 Some respondents asserted that where only some elements of the rail network are working collaboratively, there are not necessarily benefits to passengers overall. Some TOCs place great weight on their relationship with Network Rail, and if Network Rail were to give a better quality of service to these operators than to others, then this could have a detrimental impact on some groups of passengers.

2.10.7 TOCs and FOCs both reported the perception that collaboration could be very dependent on personalities within Network Rail, and the ability of the operator to persuade Network Rail to their point of view.

“All TOCs are in some form of alliance with Network Rail. How can Network Rail be partners, and have a close alliance with every train operating company? We’re witnessing behaviours at the moment where I feel like the poor relation to companies like [name of different TOC]. [Name of different TOC] seem to be getting all Network 2.10.8 Government stakeholders felt they had an important role in bringing parties together across the rail industry to build solutions to performance issues.

“We tend to be one of the drivers in ‘look everybody, just put aside your financial interest for a minute, let’s all forget we work for different companies for a minute, let’s get in a room and work the damn thing out’ and that would probably work a bit without us anyway because you know the industry does tend to do that sort of thing in a crisis anyway, but we definitely add pressure to that.” (Government/Local Government) 3. INFLUENCES ON TRAIN PERFORMANCE AT AN OPERATIONAL LEVEL

3.1 Introduction

3.1.1 This chapter provides an overview of the responses from operational stakeholders on the influences on their performance decisions. These views and experiences are complemented with their perceptions of strategic stakeholders about decision-making on the front-line.

3.1.2 Operational level stakeholders engaged with in this exercise included:

- Operations Managers;
- Control Centre Managers;
- Duty Control Managers;
- Planning and Regime Managers;
- Heads of Maintenance Delivery;
- Heads of Control and Resourcing; and
- Heads of Operations.

3.2 Day-to-day decision making

3.2.1 The day-to-day responsibilities of the operational stakeholders include:

- Managing the resources available across internal and external parties to provide services. This includes Network Rail, British Transport Police, internal passenger operator resources and train fleets;
- Managing Control Centres, including actions related to service recovery, contingency plans, cancellations, diversions and rail replacements;
- Managing asset maintenance;
- Managing signallers and train regulation policy; and
- Managing delay attributors, to ensure delay data is accurate.

3.2.2 Operational and front-line staff have to make judgement calls on how best to resolve problems on their area of the network. A common problem for front-line TOC staff to deal with takes the form of a single train experiencing a delay and running late at subsequent stops; this can often escalate to delaying other trains on the route or other routes with linked signals. Typically, the Control Room’s response is to consider how to minimise the impact of this local issue on their customers – with primary consideration to numbers of passengers who could be adversely affected, and how badly they will be affected. The response to this often comes down to a judgement call based on the experience and understanding of Control Room operatives, although there is an increasing reliance on software to advise of the best course of action.

“If we think we have a delay or problem with our service, then we try to isolate the problem so that it affects the fewest number of people... For example, if you had a little compound delay – say you’ve got a service running 10 minutes late, all the other services behind it are running 10 minutes late – we might try and tuck that train away. So we might either cancel it, or move it out of the way [of the others]... so we let that one get worse to avoid more passengers being delayed bigger amounts of time.” (TOC) Case Study 1. Operations Manager, Passenger Train Operating Company

Responsible for the day-to-day operations of a passenger train operating company, this representative manages drivers, train conductors and the control office, which is joint with Network Rail. The primary influence on their day-to-day decisions is achieving performance targets. For example, on an especially congested part of their route, they have developed a traffic management policy which asks Network Rail signal controllers to prioritise trains with higher performance benchmarks, such as their own, over those with lower performance benchmarks. However, this policy only acts as a guide for Network Rail signal controllers and they may also make decisions based on their own experience, knowledge and skill.

Nevertheless, there are instances where maximising service capacity is prioritised over performance targets, and affects performance-related decisions. For instance, when there is severe weather affecting part of the network that may make it difficult to run the full timetable punctually, the decision may normally be taken to cancel a number of trains to enable the remaining trains to run to time. This decision involves a trade-off between two aspects of the performance regime: the benefit in terms of punctuality (running on time) is seen as outweighing the disbenefit in terms of reliability (running trains as per the planned timetable, i.e. not cancelling). This results in some passengers experiencing cancellations, over-crowding and longer wait times, but better punctuality for most.

However, if this adverse weather event coincides with a period of high demand, such as a large sporting event at a venue served by their network, the balance of this trade-off can change. To cater for the additional demand, maximising capacity through avoiding cancellations takes priority, so the decision has been made to run the full timetable. This results in worse punctuality for most passengers, but trains are less likely to be crowded.

3.2.3 The types of decisions and trade-offs operational stakeholders make include:

- Prioritising services with the highest capacity, when a reduced frequency of service needs to be put in place;
- Re-routing a service to a route that means it is less disruptive to other services but which increases overall journey time; and
- Cancelling more trains early on during disruption in order to recover back to normal timetable more quickly, or maintaining delayed services to ensure passengers can travel.

“The longer trains with more capacity, if there was a reduced service, we would run through. Whereas, other much shorter trains may be cut out from the service. That is the right thing. The trains with most capacity should be the ones to run through, it’s logical... if you have an 11 car train, or a 3 car train and only one of them can go to Glasgow, you’re gonna run the 11 car one.” (TOC)

“If we have a train that is running at reduced speed... we would put it on the least disruptive route. We do have guidance for that, which routes are considered the least disruptive. Because, obviously, if you’ve got a train running up to 15mph delay, there’s inevitably going to be some impact on performance and it could impact on other trains that are behind it or on the same route.” (TOC) 3.2.4 At an operational level, keeping passengers moving and happy, by maximising the number of services running at any given time and recovering back to the normal timetable as quickly as possible, clearly underpinned decision making. Day-to-day decision-making is therefore highly associated with service performance outcomes, with stakeholders noting that the following are key considerations in their day-to-day decision-making:

- The sense of pride created from delivering a service that meets the needs of passengers;
- Achieving the public performance measure (PPM) and franchise obligations, and how their organisation communicates around these; and
- Data-led decision making, using both local and industry-wide systems such as Traffic Management systems, where available, and decision-making based on previous experience and personal understanding.

“I would say that unless you care about our passengers, you can’t do this job.” (TOC)

3.2.5 These motivations, and other more secondary influences, are discussed, in turn, below. These influences on train performance are explored in descending order of impact on performance-related decision making, as perceived by decision-makers. Priority in ordering is given to internal company decision making, followed by perceptions on external company decision making.

3.3 Sense of pride

3.3.1 The sense of pride that operational level staff at TOCs derived from keeping passengers moving and delivering high levels of service performance suited to passengers’ needs emerged as a significant decision driver. This incentivises decisions which lead to high performance delivery. This was also reflected in the survey results (primarily TOC respondents), where ‘Taking pride in your work/’doing the right thing’ and ‘Wanting your organisation to succeed’ were the highest rated factors that influence decisions affecting train performance.

“You cannot be successful in the railway at all unless you have a desire to do the right thing for people. So I think I have personal goals [...] of course when I make mistakes I feel terrible about it.” (TOC)

“I think pride in the business... and wanting the control centre as a whole to be well regarded... I think is of importance to quite a lot of people in control... you see when you’re in there during an incident that it can get quite tense and heated. You can tell that people are really passionate about wanting the service recovery to go well and I think a lot of that comes back to pride in the role.” (TOC)

“Sometimes we get letters from our customer services team which say ‘yes, there was disruption but you handled it really well’...you get a lot of satisfaction when you have dealt with disruption and you can walk out of the door and say we handled that really well.” (TOC) 3.4 Company policy

Targets/Metrics

3.4.1 Guidelines, targets and metrics highlighted by operational staff in informing decisions, included:

- Passenger Charters, which provide performance commitments to passengers and can include specific and tailored financial consequences for failing to deliver;
- Targets for recovering the railway back to the timetabled service in a given amount of time following a major incident. Progress against this target is reviewed after each incident, both internally and also externally, by Network Rail, depending on the level of disruption. Learnings from these reviews are then built in to future recovery plans;
- Recovery Plans, which include targets (as above) but which can also be more flexible and provide decision guidance based on e.g. high passenger loads;
- Joint TOC and Network Rail contingency plans; and
- Moving to ‘Right-Time’ metrics, data and regulation to encourage decisions which reduce even the smallest of delays for passengers.

“The problem I think for the industry is that we still report PPM. Most franchises have been built and are constructed around PPM. But in reality, passengers don’t give a monkeys about PPM, they just want to know if it’s on time or not. And if the train is five minutes late, then that is not on time, but I think there are changes afoot within performance regimes to measure right time.” (TOC)

“[Right Time Regulation Policy] incentivises everyone to run trains on time and it makes regulators’ decisions that bit easier and simpler.” (Infrastructure Manager)

“[Name of TOC] have a recovery plan which is very relatively high level... and also the joint Network Rail and TOC Contingency Plans which describe how you would respond to any line block on a particular part of the network and how you would divert around it, and cancel, all that sort of stuff.” (TOC)

Case Study 2. Head of Train Service Delivery, Train Operating Company

Responsible for overseeing service resources such as drivers, guards and dispatchers, this representative also manages the TOC’s control office. The primary motivations for their day-to-day decisions are providing services to suit their passengers’ needs, and reducing the impacts of delays and recovery. A key influence on these decisions is the TOC’s agreed passenger charter with DfT. This charter promises to deliver a minimal amount of service disruption in the morning peak, otherwise the TOC has an obligation to financially refund their passengers. Therefore, decisions often focus on setting up the peak-time services, with practised contingency plans in place.

3.4.2 When prompted, operational level stakeholders at TOCs generally reported that franchise agreement obligations are not influential on performance-related decision making. In fact, operational personnel tended to show low levels of awareness of the specifics of their franchise agreement obligations. However, the targets referred to as influential on decision making may relate to franchise targets, which therefore directly affect decision making. Examples given included the importance of service arrivals within 2 minutes 59 seconds or 14 minutes and 59 seconds, to meet targets. To meet these targets TOCs may ask Network Rail to prioritise a late running train over an early train at an approaching junction, even if the early train is run by a different operator.

3.4.3 Low levels of franchise agreement awareness was not exclusively the case, however, with some respondents demonstrating awareness of the risks to franchised operators of breaching targets, including in terms of reputation, finances, and in bidding for future franchises. However, even in this case it was noted that the influence of targets on performance decisions is not always appropriate, and keeping passengers moving takes precedence even if this does not result in the ‘best’ outcome against the targets set.

“At the vast majority of our decisions, [performance targets] will be at the forefront of our minds... the franchise is at risk if we don’t hit targets... adverse consequences both contractually and in terms of our image and perception of us... however, there are times when other priorities come to force, such as moving people [i.e. maximising the number of services in operation, which may be at the expense of punctuality].” (TOC)

3.4.4 Government and strategic staff at another TOC showed concern over the influence of PPM targets on signallers’ decisions, noting that larger operators and high speed services could be prioritised due to differing PPM targets, and local services could be left behind. An alternative suggestion was that signals should be regulated based on a ‘hierarchy of needs’, including who is on the train, where they are going, and how time critical their journey is. However, currently there is a lack of openly-available data to underpin this hierarchy, and the issue of mistrust between TOCs and infrastructure managers means there is a perceived unwillingness to share such data, with respondents believing it to be commercially sensitive.

3.4.5 An operational stakeholder at Network Rail suggested that operator and Network Rail’s priorities were not necessarily in alignment, with Network Rail taking a longer-term view on performance and operators being most immediately focused on short-term impacts of disruption.

“I don’t always think we’re completely aligned on the Putting Passengers First things, so an example would be so to deliver a sustainably reliable asset, we have to renew the railway, to renew the railway we have disrupt it, there’s no two ways about that. So quite often getting access to do some of those renew is constrained.” (Infrastructure Manager)

3.4.6 When asked which intervention has the greatest positive impact on performance in the survey (which was predominantly completed by TOC respondents), ‘Network Rail’s performance metrics’ was most frequently ranked in the top three, whereas TOC’s franchise, or equivalent, targets featured much less prominently.

Company Culture

3.4.7 For operational-level stakeholders, TOCs’ internal training, communications, and culture are thought to encourage decisions which keep passengers moving, and recover services as quickly as possible. This is fed-down from strategic level, but influences the decisions of operational staff. Specific examples of this can be grouped as follows:

- Providing training, including on service recovery plans and policy; Providing regular updates and feedback on service recovery from the customer service department and stakeholder management teams, to assess how well incidents are managed at an operational level; and

Fostering a company culture that emphasises the importance of performance and allows good performance to be delivered e.g. through high levels of resourcing, shift times that allow for handovers between staff, good depots and facilities and suitable driving terms.

“We show them on a daily basis that performance, safety and good customer information matters.” (TOC)

3.5 Data and experience-led decisions

3.5.1 Whilst not a formal metric intended to incentive performance, the use of data and drawing on personal experience arose as key influencers on performance-related decision making at operational level. The different factors at play when operational staff are making decisions in the event of delays include a combination of targets/metrics, data that informs on the likely outcome of the decision, specific circumstances (e.g. a high capacity event), and predictions based on previous experience. Each delay event was seemingly considered unique, and whilst there are guides in place, the guidelines are interpreted and actioned with the use of prior personal experience as well.

“If you cancel one train and everything after then can depart on time that may be a better decision than everything leaving 10 or 15 minutes late... one of the things we look at is if we are having trains leaving origin late, when they get to their destination, is that going to cause a further delay... we would use that as part of the judgement [to cancel]... by having one cancellation we could probably recover [any secondary delay] and have everything running on time... I see control’s role as making the least worse decision.” (TOC)

“[We use our] experience, yes. But, it isn’t a case of wetting your finger and putting it in the air to see what kind of approach [should be taken]. It is very much adhering to guidelines as well as possible, with a bit of flex, depending on the situation.” (TOC)

Data-Driven Decisions

3.5.2 Operational level staff cited data-led performance decision-making, however there was a strong suggestion that access to more, high-quality data would further improve the outcomes of performance decisions.

3.5.3 For some TOCs, whilst data-led decisions are made using, for example, averaged passenger counts, real-time data is seen as lacking. The provision of real-time information on passenger loadings, through on-train technology such as door counts, or weighted seats, is thought to be possible in the near future, and would improve service prioritisation during delay incidents. Where this is currently not available, TOCs use reservation information and conversations with train managers to understand load, which is not considered reliable. Live data would also assist because decisions needs to be made quickly.

“I think [data] is gradually improving but it’s not as good as I would like it to be at the moment. For example, the number of the people on the trains. We did not know until relatively recently how many people were on the trains we are managing. It was only from the controls’ experience... We’ve now got numbers in control which are based on historical passenger counts which allow somebody to look up how many people will be on a train, on average. It is a step forward but it still doesn’t tell you how many people are on the train on this day... We’re in the process of procuring a complete new set of trains and the new trains will have the capability to tell you how many people are on the train.” (TOC)

“What would be really helpful would be to have live loadings of services at any given point, and accurate ones as well. That would very much help in decision making.” (TOC)

“What we would love to have, and I’m sure it’s in scope somewhere, is some kind of decision making model that we can simulate something and it would spit out what the impact of that would be in terms of performance and really help to guide those decisions and ensure that we really are as consistent as we can be.” (TOC)

3.5.4 Data on real-time driver and guard location, key when actioning recovery plans, is also perceived as difficult to access. For TOCs, this often relies on an assessment of where traincrew are scheduled to be, and through speaking with drivers directly. However, the latter is not always possible as drivers are not able to use their mobile phones whilst in the train cabs.

3.5.5 Data on the socio-economic impacts of delays, such as access to employment, was also noted as required by one TOC, with the assumption that this data would help the TOC to make the case to Network Rail for better regulation of their services.

“The thing that is relatively easy about managing trains is that they are always where you put them and they are always where you send them and you can always track where they are at any given time. That’s not true of the people on the trains. In order to run a train service, you need a train, you need a driver and in some cases you need a conductor. In service disruption, the most difficult elements of that are the driver and conductor... All of our drivers have mobile phones which mean they are contactable by control... The difficulty with mobile phones is they are not allowed to have them switched on when they are in the cab of the train... It’s not easy communicating with them by mobile phone.” (TOC)

“How do you build into an incentive regime, something that considers passenger impact, rather than just delay minutes? You need to add in the socio-economic impact into the Schedule... we’re moving people into key urban centres for economic reasons.” (TOC)

3.5.6 Operational level staff at Network Rail noted that a lack of analytical capability can affect the ability of those across the industry to makes decisions in relation to train performance. Specifically, there is a lack of understanding as to the causation factors for sub-threshold / low level delays, and therefore these are automatically attributed to Network Rail and not often subjected to a root-cause analysis. Network Rail focuses on the delays attributed to it, but taking actions to resolve issues related to sub-threshold delays can be difficult, when the known cause is undefined and the analytical resource available is insufficient to follow it up.

19 ORR notes that Schedule 8 compensates TOCs for lost future revenues, but does not factor in wider economic impacts. “We’ve got tonnes and tonnes of data and tonnes and tonnes of information but insufficient analytical capability. When you look at the amount of sub-threshold delay... and so much delay is in that category and you think that there must be a reason for all this. How do we delve into this and work out what is actually going on?... It just is not sufficiently refined to be able to get to root cause... The focus is on things that are attributed, and not things which aren’t attributed. Focusing on sub-threshold, it’s secondary. It’s not going to anyone so we need people looking at that delay... It takes the analysis to show you the size of the prize so you can focus on it.” (Infrastructure Manager)

3.5.7 Operational level staff at TOCs and Network Rail also highlighted the development and use of IT systems in helping them to make performance related decisions, some of which outline the outcomes of decisions before they are made. For example:

- Running a delay scenario through a simulator to build a suitable contingency plan for service recovery. This plan is then distributed to front-line staff to action, and these individuals are able to provide feedback into the simulator to minimise performance impacts even more in future scenarios. Front-line staff are also held accountable in their efforts to deliver the agreed contingency plan, and this is assessed through their personal development reviews; and
- Installing automatic signal control for simple signals on highly-congested routes, giving more time for signallers to manually deal with more critical signals.

3.5.8 These systems were thought to simplify the process of control, assessing the impacts of control-level decisions on the full network; a task which can otherwise be quite difficult for individuals to make based on prior experience alone.

“It’s an iterative process so we all have ideas about how we can minimise performance impact, but it’s actually the simulator that tells us the best solution.” (TOC)

“We’ve got a project to install automatic route setting on the [route]. And that project is something I’m supportive of, from the perspective of enabling signallers to focus on critical areas of regulating by having the system automatically doing the simpler things.” (Infrastructure Manager)

“Each controller has quite a lot of trains to consider... It is quite a difficult picture to fully get your head around what are the implications, what are the results of any decisions you take... There are certain IT systems which can, to a degree predict, and say, ‘if you do this, this is what will happen, here is what the system believes will happen’ and will allow to you see the results of decisions before you take them... At the moment, it’s really based on experience of control of the network.” (TOC)

Personal Experience

3.5.9 Decisions are made on individuals’ knowledge and understanding of the situation based on their previous experience and policies. This appears to particularly be the case if supporting data is missing or incomplete. Prioritisation of passenger needs and performance decisions were recognised as being made often in the ‘here and now’, which may not encompass the bigger picture and potential knock-on impacts. The process was described as making ‘the least worse decision’, which needs to be made quickly with minimal thinking time, relying on personal experience. “When you’re at the front-line – if you’ve got a passenger right in front of you, you can have a situation where a member of staff makes a decision on the basis of the micro-universe they’re experiencing at the expense of a much larger group of passengers”. (TOC)

“We often leave it down to the individual controllers, experience, knowledge and skill to make those decisions... traffic management statements guide them to make the right decisions.” (TOC)

“From a control point of view, we make the decisions... it’s quite an isolated job. You work 24 hours a day, 7 days a week and many other departments are Monday -Friday 9-5. We have to work autonomously, based upon what is important to us. I’ve been in the railway for 10 years. Human nature will always play a part... as a rule of thumb, most people appreciate that what we do is done with the information that we have got at the time.” (TOC)

3.6 Secondary influences on train service performance decision-making

Schedule 8

3.6.1 Respondents on the operational side within TOCs suggest Schedule 8 does not influence performance or delay attribution decisions. Reasons for this can be grouped as follows:

- Schedule 8 is not fully-understood at an operational level;
- Performance decisions are mostly driven by the need to keep passengers moving, based on individual knowledge and skill and TOC’s local targets;
- Delay attribution decisions are primarily driven by the need to provide accurate data on delay cause, so that data can be analysed to build robust and valid business cases for improvements.

3.6.2 Additionally, there is a perception from Network Rail and TOC’s operational level staff that Schedule 8 causes a blame driven environment which is not conducive to running a high performing railway and addressing root causes of performance issues. However, this perception is not necessarily based on a detailed understanding of how the mechanisms work in detail.

“I would say most people don’t really understand how Schedule 8 works, if you ask people to explain it, they’ll probably not be able to explain in the way regime actually works in practice and that in itself may lead to different behaviours. We need to find a way of removing the contractual financial impact of all of this from day to day decision making.” (TOC)

“I don’t think anyone really has enough on a grip on Schedule 8 to make decisions around it.” (TOC)

“The people who are responsible for day to day train service provision, care about an on-time train service and if I were to dare to say to my controllers, ‘make sure you mince that incident so we can get Schedule 8’, my controllers would be absolutely shocked. I wouldn’t even dream of asking people who look after the day to day service to behave in such a way... if that did happen, I would expect Network Rail to call me up on it and have an investigation as to why an incident was so badly managed.” (TOC) “In 20 years working in the rail industry, I’m not aware of Schedule 8 driving really any frontline operational behaviour... the people making those day-to-day decisions don’t really understand Schedule 8, and quite frankly, neither should they.” (TOC Owning Group)

“In terms of delay attribution... we want to get to the actual cause of an incident and make sure it is recorded properly because then we have good data that we can use for future business cases. Schedule 8 does not come in as a driver for delay attribution. It is not something that we pay attention to. Delay attributers do not have any targets around financial targets, they have a target around clear up of incidents.” (TOC Owning Group)

“I personally think that Schedule 8 is negative. I don’t think that it drives the right focus. Whether it actively affect performance or not, you know you lose a lot of money or you can gain a lot of money out of Schedule 8, and to me it has the potential to drive perverse behaviour and perverse focus... I think there are better ways to incentivise one another and hold one another to account than Schedule 8 money changing hands.” (Infrastructure Manager)

“Schedule 4 and Schedule 8 are just not fit for purpose, they encourage the wrong behaviours and the industry spends far too much time and effort squabbling over where 3 minutes [delay] should sit, rather than how you will improve the big-hitting incidents... it’s a compensatory framework, it is not a framework that encourages proactive improvement behaviour.” (TOC)

3.6.3 In contrast, strategic level stakeholders from operating companies, Network Rail, government and consumer/trade associations suggested that operational level staff could make performance related decisions to enhance Schedule 8 payments for TOCs, or reduce Schedule 8 penalties for Network Rail. For example:

- Network Rail Signallers may prioritise long-distance operators at junctions as these operators hold higher Schedule 8 payments if delayed(^{20}); and
- Passenger train operators and Network Rail may not respond to incidents as quickly as they could in order to reduce Schedule 8 penalties e.g. TOCs may wait for Network Rail to impose speed restrictions and Network Rail may wait for TOCs to cancel trains(^{21}).

3.6.4 However, a strategic level viewpoint at Network Rail, suggested that it is not possible for decisions to be influenced by Schedule 8 because there is not the capability to do so at TOC operational level.

“At the moment, commuters will get a raw deal if they are on one of the local stopping trains because Schedule 8 has enormous amounts of money attached to the big operators that go down to London, and [name of TOC’s] payments are much smaller so if you’re Network Rail who are in charge of the signalling, you are going to prioritise the trains that are going to cost you a lot of money.” (Government/Local Government)

“You talk to senior management within the railway who will look you in the eye and say, ‘we make the right passenger decisions, irrespective of Schedule 8’, you talk to the

(^{20}) ORR note this is a built-in incentive property of Schedule 8 due to differences in payment rates which reflect the varying impact of delay on different services.

(^{21}) ORR note this would be an unintended effect of Schedule 8. people working in Control and they find it very difficult to actually do their jobs without worrying about Schedule 8 beyond everything else... you have to watch for perverse incentives... such as in severe weather where it is a ‘whoever blinks first’ situation. If the weather forecast for tomorrow was absolutely dreadful, TOCs, financially, wish to wait for Network Rail to say, ‘we are not letting you run your full timetable tomorrow’. If the TOC goes to Network Rail and says, ‘we want to run half the trains tomorrow’, then Network Rail have not done anything to the TOC... it means the TOCs will hold off and hold off and hold off, meaning Network Rail will impose the restriction and then they [operators] get compensated... the performance consequences are, you end up delaying the alternative timetable until after it is possible to put it in... there needs to be a strengthening of collaboration on both sides so we run the best possible service for passengers.” (Consumer/Trade Association)

**ORR’s Monitoring & Scorecards**

3.6.5 Operational level staff at Network Rail note that Network Rail Scorecards are mostly important for the operational management team, rather than frontline operational staff. For instance, in one region, a local focus on ‘On-Time, all stations’ metrics means that performance decisions are required to be made for all stations on a line, rather than just arrival time at terminals, which would be the focus using PPM. On the other hand, ORR’s monitoring of Network Rail is not viewed as influential on operational level decision-making.

“Our route Scorecard is really important. The Scorecard we have for the route and the measures we put in to that are a big drive and focus for the management team. Whether it’s quite so important to the frontline teams, I think it’s less so... The ORR aspect, I don’t see as particularly relevant.” (Infrastructure Manager)

**Performance Related Pay**

3.6.6 Bonus payments are not viewed as influential on performance decisions by operational level staff. Reasons for this included the decisions of one person being unlikely to create a material increase in bonus.

“Generally I don’t think people are particularly motivated by ‘if I make the decision rightly or wrongly, I’ll get more money or whatever’.” (TOC)

“We have a bonus system and some elements of the bonus link in to the train running performance and customer satisfaction levels... that’s a relatively small matter. The numbers that we use to measure that bonus are quite broad, they are about the overall performance of the company. If you’re one individual, to what degree do you think that a decision you make on one day is going to affect your bonus. You probably recognise that it doesn’t.” (TOC)

**3.7 Collaborative working**

3.7.1 There is a drive for operational level staff across Network Rail and TOCs to work in collaboration, especially to generate and share learnings from service recovery.

3.7.2 Collaboration was often seen to be down to the mindset of individuals, who may choose to work collaboratively, or not. This means that in many areas, effective collaboration is primarily reliant on the availability of individual resources and skills, which can limit the ability to collaborate in practice.

“I think that mostly the collaboration is quite good in terms of the individuals who work in the industry tend to be quite industry minded... That tends to mean that people work as individuals fairly collaboratively. There can be certain cases where structure in the industry doesn’t necessarily encourage collaborative working. But in my experience it tends to come down to the way the individuals work.” (TOC)

“Collaboration will improve performance for a larger number of people. I think, as a rule of thumb, we always do [work collaboratively], but if we wish to do more of it, and improve it, then we need to invest in people, skillsets and infrastructure to allow that to happen. If we were to spend a lot of our time talking about strategic performance and regional performance plans, then we need people employed in those positions to discuss them. What tends to happen is you get heads of controls stepping in to do that but alongside doing that, they are also doing the work within their TOC, and within their team.” (TOC)

3.7.3 Operational FOC stakeholders reported that, despite regular communication with Network Rail, there were limited levels of collaboration with them to resolve problems, suggesting that there were more limited levels of collaboration to resolve problems than Network Rail has with passenger operators.

3.7.4 Operational level staff at Network Rail noted that there are inherent differences in the relationships it has with different TOCs, in particular larger operators who they have more frequent discussions with, versus smaller operators. They acknowledge that this could be perceived as influencing decisions related to performance, but suggest that they try to make sure this is not the case, and that smaller operators, including freight operators, are considered on a fair basis.

“When I look at the number of operators who operate over [region]... I probably have a stronger relationship with [name of TOC]. They are by far the biggest operator in [region]... we’ve got a greater degree of common interests... [However,] I make every effort to make sure we treat freight customers [fairly] by prioritising their needs.” (Infrastructure Manager)

3.7.5 A strategic level member of staff from Network Rail suggested that there is not enough collaboration within Network Rail, especially when it comes to the operational level staff. Specifically, there is a perspective that middle management do not receive enough training and lack understanding of what is required to do front line jobs, and that those doing front line jobs do not always receive enough support and therefore have a tendency to work ‘to procedure’, even when the outcome is likely to be sub-optimal. 4. IMPACT OF COVID-19 ON RAIL INDUSTRY

4.1 Introduction

4.1.1 This chapter summarises the key impacts on performance of the industry response to Covid-19. These views should be considered as a ‘snapshot’ findings, as the Covid-19 crises continues, and response is likely to change and evolve.

4.1.2 These views were captured between 24th June and 14th July 2020. At this point TOCs were still operating reduced timetables under Emergency Measures Agreements with the DfT, albeit at higher service levels than at the beginning of lockdown, and passenger numbers were beginning to recover. They are primarily the views of TOC representatives, but also include the views of Owning Groups, FOCs, local government and Network Rail.

4.2 Key factors behind the performance improvement

4.2.1 Key to the higher levels of performance during the Covid-19 crisis is the resulting impacts from having fewer trains and passengers on the network. The direct impacts from having fewer trains and passengers on the network include:

- Reduced wear and tear on assets and increased time available for maintenance, resulting in better reliability;
- Reduced requirements in terms of the numbers of trains and crew required to operate the reduced timetable, providing more operational ‘slack’ than usual;
- Fewer problems associated with congestion and delay in train paths, including fewer reactionary delays;
- Shorter dwell times due to reduced passenger numbers; and
- Fewer trespass incidents.

4.2.2 ‘Fewer trains on the network’ and ‘Achievable timetables’ were the most frequently cited reasons for current levels of high performance cited by survey respondents.

“Even with very minimal numbers managing the office, the performance has been exceptional... which I think is absolutely symptomatic of less trains running around on the network, and equally, if not more importantly, significantly fewer people running around on the trains as well... when you carry fewer people, there’s far fewer incidents to occur.” (TOC)

“At the moment, the timetable is pretty much looking after itself – we’ve got far more trains than we need, we’ve got far more crew than we need, the network is less congested so trains are finding their way through their paths much easier than they would do normally, there’s a bit more flex in the timetable because it’s not as resource hungry... it’s really a case of where control see incidents emerging that they are being as proactive as they can be.” (TOC)

“We’ve not got customers in the system causing problems... we also have reduced timetables which has meant there is less of a chance of our trains causing reactionary delays on other trains... we’ve had some of the best performance we’d ever seen.” (TOC)

“COVID-19 has shown how good performance can be if the network is allowed to have some spare capacity to absorb the impacts of perturbation!” (TOC – survey) 4.2.3 The Covid-19 crisis is reported as generating a more collaborative approach between operators and Network Rail, with stakeholders noting:

- TOCs and Network Rail are able to work together more effectively to make improvements to assets and infrastructure as fewer trains are using the track and so Network Rail have more time to make infrastructure improvements;
- TOC Performance Teams have more time to identify root cause of delays and work with Network Rail to identify and implement changes, including developing individual action plans which are all followed through. This means performance improvements have been better realised and delivered, even after just once instance of a delay, whereas previously it might have taken a repeat issue to take more detailed action;
- Both the reduced passenger numbers and the pausing of Schedule 8 has meant decisions related to infrastructure improvements can be more agile and infrastructure focussed;
- The improvements to assets and infrastructure now will help ensure that services can run at better performance levels when more customers, and more trains, return.

4.2.4 It was also noted that performance is typically better in warmer months. One TOC representative, via the survey, noted that the pressure to deliver franchise obligations has been lessened, leading towards a more collaborative relationship with DfT.

“Network performance before Covid was diabolical... it was far too reactionary [reactive]... our focus has probably shifted from being quite reactive and trying to claw back what we could... the current set-up is very much about trying to fix the roof while the sun shines, to try and make sure that the network is as reliable as it can be when we start seeing people return to it. We are working very closely with Network Rail to make sure that they can have the time and access to go in and enhance key assets to make them reliable. We are able to be much for flexible about how we do that; previously we had timescales, we had to give them 12 weeks’ notice... we’ve done that a lot more nimbly since Covid started. We’ve done some big blockades without 12 days’ notice which is unheard of.” (TOC)

“Whilst Covid’s been going on, Schedule 8 has been suspended, so we don’t have that revenue... normally, and quite rightly, we’d be obsessed by what impact that would have to the customers that were displaced... we can be a lot more infrastructure friendly, rather than customer friendly.” (TOC)

“There is clearly much closer working in the industry, particularly with Network Rail. This is prompted by the EMA [Emergency Measures Agreement] and industry organisations such as RICF [Rail Industry Coronavirus Forum].” (TOC - survey)

“I think it’s probably provided an opportunity to work even closer throughout this period, because we faced similar challenges in terms of how we managed our frontline staff... even issues as simple as things like wearing a face mask... So there have been lots of good discussions which have reached a regional level, also nationally through the Rail Delivery Group... I think that’s worked really well.” (Infrastructure Manager)

4.2.5 FOC stakeholders noted that increased priority was given to freight movements, due to supply chain pressures and the reduced number of passenger trains. “Coronavirus has been a really telling point. Because there are fewer passenger trains on the railway, and because the Government have said getting food trains from England up to Scotland was a priority, suddenly, there was nothing that was allowed to get in the way of running a food train. So, the government guideline clearly came in and influenced.” (FOC)

4.2.6 Considering the unprecedented nature of the Covid-19 crisis, it was difficult for stakeholders to identify lessons learnt that could be used to help maintain high performance, however, points raised included:

- Capacity has been demonstrated to be key to performance during this period, suggesting that longer rolling stock, rather than more services, is a better solution to increasing capacity and delivering good performance; and
- There have been instances around station dwell times not being met, even with the reduced capacity, potentially indicating fundamentally incorrect timetable assumptions.

“Covid has proved that it is too many services which are causing performance issues. We need to reassess our services, length vs frequency, build for longer trains but still build because we need people back on trains.” (Infrastructure Manager – survey)

4.3 Short, medium and longer terms risks to performance

4.3.1 Whilst performance is currently high, there were risks flagged that could challenge current performance levels and affect future performance. These include:

- The challenge of resourcing during periods of high absence due to illness;
- The need to factor in more time to get spare train crew onto services as they are travelling from home, rather than a central staff room, due to social distancing;
- Using cancellations for faster service recovery is much less desirable or possible, as it would cause large gaps in the reduced timetable and the reduced capacity could cause issues for social distancing as passengers attempt to board the next available train; and
- Driver training is not able to happen due to social distancing (two people cannot be in the cab together). This means TOCs may not be able to step up services when needed and the longer-lasting effects of delays to training now could be felt over the next two years.

“There’s the repercussions of Covid on our plans, trying to find out what they are, trying to understand what the impact is and how we can recover from that... one of the things is the impact on driver training... we’ve already had to suspend four courses of driver training [due to social distancing] and the impacts of that we’ve not seen right now, but we will of seen the productivity over time.” (TOC)

“You wouldn’t want to cancel trains because then that’s two lots of passengers getting on a service and social distancing becomes an issue.” (Government/Local Government) 5. SUMMARY OF FINDINGS

5.1 Key Themes

5.1.1 The research has uncovered key themes, when considering rail performance incentives. Primarily, these are:

- A demonstration that the way some formal incentive mechanisms are intended to work in principle, is not necessarily how they play out in real-life situations;
- In many cases, clear distinctions between decision-making at strategic and operational level can be drawn, revealing that different people across the industry face different motivations, and place greater weight on difference incentives, depending on the focus of their role;
- Personal job satisfaction and customer satisfaction derived from running trains as planned, on time are key motivators for delivering a high performance service, however other considerations come into play, such as a capacity/performance trade-off and the need for some formal mechanisms to provide protection to franchised TOCs;
- Prioritising increased capacity, an ambitious timetable, and theoretically efficient use of resources, as some TOCs believe is required to be successful in the franchising process, can be to the cost of performance on the network;
- There are very different views on the effectiveness of performance targets, particularly those in franchise agreements even within the same stakeholder types, and this may reflect different experiences due to differences in the effectiveness of the calibration;
- Increased collaborative working was viewed across the industry as beneficial for improving performance but some incentive mechanisms were perceived as discouraging this way of working; and
- Theoretically optimal incentive mechanisms must be supported by high quality data, shared across different parties, to facilitate good performance outcomes.

5.2 Summary

5.2.1 The following section summarises the key findings from the research study.

Customer experience

5.2.2 This was a theme brought out most clearly by strategic-level stakeholders.

TOC perspective

5.2.3 Train performance is seen as crucial for the success of the business, as company revenue and reputation are inextricably linked to customer satisfaction which, itself, is underpinned by passengers’ perceptions of service performance. There are differing views on which performance metric best captures customer needs.

5.2.4 Network Rail is not perceived to have the same level of passenger focus, despite some examples at a regional level that they do. Reasons for this include:

- Disparity in TOC and Network Rail targets; and
- Network Rail is potentially not perceived to be responsible for poor performance.

Network Rail perspective 5.2.5 The focus on train performance is, seemingly, more from a process-driven perspective – reflecting that their contribution affects rail-industry customers, rather than the end-customer. Strategic ambitions are set more on developing better ways of monitoring and understanding issues affecting train service performance with the aim, ultimately, of improving it.

Other perspectives

5.2.6 In the rail freight industry, customer satisfaction is an important influence on train service performance decisions; FOCs wish to provide high quality services which encourage ongoing custom, in order to maintain the bottom line.

5.2.7 Strategic decision-makers in government, and consumer advocates, share the same view as TOCs, that train performance lies at the heart of public perception of the rail industry.

Franchise agreements

5.2.8 This was a theme brought out most clearly by strategic-level stakeholders.

TOC perspective

5.2.9 Some TOCs blame over-optimistic performance proposals during the franchise bidding process on the basis on which DfT evaluates the bids and makes their award. The result is that some TOCs feel they are under commercial pressure to increase the number of services run to generate higher ridership and increased revenue, in a context where Network Rail is obliged to accommodate such requests ‘where feasible’, thereby putting further pressures on an already capacity-constrained system.

5.2.10 Mixed views were reported on the benefits of performance targets set within franchise agreements. Some TOCs suggest they create strong financial incentives to deliver and are influential on decision making, whilst others suggested they are not a key driver, or are viewed as arbitrary and thus not incentivising.

Network Rail perspective

5.2.11 Senior representatives at Network Rail suggest that there is a desire to improve the industry process for creating new timetables, not least in response to the significant negative media and ministerial interest. This is especially in the context of the problems caused by the May 2018 timetable change; this led to the development of a stakeholder steering group aiming to review infrastructure, rolling stock and route/operator readiness for future timetable change.

Other perspectives

5.2.12 The industry regulations affecting timetabling, and the local processes for regulating off course trains at key locations, were also seen to have direct adverse performance outcomes for freight services and local/smaller services, as well as indirect reactionary delays on passenger services.

Joint perspectives

5.2.13 Concerns were raised by representatives from operators, Owning Groups, local government, and Network Rail over the target setting process at franchise bidding stage. Individual motivators

5.2.14 Representatives at TOCs, FOCs and Network Rail highlight that feeling pride in their work and operational performance is a key motivator for staff at all levels, and indicate that this drives the delivery of a high performing train service.

5.2.15 There is limited evidence suggesting that performance related pay or bonuses impact decision making at Infrastructure Managers or TOCs. Often, performance is only one element of a bonus scheme and there is a perception that little can be done at an individual level to influence it.

Media and political influencers

5.2.16 This was a theme brought out most clearly by strategic-level stakeholders.

TOC perspective

5.2.17 TOC representatives noted that whilst maintaining good relationships with media, MPs, and consumer advocacy groups was important, it was likely that actions were already being taken to resolve performance issues before they were raised by any of these group types.

ORR monitoring

5.2.18 This was a theme brought out most clearly by strategic-level stakeholders.

TOC perspective

5.2.19 Strategic stakeholders at some TOCs feel that Network Rail Scorecards have some importance to Network Rail decision-making. There is also some desire from TOCs, and passenger/trade representatives, that the Scorecard metrics, and ORR’s monitoring of Network Rail, should be strengthened.

Network Rail perspective

5.2.20 Strategic stakeholders at Network Rail and in local government perceive ORR’s Network Rail Monitor, including the use of Scorecards, as having some influence on performance decisions within Network Rail. However, for others in Network Rail, Scorecard targets are secondary compared with individual pride in their work.

Other perspectives

5.2.21 Strategic stakeholders at FOCs suggest that Scorecards have little impact on Network Rail performance as Network Rail is believed to focus their performance related decision-making primarily on senior level staff’s priorities.

Schedule 8

5.2.22 This was a theme brought out most clearly by strategic-level stakeholders.

TOC perspective

5.2.23 Strategic representatives at franchised TOCs strongly rebuff assertions that they use Schedule 8 to maximise their financial gain, in a way that results in poorer performance outcomes. They state that passenger satisfaction is an absolute priority, and that revenue loss from dissatisfied passengers, due to delays, far outweighs any financial gain to be had through Schedule 8 payments.

5.2.24 Different payment rates across different TOCs mean some TOCs argue that the trains with the highest associated payment rates are those prioritised by Network Rail employees during delay incidents, irrespective of volume of passengers and any likely time-sensitivities of journeys, and therefore at a potentially wider cost to overall passenger satisfaction.

Network Rail perspective

5.2.25 Views expressed by Network Rail, non-franchised TOCs, government and trade/passenger associations suggest that some TOCs use Schedule 8 to maximise their financial gain, resulting in poorer performance outcomes. TOCs are perceived as actively disincentivised to assist in the resolution of delays because, as the delays accrue, so do the associated payments.

5.2.26 The payment rates themselves are also a source of contention across the industry. For example, delays caused by extreme weather conditions are assigned under Schedule 8 to Network Rail to compensate operators for revenue loss, but long-term revenue impact from these events is disputed by some.

Other perspectives

5.2.27 Schedule 8 penalties are a clear driver of FOC decision making, dependent on the situation.

Joint perspectives

5.2.28 There was recognition across all stakeholder representative types that, in principle, financial compensation when performance fails due to another party was necessary. There was a desire for accurate recording of delay incidents, so that performance related investment decisions can be justified and are correctly made.

5.2.29 Across the industry, the purpose, process of allocation and payments of Schedule 8 are disputed. There is a desire for a system that better encourages collaborative working and tackles the existing blame culture within the industry. There is a perception that Schedule 8 does not encourage the collaborative working required to tackle the cause of some delays, which would have a positive impact on performance.

Collaborative working

5.2.30 Strategic level stakeholders across the rail sector highlight the importance of working collaboratively with other industry partners to deliver high levels of performance.

5.2.31 However, there are identified barriers to collaborative working to deliver these outcomes, including perceived commercial focus of TOCs and FOCs, the ‘blame culture’ Schedule 8 is seen as propagating; and misaligned targets and incentive mechanisms (not universally agreed).

5.2.32 Where only some elements of the rail network are working collaboratively, there are not necessarily benefits to passengers overall. TOCs generally place great weight on their relationship with Network Rail. Some perceived that on particular parts of the network, the more dominant TOC can benefit from more favourable quality of service from Network Rail, and that this could have a detrimental impact on some groups of passengers travelling with less influential TOCs.

**Sense of pride**

5.2.33 This was a theme brought out most clearly by operational-level stakeholders.

**TOC perspective**

5.2.34 The sense of pride that operational level staff at TOCs derived from keeping passengers moving and delivering high levels of service performance suited to passengers’ needs emerged as a significant decision driver. This incentivises decisions which lead to high performance delivery.

**Company policy**

5.2.35 This was a theme brought out most clearly by operational-level stakeholders.

**TOC perspective**

5.2.36 Guidelines, targets and metrics highlighted by operational staff in informing decisions, included:

- Passenger Charters, which provide performance commitments to passengers;
- Targets for recovering the railway back to the timetabled service in a given amount of time following a major incident; and
- Moving to ‘Right-Time’ metrics, data and regulation to encourage decisions which reduce even the smallest of delays for passengers.

**Network Rail perspective**

5.2.37 Guidelines, targets and metrics highlighted by operational staff in informing decisions, included:

- Recovery Plans; and
- Joint TOC and Network Rail contingency plans.

5.2.38 An operational stakeholder at Network Rail noted that operator and Network Rail’s priorities were not necessarily in alignment, with Network Rail taking a longer-term view on performance and operators considering the short-term disruption impacts.

**Other perspectives**

5.2.39 Government and strategic staff at a TOC showed concern over the influence of PPM targets on signallers’ decisions, noting that larger operators and high speed services could be prioritised due to differential PPM targets, and local services could be left behind. An alternative suggestion is that signals should be regulated based on a ‘hierarchy of needs’, including who is on the train, where they are going, and how time critical their journey is. However, currently there is a lack of openly-available data to underpin this hierarchy, and the issue of mistrust between TOCs and infrastructure managers means there is a perceived unwillingness to share such data, with respondents believing it to be commercially sensitive. Data and personal experience

5.2.40 This was a theme brought out most clearly by operational-level stakeholders.

TOC perspective

5.2.41 Operational level staff cite data-led performance decision-making, however there is a strong suggestion that improved access to more, high-quality data on passenger load and the socio-economic impacts of delays would further improve the outcomes of performance decisions.

5.2.42 Data on real-time driver and guard location, is key when actioning recovery plans, is also perceived as difficult to access.

Network Rail perspective

5.2.43 Operational level staff at Network Rail noted that a lack of analytical capability can affect the ability of those across the industry to make decisions in relation to train performance. Specifically, there is a lack of understanding as to the causation factors for sub-threshold / low level delays, and therefore these are automatically attributed to Network Rail and not often subjected to a root-cause analysis.

Other perspectives

5.2.44 Operational FOC stakeholders reported that, despite regular communication with Network Rail, there were limited levels of collaboration with them to resolve problems, suggesting that passenger operators were prioritised over freight operators.

Joint perspectives

5.2.45 Operational level staff at TOCs and Network Rail also highlighted the development and use of IT systems in helping them to make performance related decisions, some of which outline the outcomes of decisions before they are made. These systems are thought to simplify the process of control, assessing the impacts of control-level decisions on the full network; a task which can otherwise be quite difficult for individuals to make based on prior experience alone.

5.2.46 Decisions are made on individuals’ knowledge and understanding of the situation based on their previous experience and policies at TOCs and Network Rail. This appears to particularly be the case if supporting data is missing or incomplete. Prioritisation of passenger needs and performance decisions were recognised as being made often in the ‘here and now’, which may not encompass the bigger picture and potential knock-on impacts.

Franchise agreements

5.2.47 This was a theme brought out most clearly by operational-level stakeholders.

TOC and Network Rail perspectives

5.2.48 There is a drive for operational level staff across Network Rail and TOCs to work in collaboration, especially to generate and share learnings from service recovery. 5.2.49 Collaboration is often seen to be down to the mindset of individuals, who may choose to work collaboratively, or not. This means that in many areas, effective collaboration is primarily reliant on the availability of individual resources and skills, which can limit the ability to collaborate in practice.

5.2.50 Operational level staff at Network Rail noted that there are inherent differences in the relationships it has with different TOCs, in particular larger operators who they have more frequent discussions with, versus smaller operators. They acknowledge that this could be perceived as influencing decisions related to performance, but suggest that they try to make sure this is not the case, and that smaller operators, including freight operators, are considered on a fair basis. SYSTRA provides research and advice on transport, to central, regional and local government, agencies, developers, operators and financiers.

A diverse group of results-oriented people, we are part of a strong team of professionals worldwide. Through client business planning, customer research and strategy development we create solutions that work for real people in the real world.

For more information visit www.systra.co.uk

Birmingham – Newhall Street 5th Floor, Lancaster House, Newhall St, Birmingham, B3 1NQ T: +44 (0)121 393 4841

Birmingham – Edmund Gardens 1 Edmund Gardens, 121 Edmund Street, Birmingham B3 2HJ T: +44 (0)121 393 4841

Dublin 2nd Floor, Riverview House, 21-23 City Quay Dublin 2, Ireland T: +353 (0) 1 566 2028

Edinburgh – Thistle Street Prospect House, 5 Thistle Street, Edinburgh EH2 1DF United Kingdom T: +44 (0)131 460 1847

Glasgow – St Vincent St Seventh Floor, 124 St Vincent Street Glasgow G2 5HF United Kingdom T: +44 (0)141 468 4205

Glasgow – West George St 250 West George Street, Glasgow, G2 4QY T: +44 (0)141 468 4205

Leeds 100 Wellington Street, Leeds, LS1 1BA T: +44 (0)113 360 4842

London 3rd Floor, 5 Old Bailey, London EC4M 7BA United Kingdom T: +44 (0)20 3855 0079

Manchester – 16th Floor, City Tower 16th Floor, City Tower, Piccadilly Plaza Manchester M1 4BT United Kingdom T: +44 (0)161 504 5026

Newcastle Floor B, South Corridor, Milburn House, Dean Street, Newcastle, NE1 1LE United Kingdom T: +44 (0)191 249 3816

Perth 13 Rose Terrace, Perth PH1 5HA T: +44 (0)131 460 1847

Reading Soane Point, 6-8 Market Place, Reading, Berkshire, RG1 2EG T: +44 (0)118 206 0220

Woking Dukes Court, Duke Street Woking, Surrey GU21 5BH United Kingdom T: +44 (0)1483 357705

Other locations:

France: Bordeaux, Lille, Lyon, Marseille, Paris

Northern Europe: Astana, Copenhagen, Kiev, London, Moscow, Riga, Wroclaw

Southern Europe & Mediterranean: Algiers, Baku, Bucharest, Madrid, Rabat, Rome, Sofia, Tunis

Middle East: Cairo, Dubai, Riyadh

Asia Pacific: Bangkok, Beijing, Brisbane, Delhi, Hanoi, Hong Kong, Manila, Seoul, Shanghai, Singapore, Shenzhen, Taipei

Africa: Abidjan, Douala, Johannesburg, Kinshasa, Libreville, Nairobi

Latin America: Lima, Mexico, Rio de Janeiro, Santiago, São Paulo

North America: Little Falls, Los Angeles, Montreal, New-York, Philadelphia, Washington