The Role of Indicators, Targets and Monitoring in Decision-Support for Transport

<jats:p>Indicators can be seen as crucial to the transport planning processes as they can be used for decision-support in all aspects of policy development and assessment from option generation, through policy testing and appraisal to monitoring and feedback. Indicators are however, also seen by many practitioners as artificially focussing transport policy on what can be counted and on which indicators can most easily be improved. This paper presents the findings of a four year study into the use of indicators in the UK. In response to requests from local government practitioners, the research developed advice on how to develop a coherent indicator framework. The framework was tested through application in four case studies. The implications of the application of the framework are described in the paper and the degree to which the main criticisms of indicators can be overcome assessed. Many of the criticisms levelled at indicators such as their role in the distortion of decision-making processes arise not because indicators inevitably lead to significant conflicts but because of poor internal planning processes or unnecessary external constraints. The evidence presented in this paper suggests that, developed properly, an indicator framework is an essential part of an effective transport planning process for internal management and decision support and for external communication. Advice is provided on the conditions necessary for the development of an effective framework.</jats:p>

section_content: This The Role of Indicators, Targets and Monitoring in Decision-Support for Transport placed a particular emphasis on research which met the needs of practitioners. It also sought research proposals which were multi-disciplinary, reflecting the complex nature of the problems to be tackled, and multi-institutional, given a concern that no one institution might have the critical mass of research skills needed. 

The DISTILLATE programme responded to these challenges by involving local authorities and related actors directly in the research programme and by bringing together the research skills of two interdisciplinary transport research groups, a planning school, a policy-oriented research centre, and a national research establishment. It was designed to help overcome those barriers to decision-making which were judged to be most serious, and most amenable to research-led solutions. It set itself a vision of helping to achieve a step change in the way in which sustainable urban transport and land use strategies are developed and delivered. Further details of the programme as a whole, and of the role of the project reported in this paper, are provided in the overview paper (May, 2009). 

This paper presents the findings from the research within DISTILLATE into the use and application of indicator sets throughout the local, sub-regional and regional transport planning decision-making process in the UK. Indicators can be seen as crucial to the transport planning processes as they can be used for decision-support in all aspects of policy development and assessment from option generation, through policy testing and appraisal to monitoring and feedback. 

Indicators range from counting what has been spent, through what has been built to the actual impact of the investment (such as road casualties or air quality) and they can be collected and reported at different points in time. For many indicators, such as traffic flows and delay, there has been a long tradition of recording and forecasting, whilst for others policy demands have more recently emerged for example relating to equity and inclusion (Solomon, 2004). Whilst for the most part, indicators have been used for internal management processes and to demonstrate accountability for spending (Carter et al., 1992), increasingly public authorities are being asked to demonstrate how they are meeting their goals and how well they are serving their citizens (Talbot, 1994). This places new demands on selecting indicators. A further important contextual development is that improving information and communications technology has provided access to more information which could be used as part of the decision-making process. Deciding which pieces of information are important, why and how they will be obtained and used is therefore a more challenging task. 

As Hull established in 2004 through the initial survey of 16 local authorities involved in DISTILLATE (Hull, 2009) there existed widespread dissatisfaction with indicators relative to other parts of the decision-making process. Criticisms focussed on inconsistency in their application throughout the decision-making process, a lack of connectivity to the public and politicians, too many conflicting demands for information and the overarching impression that "what is counted is what counts" and that, as it is difficult to identify indicators for some policy areas, this distorts the decision-making process (Marsden and Kelly, 2005). 

This paper describes an approach to developing a logical framework of indicators for a local or regional authority and explains how it has been tested and applied in the UK. Whilst grounded in UK experience, the principles are generic to the European context. The paper is organised as follows: Section 2 introduces some definitions which are used throughout the paper, Section 3 describes the aims of the research and the methods used, Section 4 then introduces the logical framework of indicators and how this approach has been applied. Section 5 presents some of the key barriers to effective indicator use which were highlighted by the local authorities in DISTILLATE and describes how the indicator framework can and has been used to overcome these before Section 6 presents the key conclusions from the work.
section_name: Introduction

section_content: Whilst different approaches to planning can be adopted, an objectives-led planning approach is most common in transport (Meyer and Miller, 2001). Objectives are a set of goals that should be worked towards (May, 1996; WebTAG, 2008), they can be aspirational (for example improving the local economy) or quantifiable (such as ensuring less than 5 days of poor air quality per year). 

Where quantified goals are specified to be achieved by a particular point in time (e. g. by 2015), they become targets. 

There are several different types of indicators that can be measured as part of an indicator framework. The UK Audit Commission (2000) suggests the following broad categorisation:

• Outcome -the actual impact and the value of the service delivery -i. e. achievement of objectives (e. g. delay per person kilometre, fatalities)

• Intermediate outcome -the changes to the transport system that can be observed (e. g. the number of bus users, the number of kilometres travelled, 85 th percentile speed)

• Output -the service provided to the public (e. g. the number of bus miles run)

• Input -the resources employed to provide the service (e. g. amount of tarmac laid)

• Cost -the money spent to acquire the resources (e. g. transport plan expenditure)

These indicator types can also be used in combination. For example, the cost effectiveness of a set of safety interventions might be assessed by looking at the unit cost of a reduction in casualties. 

Similarly, the efficiency of an investment programme might be examined by looking at the unit cost of an output (e. g. cost of every kilometre of road resurfacing conducted) (Ibid. ). 

Within transport the category of intermediate outcomes warrants special attention. These are typically the transport system usage variables that are easily measurable (e. g. traffic flow) and which are directly responsive to policy measures. Whilst these are interesting system variables they contribute towards the achievements of outcomes rather than being objectives to be achieved in their own right. For example, increasing bus use might contribute to reduced congestion and pollution but is not a policy objective of itself.
section_name: Definitions

section_name: Aims and Methods

section_content: The initially formulated aims of this research were to specify the requirements for a core set of indicators at each stage in the decision-making process and, subsequently, to identify a core set of outcome indicators that best meets those requirements. Initial research with the local authority partners suggested however that "another set of indicators" was the last thing that would be useful to them as there are several already in existence, many of which are already largely ignored (Figure 1 ). It is worth noting that the indicator sets which have greatest importance to the authorities are those that most directly impact on their funding allocations and assessment scores (Local Transport Plan and Public Service Agreements). We do not comment further on the strengths and weaknesses of the particular indicator sets but note that many countries place at least some top-down data requirements on their cities and that cities therefore have to organise their monitoring frameworks in a partially constrained manner (May et al., 2008). The concerns the practitioners raised were about how to decide what to include in their indicator frameworks and how to make best use of these in developing and assessing the progress with their transport strategies. 

The main aims of the project were therefore reformulated to be:

The Role of Indicators, Targets and Monitoring in Decision-Support for Transport 1. To establish a structured approach to developing an indicator framework 2. To test the usefulness of the approach in real case studies and amend accordingly 3. To demonstrate how the framework can and has been applied to solve common policy problems that relate to indicator development and application. 

These aims were to be met through a process of partnered enquiry with a group of local authorities, each of which offered a case study of the use of indicators.
section_name: Aims

section_name: 0%

section_content: The key methods employed to facilitate the partnered enquiry were:

• Desktop review;

• Interviews; and

• Workshops. 

The structured approach to the indicator framework was developed through a combination of desktop review and interviews. The desk-top review identified the range of indicator types, the issues with applying and managing information across multiple information sources and the problems caused by partial monitoring of system impacts. Interviews with practitioners leading the development of their LTP monitoring framework were used to discuss the usefulness of a framework for monitoring and other issues which were not identified in the review. In all, twenty five interviews were conducted and the findings of the research were tested at a national, regional, sub-regional and local scale. 

The Role of Indicators, Targets and Monitoring in Decision-Support for Transport

The application of the framework was then further assessed through four case studies. All of these case studies were conducted after the framework had been developed and were used, in different ways, to test the usefulness of the framework as set out below. The first case study investigated the application of the framework in real-world strategy development. This involved working with a sub-regional Transport Passenger Executive (PTE) at the transport plan development stage and a in subsequent round of interviews with a wider range of stakeholders engaged with the plan delivery. The second analysed the extent to which common indicators are used across different departments within five local authorities and was based on desk-top review. This addressed issues of data integration from multiple sources and was an analysis by the research team of the extent to which the framework might assist with integration. The third was a case study of the main difficulties in information exchange to connect planning and transport decisions between a local planning context and a wider transport planning authority area. It therefore looked at integrating information from the smallest spatial scale (bottom-up). 

The final case study considered the application of indicators in transport and land-use planning at a regional, sub-regional and local scale and was conducted with the regional layer (top-down, although substantial national influence was also identified). Case study 1 has been used to illustrate the application of the indicator framework in Section 4 as it most closely mirrored the assembly of the framework. Evidence on how this approach assisted with communication of monitoring requirements is used in Section 5 along with evidence from the other case studies which are used to discuss the extent to which a more formal indicator framework might contribute to resolving some of the practitioner identified problems. 

The UK has a different formal institutional structure from other European countries (Zografos et al., 2004 ) so the detail of the findings relating to different government levels should be considered to be location specific. The implications of information sharing across different layers of government are however prevalent elsewhere and are therefore of more generic value (May et al., 2008).
section_name: Methods

section_content: Section 2 of the paper introduced the four different categories of indicators (ranging from outcome to output). We argue that for a monitoring framework to be comprehensive in supporting decision-making, the different types of indicators have to be joined together. In this way, the reason for collecting information or deriving modelled estimations of any indicator is clear. Whilst this may sound obvious, the first round of local transport plans was characterised in a number of authorities by the inclusion of too many indicators and the setting of too many targets which authorities either did not know how to meet or could not actually monitor (Atkins, 2007). 

This section presents the findings from our step-wise guide to developing an indicator framework based on a causal chain analysis. The framework approach is similar to the International Results Based Management literature described in (CIDA, 2000 and UNDP, 2002) which links the key outcomes that are to be achieved to the indicators that can be measured to provide a clearer linkage between what we measure and why. The categories of indicators selected are those defined by the UK government (DfT, 2004) and as set out in Section 2. 

One of the key challenges in selecting indicators is to balance the desire to be comprehensive with the need to be able to manage the information and ultimately resource its collection. The section also therefore includes a process for prioritising which indicators from the hypothetical set available should be adopted. 

The example in this instance is for a city which is aiming to increase its productivity and to cut climate change emissions. The hypothetical strategy approach on which the remaining indicators

The Role of Indicators, Targets and Monitoring in Decision-Support for Transport are based for this example is one based around promoting a mode shift to less CO 2 emitting forms of transport. It is worth noting here that a different strategy would in turn lead to a different selection of output indicators. Dependent on the focus of the strategy it may also change which intermediate outcomes are most relevant to include. The framework is simply an organising tool. The process of developing links between the different levels is set out more fully below but should be based on discussion between experts in the field with appropriate supporting evidence. 

The section presents the different steps in the development of the framework alongside feedback from the Merseyside Local Transport Plan partnership (case study one outlined above) which employed the methodology in developing the monitoring framework for the five-year Local Transport Plan which was submitted to central government in 2006. The indicator selection approach was used in order to test the extent to which it could help shorten, refine, and redevelop the indicator set used within the sub regional transport plan. Gaps and evidence of imbalance in the indicator were highlighted and fed back to those involved in the development of the plan. This research was conducted in 2005-6, and the resulting conceptual map was included in the final local transport plan.
section_name: Developing an Indicator Framework for Decision-Support

section_content: The primary objective of an indicator framework is to support effective decision-support with the aim of furthering the achievement of the objectives of the authority. The objectives need to be comprehensive as if the objectives provide only partial coverage of progress towards sustainability then the chosen indicator set will likewise be partial in its reporting on important sustainability impacts (Marsden et al., 2007). It is important to specify a full set of objectives and to do this first. A number of attempts have been made to develop holistic sustainable transport objectives (TRB, 2008; Jeon and Amekudzi, 2005). Different administrative areas are likely to have different priorities that require different objectives. If the available indicators are used to determine the objectives then it may be that policy issues which are important but which are not currently measured or forecast will be marginalised and thus distort decision-making. 

Although a full set of objectives is a requirement for a transport strategy, for the sake of simplicity, the example presented in this paper is based around just two objectives, tackling climate change (DfT, 2008) and increasing economic productivity (see Eddington, 2006). These objectives are shown in the boxes in the column to the far left of Figure 2. The following sections explain how to link indicators to these objectives.
section_name: Starting with objectives

section_content: Once the objectives have been set it is necessary to identify the key outcome indicators which will represent progress towards these objectives. In the example framework in Figure 2, productivity at a city level can be increased by reducing congestion externalities (Graham 2006 ) so a congestion measure -journey time per mile -has been selected as the outcome indicator for this objective and per capita CO2 has been selected to represent climate change mitigation efforts. Note that both of these indicators form part of the set of 198 indicators which all local authorities are required to report on in the UK (DCLG, 2007) 4. 

Evidence from the Merseyside case study shows that linking the indicators in this manner is a valuable and effective exercise. Following the mapping exercise, it was recommended to the LTP partnership that more emphasis needed to be placed on the causal links between air pollution and congestion, and that perhaps congestion should feature amongst the outcomes. The LTP monitoring team was already engaged in collecting information for these indicators, but were 'underselling' the sustainability dimension of their indicator set.
section_name: Linking Objectives and Outcomes

section_content: Outcome indicators reflect the changes in the objectives of our policies. The relationship between the transport interventions we make and the changes in objectives can sometimes be weak or at least operate over long timescales. It can therefore be difficult to make the right connections between outcomes and intermediate outcomes. 

How should the right package of intermediate outcome measures be selected? It seems most sensible for these to be guided by a combination of the objectives and outcomes initially (top down). The selection should then be moderated based on the strategy that is adopted (bottom up) as some strategies may offer very little prospect of influencing some indicators. So, initially, the list of intermediate outcomes are those which are thought to be connected to changes in journey time per vehicle mile and in the transport related carbon used per person in the local area. Nine intermediate outcomes were initially identified (third column from the left of Figure 2 ) which include mode usage figures, total traffic levels, speeds, vehicle occupancy and vehicle emission rates. The vertical arrow on the diagram indicates the likely interrelationship between changes in the use of different modes. At this stage, once the intermediate outcomes of greatest importance appear to have been identified then the indicator framework can be used to help inform option generation and assessment processes (Jones, 2009) and to focus on the most useful outputs from any modelling exercises conducted (Shepherd et al., 2006). Ideally, therefore, the options put forward for consideration and the preferred solution will be ones which achieve the biggest achievements on the outcome and intermediate outcome indicators selected. 

Once again, considering indicators in this systematic manner with the Merseyside monitoring team allowed the application of this framework, and enabled their proposed indicator set to be critically assessed. The Merseyside indicator map 5 and the process of developing the map raised a number of general points about the balance of the indicator set. The researchers recommended that several indicators were amended -for example whilst the strategy objectives placed a significant emphasis on access and public transport, these priorities were not immediately obvious in the chosen set of intermediate outcomes.
section_name: Linking Outcomes and Intermediate Outcomes

section_content: Once the preferred strategy has been identified then the output indicators should be selected to capture the implementation of the strategy. This enables the decision-maker to differentiate between a strategy which is not having the desired impact on intermediate outcomes and outcomes because it has not yet been fully implemented and one which has been implemented but does not seem to be effective. It is at this stage too that a more rational bottom up approach to prioritising the intermediate outcome indicators can be applied and this is discussed in Section 4. 5. Note here that the strategy defines the outputs and, along with the outcomes, the intermediate outcomes. Whilst it is most common for outputs to connect to intermediate outcomes, some may go directly to outcomes (e. g. accident black spot mitigation schemes may lead directly to casualty reductions). The example in Figure 2 (column to the far right) shows a series of output indicators identified for improvements to the bus network, cycling and walking and vehicle emission rates. 

Improvements to speed and changes in road traffic levels, for the sake of simplification, are assumed to follow as a result of the mode shift induced by the other measures. 

Outputs provide management information about what has been implemented and when. They also play a potentially important role in policy justification and debate (the tangible changes) at a local and a central government level. Inputs such as finances, materials and time should be logged as appropriate. This information can be extremely important in benchmarking how effective authorities are at delivering similar types of schemes (Gudmundsson et al., 2005).
section_name: Linking Outputs and Intermediate Outcomes

section_content: As well as being used to support option generation and modelling exercises, indicator frameworks form the basis of monitoring strategies. During our research the under-funding of monitoring in many local authority areas was seen as a deficiency in developing transport strategies. Indeed, monitoring was described by one monitoring officer as "the art of the possible". Of course, monitoring has to compete with other transport and non-transport priorities for resources and it will not be possible to monitor everything. Our research indicated the need to develop a prioritisation method and one is proposed below (Table 1 ). 

To demonstrate the principles of prioritisation, it has been assumed in our example from Figure 2 that the authority would not be able to afford to monitor and track all of these indicators. 

Step 1 identifies six indicators as being from the list of mandatory national indicators so these must be retained (those marked with NI). 

Step 2 shows that road traffic speeds, traffic flows and the number of schools with a travel plan are already monitored by the council. 

Step 3 begins by assessing the intermediate outcomes that remain. These are average vehicle occupancy, walk trips, cycle trips and fleet technology mix. 

Step 4 looks at the prioritisation. (Figure 3 shows those indicators which are not selected as plain text boxes). 6 As no specific policy measures have been included to tackle vehicle occupancy within the strategy this indicator seems to be low priority and could be excluded. Vehicle emission rates has been excluded on the grounds that this is a local plan and many of the policies influencing this are at a national level and this would be a comparatively weak area to focus monitoring on. Walking and cycling levels however are an important part of the policy mix through the school travel plans and investment in cycling infrastructure so these are higher priorities and are selected. There may be some overlap between monitoring of school travel and overall walk and cycle levels and the data collection processes should take account of this. 

Having completed the prioritisation of intermediate outcomes it is necessary to revisit the output indicators. As the fleet technology mix has been excluded then both of the average new car CO 2 and % Euro IV buses can be removed from the framework. Likewise, in this simplified example, as improved physical accessibility is not an objective then the percentage of low floor buses was excluded on the grounds it would have a limited impact on bus journey numbers given the current state of the fleet. Having applied the process described above to the Merseyside case study, several overlaps were found to exist between a number of non-mandatory intermediate outcome indicators. As a result, the researchers were able to recommend that these overlapping indicators be reconsidered in order to reduce the size of the indicator set. It was also evident that the output indicators could be more closely related to the outcome and intermediate outcome indicators. Over and above this, the process of mapping out the indicators systematically through this framework provided a number of additional benefits. First, the indicator set was reduced dramatically by comparison with the first LTP. Secondly, feedback from Merseyside staff suggests that the process described above is useful not just at the indicator development stage, but also as the indicators are measured and targets are reviewed, helping to identify how improvements in one indicator may have a knock on effect in another. Thirdly, the map acted as a simple visual aid, and it was considered a useful tool to gain support from elected members. The map was considered to be a useful communication tool to other organisations involved in using, collecting, or monitoring the indicators. Finally, the indicator map was a useful, succinct way of demonstrating appropriate indicator choices to the central government funder in the light of a greater focus on outcomes compared to previous transport plans.
section_name: Prioritising indicators

section_content: Based on the Merseyside case study, the indicator framework approach seems to offer a practical route to rationalise what is an increasingly complex management problem. Causal chain analysis is a useful basis for setting up the linkages between different indicators. It is worth offering a note of caution however that the linkages are only as good as the assumptions underpinning them and, where substantial changes occur in one indicator that are expected to impact on another but do not then the links may need revisiting. This is a natural part of the monitoring and feedback loop. If the approach described here has been followed however it will be more obvious to the decision-maker whether it is the implementation or the strategy which is at fault. The framework approach described in Section 4 provides a basis for understanding what information is used in decision-making and process monitoring and why. However, there are some very sticky real-world problems which face authorities seeking to implement and understand their strategies. This section presents and discusses some of the main issues encountered by the research team in discussions with practitioners about developing and applying indicator sets. The section discusses these issues and explains how the framework and its application might help to overcome some of these and to provide an objective evidence base to counter others.
section_name: Realities in monitoring and decision-making

section_content: Transport is a contributor rather than the sole link to most objectives. For example, transport is seen as an important element of the multi-faceted approach to tackling the obesity epidemic in the UK (NAO et al., 2006). The decline in walking and cycling trips as main modes or as part of journeys has been identified as one of the factors which has contributed to increased obesity. So initiatives that increase walking and cycling should improve health but the links are complex, interwoven with other lifestyle choices and long-term in nature and there is no systematic evidence on the impact of transport schemes on physical activity (NICE, 2008). Similarly, the link between transport and the economy is strong but complex (Eddington, 2006). 

In the face of such uncertainty, it might seem unrealistic to select and link outcome indicators and intermediate outcomes, at least in a firm way. Changes that might be observed may also be dwarfed by other effects (e. g. a factory closure or other economic downturn). One suggested solution is that "Authorities should be encouraged to undertake, and disseminate the results of, scheme specific before and after surveys to provide more evidence of cause and effect at the local level and overcome the fact that current aggregate data may not be the most suitable for informing strategy development. " (Atkins, 2007, pv) Indicators with unclear relationships between them can still be used as part of the monitoring framework. It will not always be appropriate to monitor these indicators at the city or regional level as the detail of changes may be lost. More localised monitoring can be used to test the assumptions about the degree and/or direction of relationships between interventions and indicators which in turn can be used to refine the more macro-level indicator framework.
section_name: The relationships between some objectives and transport interventions are unclear

section_content: Aggregation of data is a necessary feature of indicators. It would be useless to publish the speed of every vehicle on every link of a network during the morning peak as a measure of congestion. Without aggregation the information cannot be meaningfully processed but the greater the degree of aggregation the more data is lost to the user. Aggregation of data to a city or subregional level (as is the case in metropolitan areas) can hide important variations in behaviour across areas and between different groups. For example, in metropolitan areas in the UK up to 10 district councils collaborate to produce a joint local transport plan. Conurbation-wide reporting of changes in bus patronage hides the location and nature of increases and declines in particular routes. Where major interventions have been put in place, such as new quality corridor arrangements, then more scheme specific monitoring is desirable. 

Some indicators are derived from a more site specific measurement and aggregation (e. g. road traffic accidents) and it is therefore possible to present the data at different levels of aggregation to meet the target audience's requirement. However, where this is not possible it is important to identify the end-use of the data to define what it is appropriate to collect. Central government might want an overall view of speeds in major cities on an annual basis for comparison, while the information of importance to transport engineers will be link specific, highly disaggregated but The Role of Indicators, Targets and Monitoring in Decision-Support for Transport historic to prioritise interventions and that of relevance to travellers will be of their route options potentially in real-time to influence their mode, route and departure time choices.
section_name: Aggregation of data hides progress

section_content: A review of information requirement relationships between central and regional and/or local government in 17 countries conducted by the ECMT suggests that although some countries have entirely nationally managed data collection processes that monitoring in 10 of the 17 involved two or three tiers of government (May et al., 2008). The approach adopted in this paper suggests that it is critical that each decision-making tier is clear about what it needs to collect and why. Experience from the case studies suggests however that making these assessments in isolation from each other will lead to the placing of monitoring burdens on lower tier authorities, a lack of strategic fit between indicators and ultimately non-compliance in a number of instances. 

In one study area regional government had made efforts to co-ordinate its data gathering around information that its constituent local authorities were required to supply to central government. This was subsequently undermined by changes to monitoring processes at the central government level. Insufficient thought seems to be given to the interconnectivity of local, regional and national data collection process in the UK context. This is unlikely to be specifically a UK problem however as May et al. (2008) found that the four biggest problems highlighted by their survey participants were:

• Insufficient resources for systematic collection and monitoring

• Incomparability of different surveys (incomplete data sets)

• Lack of consistency in definitions

• Poor institutional coordination among levels of government. 

To be really effective across different spatial scales appears to require joined up processes for monitoring, adequate funding and better co-ordination. 

Case Study 4 exemplified the difficulties in connecting national, regional and local requirements for monitoring accessibility to key public services by public transport. The UK Government requires local authorities to develop an accessibility strategy as part of their five year local transport plans and the Regional Assembly has sought to further interpret this guidance with a regional accessibility indicator which tracks the "percentage of new residential development within 30 minutes public transport time of a GP, hospital, primary and secondary school, employment and a major health centre". The indicator is not currently monitored by Leeds City Council, the largest city in the region. Leeds suggests that "It is an ambiguously worded indicator. As drafted it is not clear whether it refers to six separate indicators of accessibility or whether, to meet it, development has to be within 30 minutes public transport time of all six subindicators. " (LCC, 2006, p. 24). Further discussions revealed a perception that for this measure, the regional requirement added nothing to existing local processes and therefore was unlikely to be adopted. Neither at a national nor a regional level is it clear what thresholds of accessibility (in terms of proportions of the population or new developments) should be aimed for. This is a good example of an idea which might have some merit not being fully developed. It is not clear what exactly is to be monitored nor why the layer of government being asked to do it should do so. It doesn't fit in the authority's indicator framework, has some duplication of existing processes and ultimately is neglected. 

Our case studies have led us to conclude that there will be some national requirements for information but that these should be kept to a minimum and, where these are not likely to be core to the local government's aims should be adequately resourced or, potentially, collected separately and fed back to the local authorities. Requirements for other data should be

The Role of Indicators, Targets and Monitoring in Decision-Support for Transport established using a bottom up approach which examines the processes already being applied and the data already being collected. If standardisation is required the reasons for this need to be absolutely clear (e. g. benchmarking, lower collection costs) and a coalition based approach which is resourced should be used to bring it about. If a regional body identifies indicators which are core to its strategy which is not available from local sources then it should fund the collection of the data directly.
section_name: Information requirements from other spatial tiers waste effort

section_content: The information is held by different groups with different objectives

Our second case study has provided insight into the number of different organisations that might be involved in indicator development, monitoring and evaluation. The different remits of these organisations clearly affect the way in which the indicators are viewed. For example, those working directly with transport plans were more likely to be working in the policy area related to the indicator. This meant that they had a direct input into policy development, and were therefore more likely to have a direct responsibility for meeting targets. Some had more control than others over the setting of targets, and where necessary their renegotiation. On the other hand, those not directly working in a transport setting tend to have limited responsibility over indicator development, target setting, or meeting targets. Interviewees falling into this category had little sense of individual responsibility for missed targets or poor performance. The main task for these actors was to collect and consolidate data, often in partnership with other organisations. These differences are characterised in Table 2.
section_name: 5.4

section_content: LEAST INTERACTION: The technical experts -e. g. Environmental Health Officers in Local Authorities are unlikely to have an effect on policy even if there are targets associated with their work. They are most likely to be involved in the technical, monitoring side. From their perspective there is little they can do to drive policy as this is the role of transport departments, bus operators and the sub regional bodies. They can recommend, but do not play a central role in policy development or evaluation. Targets are less likely in this area, but where they do exist there is a grey area about who is responsible for failure to meet them.
section_name: Table 2. Indicators and policy interaction

section_content: These have a role that interacts more with policy than the technical experts, but their work may not be central to the priorities of the local transport plan. These actors are likely to be involved in policy areas that are not politically popular, but mandatory from a national perspective. 

MOST INTERACTION: Central actors -These are likely to have a role that drives policy (albeit in the face of political adversity). The issues the central actors deal with are too big at either the local or national level to be sidelined Whilst experiences varied, those who had less direct involvement with policy development and the evaluation of targets viewed indicators as being an 'add on' where data had to be collected at certain times of the year in certain ways. This was generally viewed as arduous, especially given difficulties getting data from different organisations or departments with different priorities, collection methods, and timeframes. In an increasingly fragmented decision-making environment it is important to map out the relationships between agents responsible for different parts of the data management process.
section_name: SOME INTERACTION Peripheral actors -

section_content: The premise of Section 4 of this paper was to work out what is important to the decision-making body and then develop indicators that reflect that. This would suggest that 'counting' what matters should be the aim and if an indicator does not seem to have a connection in the system then its relevance should be questioned. The concerns raised by participants are more subtle than this, focussing on the following weaknesses in the idealised process described earlier:

The Role of Indicators, Targets and Monitoring in Decision-Support for Transport Some indicators are required by central government, these can have funding rewards attached to progress and this in turn artificially inflates their importance relative to other local priorities. 

Evidence from an assessment of target setting and performance rewards in transport suggests that such incentives do encourage greater performance in those areas that are rewarded (Marsden et al., 2009). It is therefore essential that central governments reward achievement of local objectives as well as national ones if this approach is adopted. 

Some problems are harder to measure than others, and there is a tendency to marginalise these (unless there is a legal requirement to measure them). Section 5. 1 suggests that there are examples of this in transport and that these can be overcome with suitable use of proxy indictors or scheme specific monitoring. It has to be accepted that some issues are difficult to measure including quality of life, well-being and option value. Where these cannot be monitored it does not mean they should be ignored. If they are suitably integrated into option generation processes then the strategies which are brought forward and monitored should be consistent with these issues. Qualitative indicators can also be used in review processes where required. 

Authorities are encouraged to play games with the measurement processes in order to artificially inflate performance (Smith, 1995). Standardised measurement processes help to reduce this and are generally supported for consistency and cost-effectiveness (May et al., 2008). An audit or the threat of audit of the measurement process can also deter such behaviour. This is more of an issue where the authorities stand to gain or lose financially from their performance and therefore applies to only a few contexts such as the UK. 

In summary, it is critical to count what counts! What counts needs to be generated from the perspective of the lower tier of government's consideration of the key objectives of an area. Monitoring requirements imposed from national or regional government should give due weight to these priorities relative to national measures or risk distorting local priorities.
section_name: What is counted is what counts

section_content: Indicators are central to decision-support within transport. One key role is in mapping progress against objectives through outcome indicators. The development of a framework of indicators needs to be undertaken concurrently with the development of objectives. Where indicator systems are developed in isolation from the objectives the reason for the indicators can be lost and there are risks that the indicators start to drive rather than support the decision-making process. A framework of indicators based on objectives can then be used consistently in the processes of option generation, prediction and appraisal. We have adopted this approach ourselves in the parallel projects on these three themes (Jones et al, 2009; Shepherd et al, 2009; Page et al, 2009) all of which draw on work described in this paper. 

Indicators also form a critical part of the monitoring and evaluation process for strategy implementation allowing the assessment of what is spent, what is implemented and what actually happens. It is important that once a strategy is selected, an appropriate set of output indicators selected and that these are fully connected to the intermediate outcome and outcome indicators, and through these to the objectives. 

This paper has presented a rational approach to the development of an indicator framework to support all of these activities. The framework, based on a causal chain approach, has been tested in a series of applications which have highlighted a number of limitations in the current approach to developing indicator sets and offered insights into where the proposed approach will improve the decision-making process. 

In particular, the research suggests that the lower-tier authorities must have sufficient freedom to develop a locally relevant set of indicators. Where national or regional bodies wish to influence this then this should be primarily through guidance on a range of possible indicators and standard measurement processes. Where information is required by a higher spatial tier but is of little use to the lower tier then collection should be resourced or undertaken by the higher tier. 

The development of an indicator framework has also been valuable in communicating to stakeholders what they are responsible for and where it fits within the overall plan. This has also been seen as a valuable tool to communicate with local politicians and national policy makers. 

Given the increasingly fragmented delivery structures within which transport policy operates, this approach seems likely to grow in importance over time. 

In summary, many of the criticisms levelled at indicators such as their role in the distortion of decision-making processes arise not because indicators inevitably lead to significant conflicts but because of poor internal planning processes or unnecessary external constraints. The evidence presented in this paper suggests that, developed properly, an indicator framework is an essential part of an effective transport planning process for internal management and decision support and for external communication.
section_name: Conclusions

section_name: Acknowledgements

section_content: DISTILLATE was funded by the UK Engineering and Physical Sciences Research Council under its Sustainable Urban Environment programme, and supported by sixteen UK local authorities and a steering group including representatives of the International Transport Forum, the European Commission, the UK Department for Transport and Transport Scotland. We are grateful to all of these bodies for their support for the research. The research reported here involved several other colleagues, including Charlotte Kelly, Sotiris Thanos, Mary Kimble, John Forrester., Reggie Tricker, Angela Hull, Karen Lucas, Michael Brookes and Tony May. The conclusions are, however, are own.
section_name: Acknowledgements