Dealing with Dilemma’s: How Can Experiments Contribute to a More Sustainable Mobility System?

<jats:p>Sustainable mobility has proved to be a perennial challenge to realize. Scholars have argued that experiments could point the way forward towards sustainable mobility (cf. Loorbach, 2007, Markard and Truffer, 2008). In doing so, literature attributes a vital but complex task to those who engage in experiments. However, an important knowledge gap pertains to whether and how experiments contribute to learning about transitions and in what way they should be managed to break-up the more or less inertial mobility governance system. This paper aims to analyze how state-of-the-art literature on the governance of multi-actor systems considers experiments to contribute to transitions and highlight key dilemma’s that professionals engaged in the management of experimental face in the day-to-day management and decision making processes during the experiment. The paper will highlight these dilemmas and choices and illustrate their importance for experiments in the field of transportation and more specifically in the specific context of the Dutch mobility system and the TRANSUMO research program. Identifying these dilemma’s benefits practitioners who are engaged in the management of experiments to more consciously reflect on and include issues of second-order learning in the day-to-day management and decision making during the experiment to reach a more sustainable mobile system.</jats:p>

section_content: The majority of our public infrastructure systems seem to have reached a condition where they increasingly contribute to environmental problems in our society (i. e. air pollution, noise pollution). The underlying causes for this state of affairs is often attributed to the current (technological) governance regimes that seem to block the emergence, development and breakthrough of more sustainable innovations in public infrastructure systems (cf. Hekkert and van den Hoed, 2006, Rotmans, 2003). It seems as though most public infrastructures have evolved into inertial or even 'solidified' systems. 

Sustainable mobility in particular has proved to be a perennial challenge to realize. Many countries that face the downsides of mobility try to change their transport systems into more sustainable systems. In the Netherlands, the desire to develop more sustainable transport has been around for at least 15 years (cf. Nijkamp, 1994, Nijkamp et al., 1995). During this period, many experiments to develop a more sustainable transport infrastructure have been held with a variety of success (van den Bergh et al., 2007, van der Straten et al., 2007). But, so far, any progress in this respect to reach a more sustainable transport infrastructure has been offset by a more intensive use of transportation. This holds true for both air transportation (cf. de Haan, 2007) as well as road transportation (e. g. Geels, 2007, Rotmans, 2003) and concerns both the development and use of innovative technologies as well as efforts to bring about behavioral change. 

The future development of mobility is very uncertain and a wide variety of possibilities to reach desirable end-states exist. To increase the chance of success of breaking through solidified multiactor systems, a set of approaches have been formulated in different disciplines, ranging from network and stakeholder management to strategic niche management and innovation management. In the last decade, transition management has been developed as a specific approach to reach a more sustainable society. Transition management defines a transition as "a gradual process of societal change in which society or an important subsystem of society structurally changes" (Rotmans et al. 2000, p. 19). The approach has been accepted and applied as a policy tool to reach a more sustainable mobility system in the Netherlands, resulting in 2004 in the creation of a national public-private research program called TRANSUMO (Avelino, 2009). In this research program both technological as well as behavioral innovations and experiments are investigated. The transition management approach assumes that transitions can be guided or 'managed' through a multi-level governance approach (cf. Rotmans, 2003, Loorbach, 2007). 

A central element in this approach is reserved for the development and execution of experiments to foster (trial-and-error) learning and thus pave the way towards sustainable transitions (van de Kerkhof and Wieczorek, 2005). In order to stimulate sustainable developments in the mobile system more radical and daring (technical) experiments may be considered vital tools (cf. Loorbach, 2007, Markard and Truffer, 2008). In doing so, literature attributes a vital but complex task to those who engage in experiments. To learn from experiments with regard to the development of new and more sustainable technologies and policies (first-order learning), but also to learn from the experiments from a transition management perspective (second-order learning) and thus learning about the values, assumptions and policies that drive the actions of stakeholders during the experiment (van de Kerkhof and Wieczorek, 2005). The two-sided goals of transition experiments (Kemp and Rotmans, 2004 ) can be distinguished by considering the managerial impact of the goals that experiments need to attain: "is this an innovation with potential in itself and does this innovation contribute to an overall transition?" (Loorbach, 2007, p. 111) However, an important knowledge gap pertains to whether and how experiments are able to contribute to this second-order learning and in what way they should be managed to break-up more or less inertial mobility governance system. State-of-the-art literature on sustainability and the governance of multi-actor systems includes network management (de Bruijn and ten Heuvelhof, 2008, Koppenjan and Klijn, 2004), innovation studies (cf. Geels, 2004, 2005, 2007, Hekkert et al., 2007) and transition management (cf. Elzen and Wieczorek, 2005, Hendriks and Grin, 2007). This literature is vague and ambiguous with regard to how experiments should be set up and managed in practice to contribute to transitions. This paper aims to analyze how state-of-the-art literature on the governance of multi-actor systems considers experiments to contribute to transitions. These issues are particularly interesting from an operational perspective as managers and innovators who are involved in the day-to-day management of transition experiments have to translate these principles into practice. The paper may enable both professions to more consciously include second-order learning as a goal in the day-to-day management and decision making processes during the experiment. The paper will highlight these dilemmas and choices and illustrate their importance for experiments in the field of transportation and more specifically in the specific context of the Dutch mobility system and the TRANSUMO research program. 

A short analysis of literature on change in multi-actor governance systems and the role of experiments is elaborated in section 1. Section 2 focuses on the important notions that have been identified in the management of experiments and draws a structured list of project design recommendations that professionals could use. Section 3 explores some inherent dilemma's and trade-offs that professionals engaged in the management of experiments face on a daily basis for which practical guidelines do not exist. Section 4 draws a number of conclusions on the consequences of the trade-offs and some potential lessons for those engaged in transportation policy programs that seek to reach a more sustainable transport industry.
section_name: Introduction 5

section_content: The point of departure is the assumption that systemic innovations that are necessary to reach sustainable solutions (Truffer et al., 2008). The sustainable transformation of the mobile system may be considered as a highly uncertain and unpredictable systemic governance and innovation problem where solutions are bound to take place within a multi-actor context. Different actors are involved in the development and implementation of innovations and changes in the mobile system, which together push and pull processes of dynamics and change (end users, technical developers, infrastructure operators, governments, etc. ). For example user requirements and perceptions play an important role in the adoption of successful transport innovations (cf. Wiegmans et al., 2007; van der Straten et al., 2007). This means that the adoption of innovations results from the interaction between different actors (e. g. both developers of innovation as well as users of innovations) and that no single actor has the ability to overrule other actors. From this perspective, it could be argued that the management of system innovations therefore takes on characteristics of the process management approach (de Bruijn et al., 2002) and in particular the network management aspects of it (de Bruijn and ten Heuvelhof, 2008, Kickert et al., 1997, Koppenjan and Klijn, 2004, de Bruijn et al., 2004). This governance perspective assumes that policy and production networks fail to respond to traditional top-down governance approaches. Or to put it simply, in today's complex environment in which the mobility system finds itself, simple top-down policies aimed to achieve towards more sustainability won't work anymore. 

This multi-actor perspective on governance and innovation management is further elaborated and refined in innovation studies that focus on the interplay between technology, actors and the interactions within institutional systems (e. g. Geels, 2004, Hekkert et al., 2007, Hughes, 1983, Geels, 2005, Geels, 2007, Summerton, 1994, Mayntz and Hughes, 1988, Markard and Truffer, 2008). The over-arching perspective of these studies is that changes in large-scale technological systems take place in institutionally fragmented settings and that a wide variety of factors (e. g. various institutional settings, the behavior of specific actors, the specific technologies in use, country-specific characteristics such as the population distribution or topographical characteristics) influence how large scale socio-technical systems eventually evolve (van der Vleuten, 1999 Vleuten,, 2004)). However, what is known is that the interplay between actors that are involved in the system plays an important part in the way in which systemic changes take place and in which these changes can be understood (Truffer et al., 2008). 

A final stream of literature that acknowledges the multi-actor governance nature of sustainability and addresses how to reach more sustainable future is the transition management literature (Rotmans, 2008 (Rotmans,, p. 1006)). Transition management addresses the issue how systems can change towards a desirable end-state. Transition management defines a transition as "a gradual process of societal change in which society or an important subsystem of society structurally changes" (Rotmans et al. 2000, p. 19), and "a process of structural societal change from one relatively stable system state to another" (Loorbach, 2007, p. 18). Transition management uses notions from complex adaptive systems theory (van der Brugge and van Raak, 2007, Loorbach, 2007). In doing so, transition management positions itself as a new policy approach that focuses on new forms of governance (cf. Hendriks and Grin, 2007, Elzen and Wieczorek, 2005) and self-organization that recognizes the need for multi-issue and multi-level governance whereby state and non-state actors are brought together to co-produce and coordinate policies in an interactive way (Rotmans et al., 2005, Loorbach, 2007). Multi-actor governance is thus inescapable as "[n]o single actor is able to grasp and control the full complexity of transitions" (Mourik and Raven, 2006, p. 8). 

Different types of transitions can be identified (cf. Smith et al., 2005 ), yet only one promises to provide adequate solutions to problems of sustainability in the mobile system: that of goaloriented transitions (cf. Geels, 2006, Smith et al., 2005, Geels and Schot, 2007, Kemp and Rotmans, 2004) 6. In the last decade, transition management has been developed as a specific approach to reach a more sustainable society. The approach has been accepted and applied as a policy tool in the Netherlands (e. g. EnergieTransitie, 2008, Kemp and Loorbach, 2005). So much so that to contribute to a more sustainable mobile system, the Dutch authorities established a national public-private research program called TRANSUMO and funded €30 million. TRANSUMO stands for TRANsition SUstainable Mobility (see www. transumo. nl). In TRANSUMO more than 150 participants from government, the transport industry and knowledge institutions work together. The aim of TRANSUMO is to develop and spread new knowledge for realizing a transition towards sustainable mobility in the Netherlands. In this research program both technological as well as behavioral solutions to mobility problems are investigated in 7 themes and 22 projects (cf. Avelino, 2009). 

All the abovementioned streams of literature focus on the possibilities for 'transition management'. It should be noted here that there exists disagreement with regard to the extent to which transitions can indeed be managed (cf. de Bruijn et al., 2004; de Bruijn and ten Heuvelhof, 2008; Rotmans, 2003; Smith et al., 2005; Loorbach, 2007; Teisman, 1998; Teisman and Edelenbos, 2004). 

As one of the key elements in transition management literature, the importance of experiments is widely recognized (cf. Rotmans, 2003, p. 96; Kemp and Rotmans, 2004, p. 146; Loorbach, 2007, p. 115). Experiments aimed at innovation play a key part in reaching future goals of transition (Kemp and Rotmans, 2004, Kemp et al., 2007). It is noted that realizing transition requires the development and execution of a wide variety of transition experiments to facilitate learning and knowledge creation on transition management processes (cf. Brown et al., 2003, Loorbach, 2007). The transition literature identifies the importance of strategic niche management, which argues that experiments and innovations are best developed in specifically 'protected arenas', detached from the usual short-term performance pressures that control day to day operations to increase the possibilities for learning (cf. Geels, 2005, Loorbach, 2007). 

However, despite the emphasis that transition management literature has placed on the value of experimentation and second-order learning, few practical guidelines have been developed for managers and innovators who are involved in the day-to-day management of experiments to also contribute towards more sustainability in the form of second-order learning goals (cf. Loeber, 2003, Rotmans, 2003, Kemp and van den Bosch, 2006, Grin and van Staveren, 2007). 7 Project managers and project leaders have emphasized that the current guidelines and literature available with regard to the practical management issues tend to be too generic, too conceptual; too abstract and too detached from day-to-day reality for use by practitioners (Raven et al., 2007, Caniëls and Romijn, 2006). 

The consequence of this lack of knowledge might be that many experiments fail to contribute to transitions. Transition management scholars have indeed acknowledged that many experiments and niches never break through and make any progress (Elzen et al., 2004) and it can thus be questioned whether all these experiments contribute much to our knowledge of experimentation and transition management as second-order learning in experiments rarely occurs (cf. Brown et al., 2003). 

The insights have also remained fragmented in literature (Caniëls and Romijn, 2006). Thus little attention is paid to how experiments should be set up or managed to contribute to higher-order transition goals (van de Kerkhof and Wieczorek, 2005). "What can a practitioner do to guide and modulate transitions towards sustainability given the complex, multi-level and multi-actor world he or she is operating in?" (Mourik and Raven, 2006, p. 9 ). These second-order learning goals are of vital importance for the conduct of experiments and are often ill-addressed in practice. 

The aim of this paper is to illuminate key requirements for multi-actor governance processes by emphasizing management principles for experiments to innovators and project managers. Secondly, this paper highlights a number of dilemmas with regard to multi-actor governance processes that managers in charge of experimental projects face. A number of seemingly contrasting/paradoxical requirements emerge from the practical recommendations of various bodies of literature on multi-actor governance, outlining the ambiguous and complex nature of what transition management seeks to achieve. Identifying these paradoxes may contribute to further insight in multi-actor governance processes and may benefit practitioners who are engaged in experiments that contribute to developing a more sustainable mobile system as it allows them to more consciously reflect on and include issues of second-order learning about transition management in the day-to-day management and decision making during the experiment.
section_name: Governing for Change in a Multi-actor Mobile System

section_content: The previous sections have outlined that there is need for a set of operational recommendations for the management and conduct of experiments that might increase their value and contribution to multi-actor governance and 'managed' transitions in the mobile system (i. e. to stimulate second-order learning in the mobile system). 

Existing "steering notions" or managerial principles in literature regarding multi-actor governance have been identified and combined with more practical guidelines for (transition) experiments (Grin and van Staveren, 2007, Loeber, 2003, Kemp and van den Bosch, 2006, Raven et al., 2007, van der Laak et al., 2007, Mourik and Raven, 2006, Raven et al., 2010). The notions have been clustered in three key managerial aspects of experiments that are of principle concern for experimental project managers. These notions/ managerial principles are labeled 'understanding', 'safeguarding' and 'utilizing' and address different important aspects that need to be managed during experiments. 8 Depending on the progress that experiments make, emphasis shifts from understanding towards safeguarding and finally utilizing. However, it should also be emphasized that project managers should monitor and work on all these notions at any time.
section_name: Experimenting in Multi-actor Governance Systems

section_content: A first focal point for project managers is arguably the need to focus on generating a broad and in-depth 'understanding' of what the experiment is all about. This has been dubbed "the voicing and shaping of expectations and visions" (Raven et al., 2010, p. 64). First of all, it should be important to note to project managers that even though they tightly resemble each other, experiments within a transition context are in fact very different from regular innovation processes. In experiments, more than in regular projects, "function, rather than product needs need to be specified" (Van der Zwan and Bhamra, 2003, p. 900). 

Project managers are furthermore encouraged to invest in the development of processes that stimulates participants in the experiment to generate a deeper, shared understanding of the experiment as well as its role within a broader context. The key aspect in this phase is the formation of an environment in which "high quality learning experiences" with regard to (sustainability) issues can take place (Caniëls and Romijn, 2006, p. 10). Previous experience has shown that project management should be aware of the various types of learning that take place in the projects. Managers and actors involved in experiments could facilitate this process and thereby increase the chance that new knowledge is created, and shared that will contribute to transitions. 

Project management should develop processes that invite project participants to exchange and share as much knowledge as possible and possibly contribute to the project. This knowledge exchange should not just be about the experiment itself, but also about -for example -working definitions of sustainability that are going to be used during the project. The voicing of expectations by project participants and the management of these expectations is also important for project managers (cf. Raven et al., 2008). In order to retain this largely tacit knowledge exchange, project management should document as much of the discussions during the project as possible. To increase the possibilities for knowledge development and learning, project management should endeavor to invite a wide variety of stakeholders to take part in these experiments and to seek a wide range of opinions and visions with regard to the experiments (Raven et al., 2007). Highly concrete project design recommendations include:  Project goals. Organize multiple rounds of discussion about problem formulation, the (definitions of) the project goals and their relation to wider sustainability and transition goals analogous to steps identified in multi-actor policy analysis. Goals, objectives, strategies, tools, interpretations of data and other key dimensions require negotiations between participants (Brown et al., 2003, p. 295). The aim of the recommendation is to stimulate project participants to share knowledge and exchange viewpoints and reach a shared body of knowledge with regard to vital issues within the project. These include shared definitions of the goals of the experiment, a shared identification of relevant criteria, a shared definition of sustainability and a shared vision of transition and the contribution of the experiment to this end (cf. Mourik and Raven, 2006, pp. 17-20). "Incongruent visions and interests, low commitments and risk-taking by key actors, and complex sets of variables to consider are likely to emerge" (Brown et al., 2003, p. 312). The discussions may highlight differences of opinion, but also create new innovative ideas or highlight important issues that were not incorporated in the original project description and contribute to the shaping of robust expectations about the technology, and the wider context in which the technology is embedded (e. g. sectoral, but also societal embeddedness) (cf. Truffer et al., 2008). Furthermore, in these discussions, knowledge about the 'position' of the experiments within the wider transition framework should be explored as well (cf. Raven et al., 2010)  Actor analyses and stakeholder consultation. Project managers should analyze who the relevant actors are with regard to the particular problem they are faced with and they should consult and incorporate as many different stakeholders as possible in the design and execution of the experiment (cf. de Bruijn and ten Heuvelhof, 2008, pp. 36-42). The general rule of thumb that can be used is the more complex the problems and the solutions under consideration, the more diverse the number of stakeholders and experts that should be involved in the experiment. Furthermore, project managers should stimulate different categories of 'outsiders' to become engaged in experiments to increase the chance of more radical innovations to take shape (cf. van de Poel, 2003). 

 PR. To reach and 'attract' stakeholders, project managers should invest in communication and P. R. to inform about the experiment and invite stakeholders to participate (e. g. create a flyer and a website to inform interested stakeholders about the experiment). Examples of stakeholders include environmental and/or local interest groups, (groups) of end-users, (political) decision makers, scientific institutes, as well as consultative and deliberative bodies/intermediaries (cf. Caniëls and Romijn, 2006, pp. 11/12). The information should be short and to the point and should at least outline the project goals and indicate how the experiment is embedded in broader initiatives and how the experiment contributes to sustainability.
section_name: 'Understanding' the experiment

section_content: With regard to knowledge management, project leaders are encouraged to design a number of features in the experiment that make sure that as much (tacit) knowledge as possible is retained in the course of the experiment. For example, detailed records and notes of all deliberations of the project team should be retained as well as prototypes, etc. (cf. de Bruijn and de Neree tot Babberich, 2000). Other tools include 'narratives', and the (re)construction of 'defining moments', 'context' and 'actor' tables (Raven et al., 2009).
section_name: 

section_content: Log book. In addition to the previous point, project leaders are advised to maintain a separate experimental design log book that records the most important decisions and considerations/arguments that have been taken during the experiment. Project leaders should take the time to clearly outline the design process during the experiment and describe the issues and dilemmas the project team encountered and clearly indicate how these issues were solved. Together with the primary material, a secondary analysis of the process is of vital importance to reconstruct the design process.
section_name: 

section_content: A second focal point for project managers concerns the protection of the knowledge that is created during the experiment. Protection mechanisms should similarly focus on different 'products' that the experiment provides. Project management should ensure that essential knowledge is retained within the experiment and that all other knowledge is made available to the outside world. Many of the considerations that are central in Strategic Niche Management (SNM) emerge in highly concrete design recommendations (e. g. Caniëls and Romijn, 2006, Raven et al., 2007, van der Laak et al., 2007, Kemp et al., 1998, Mourik and Raven, 2006). The experiment should be isolated from the day-to-day routines and performance pressures and management should be focused on the creation of procedures and routines that ensure that the knowledge that is created within experiments is protected. The practical recommendations to managers are:  Learning objectives. Develop a negotiation process in which project members reach unanimous agreement on the various learning objectives of the experiment. These objectives could be considered as project goals (cf. Dunn, 1994; Miser and Quade, 1985). These goals should be achievable yet challenging (Caniëls and Romijn, 2006, p. 10). Project management should distinguish multiple learning goals. The experiment should be able to answer at least the following questions: "What is learned in the experiment?", "What is learned from the experiment?", "What is the scientific relevance of the experiment?" and "What is learned from the experiment with regard to reaching more sustainable innovation?" These learning goals should be explicitly recorded and translated into concrete and unambiguous (i. e. 'objectively' measurable) goals (i. e. SMART; Specific, Measurable, Attributable, Realistic and demarcated in Time). However, project managers should be aware of the perverse effects of performance measurement and show constraint in its application (cf. de Bruijn, 2002).
section_name: Safeguarding the experiment

section_content: As a refinement of the learning objectives, project management should invest in the development of so-called causal analysis and the formulation of 'learning processes' that will be matched with concrete actions towards intermediate goals. This setup will ensure that project members remain motivated and will facilitate the attainment of the end-goals. Transition management emphasizes to keep experiments as simple as possible (Caniëls and Romijn, 2006, p. 11).
section_name: 

section_content: Project integrity. Project management should develop a process in which the essence of the experiment is discussed among the project members to determine how the 'integrity' of the project can be protected (Mourik and Raven, 2006, p. 17). Project members should reach unanimous agreement on the minimal goals/output of the project and the procedures that ensure that these goals are met (cf. de Bruijn and ten Heuvelhof, 2008). Apart from setting unambiguous goals, core concepts and knowledge within the projects should be protected to avoid free-riding behavior among project members (cf. de Bruijn et al., 2002). For example, project management can think of procedures such as concept-certification, which ensures that knowledge that has been created within the experiment (i. e. a specific concept) can only be used within this context.
section_name: 

section_content: Stakeholder constellation. Project management should design processes to ensure an effective constellation of stakeholders during the experiment consisting of a sufficient degree of reciprocity and of a multidisciplinary nature (cf. Caniëls and Romijn, 2006, p. 12 ) (See also 2. 1: Actor analyses and stakeholder consultation). Questions that should be asked to analyze the constellation include: "what are the desired characteristics of the innovation?", "whose expertise or approval is needed for the development and diffusion?" and "what are the interests of the key actors and how can their commitment be gained?" (Kivisaari et al., 2004, EJTIR 10(3) p. 227). Project leaders and innovators should be aware of power distribution issues in experiments, keeping in mind that dominant actors could potentially manipulate experiments. Gains of winners and losers should be managed in a way that all members should be rewarded reciprocically for their efforts. Furthermore, project managers should ensure the active involvement of all groups from an early stage in the experiment, including local project definitions, overall program goals, budgets and schedules. At the same time the desired stakeholder constellation is a dynamic and fluid concept and can depend on the specific conditions in which the experiment is conducted, meaning that entry and exit should be possible (cf. de Bruijn and ten Heuvelhof, 2008).
section_name: 

section_content: External evaluators. As part of any process, evaluation is an important, but often overlooked, aspect (cf. Walker, 2000). Project management should seek out independent external evaluators to assess the progress of the experiment (Caniëls and Romijn, 2006, p. 11).
section_name: 

section_content: Process rules. Design elaborate rules and procedures that provide clear guidelines how and under which specific conditions key decisions can be made during the experiment (Mourik and Raven, 2006). Experiments should be analytically divided in a number of decision making rounds in which different sets of stakeholders cooperate to contribute to the experiment. Each of these 'rounds' should have its own procedures and rules. Project management should pay specific attention to process rules that enable project members to (fundamentally) alter the design or execution of the experiment (cf. de Bruijn and de Nerée tot Babberich, 2000, pp. 91-104). Finally, for each decision-making round, clear procedures should be developed with regard to the exit-options of project participants. These procedures should make clear in what way and under which conditions project members can decide to stop their participation in the experiment. Unambiguous and 'objective' criteria should be formulated to determine how failure can be recognized and proper attention should be paid to maintain a balanced pay-off between 'winners' and 'losers'.
section_name: 

section_content: Both previous focal points were more or less focused on the internal cohesion and organization of the experiment. A final focal point for project managers during experiments concerns possibilities to utilize the knowledge that is gained during the experiment. This means a focus on the environment. Project leaders are encouraged to look for ways to hook up with developments outside the 'protected environment' of the experiment. During the experiment project management should ensure that sufficient time and funds remain available for 'utilization' purposes. As the experiment progresses, pressures will increase to reduce the time and funds that are available for these processes in support of more concrete project goals that can be readily achieved (e. g. (Teisman and Edelenbos, 2004) ). However, especially with regard to second-order learning from experiments, 'utilization' may be considered of key importance. Experiments need to be linked up to other experiments, new technologies and new stakeholders that may even include competitors. Several practical recommendations that may be deduced from the notion are aimed at the various ways in which experiments may be connected:  To actors. Project management should draw up a list of (potential) demands of all (potential) users that may be interested in the end-result of the experiment (cf. Dunn, 1994; Miser and Quade, 1985). This list may be part of the stakeholder analysis (See also 2. 1: Actor analyses and stakeholder consultation). This list should include demands from direct users, but should also take into account the demands of more indirect 'users' (e. g. political decision makers, government agencies or end users) (cf. Kivisaari et al., 2004).
section_name: 'Utilization' of the experiment

section_content: To new or ongoing developments. Project management and project members should be continuously aware of potential 'interfaces' with other (innovation) experiments or developments (innovations, policies, trends, etc. ) environment' in which the experiment takes place. Whenever possible, project management should sound out possibilities to link the experiment with other activities and seek the active support and cooperation of stakeholders that are not involved in the project and achieve serendipity (cf. de Bruijn and ten Heuvelhof, 2008).
section_name: 

section_content: To the larger infrastructure system. An often-mentioned problem in innovation management is the so-called 'Valley of death'. After innovative breakthroughs have been achieved, many innovations fail because they simply are too detached from their immediate environment (other technologies, preferences of users, manufacturers, etc. ). Innovations that seek to achieve transitions could meet a similar fate. Project management should ensure that apart from the project goals, sufficient attention is paid to whether the experiment is embedded in a broader systemic design. Experiments should pay attention to the interfaces between the innovation and the institutional and technical environment. Project management can stimulate this focus through feasibility studies and by stressing the real-life application of the experiment (e. g. including a 'real-life' experiment with end-users). These tests could lead to adjustments in the list of demands for the experiment and could thus lead to changes in the experimental design, or they could result in the necessity for changes in the environment.
section_name: 

section_content: In addition to the abovementioned management issues, a number of practical recommendations can be provided that are of importance for the entire duration of the experiment:  Interaction. Project management should always be mindful of opportunities to increase the level of interaction between project members within the experiment as well as opportunities to interact with the wider environment. Although project managers should be careful to prevent interaction from becoming an end-goal of the experiment (and thereby wasting costly resources), interaction among project participants (and their environment) generally increases the possibilities for serendipity and creativity that may contribute to sustainable innovations.
section_name: Cross cutting issues

section_content: Iteration during the experiment. Project management should ideally be committed to continuously monitor and address all the abovementioned management issues, through socalled 'reflexive experimentation' (Caniëls and Romijn, 2006, p. 11). In practice, the design should be used at least three times during the length of the experiment. However in practice "as the experiment proceeds, the optimal network composition, required tasks and needed interactions" that determine the relations between the project participants evolves and must be matched by the dynamical in the rules and procedures that project management employs to manage the experiment (Caniëls and Romijn, 2006, p. 12 ).
section_name: 

section_content: Flexibility. Project management should consider the experiment as a cyclic process that alternatively facilitates variety and selection. Project goals can and should be adjusted in scale and scope along with this process. This means that project members should be allowed to challenge (long-held) assumptions or specific issues within projects. Project management should design procedures to determine when and based on what considerations the experiment can be changed (See also 2. 2: Process rules).
section_name: 

section_content: When the abovementioned list of practical recommendations derived from state-of-the-art literature was constructed, it was found that much of the multi-actor governance literature emphasized recommendations that focused on issues such as broadening support for experiments and second-order learning. However these prove anything but practical for those that have to manage these projects. For example, strategic niche management emphasized the need for coherent sets of experiments: parallel experimentation, the constant reformulation of product designs following evaluation and learning from failed experiments as conditions for niche management (Mourik and Raven, 2006, p. 13). Similarly, Brown et al. (2003, p. 295 ) more or less take the "appropriate design of an experiment" for granted, focusing on the possibilities to learn from experiments. 

Both authors claim that this emphasis can be readily made as sufficient project management tools exist that enable practitioners to adequately set up experiments (Mourik and Raven, 2006). However, from the list of practical managerial recommendations, it could be argued that many project parameters are indeed affected by the demands that multi-governance literature place upon project management. In fact, inherent tensions may exist between the experimental goals at the operational level (i. e. first order learning) and experimental goals with regard to transition management perspective (i. e. second order learning). Especially as experiments in the Dutch TRANSUMO research program find themselves being subjected to stringent, project-based performance criteria (Avelino, 2009, pp. 372-373). We would argue that this tension between experimental project principles that guide day-to-day of project managers and the multi-actor governance requirements create dilemmas that are not so easily overcome as might be expected or assumed in literature and, more importantly, that choices that project managers or innovator make in order to reach first-order project experimental goals affect the outcomes. Failure to recognize these aspects may affect the way in which these experiments contribute to more sustainable transitions. 

As a consequence of these issues, a number of dilemmas emerge from the compiled list that may require more detailed attention from experimental project managers regarding multi-actor governance processes. Based on state-of-the-art knowledge that has been formulated in practical recommendations, the following dilemmas emerge for project managers engaged in (transition) experiments 9 :  For example, on the one hand, experiments should focus on second-order learning (cf. Brown and Vergragt, 2008, Brown et al., 2003), or 'reflexive' experimentation (Hendriks and Grin, 2007), but on the other hand their goals should also be formulated very concrete and measurable if any progress is to take place (cf. Mourik and Raven, 2006, Raven et al., 2007, Caniëls and Romijn, 2006). This requires a trade-off between project integrity and project flexibility. The desired predefined, unanimous, objectified agreement of stakeholders will most likely have a negative effect on creativity, serendipity, challenging assumptions and variety, which blossoms when second-order learning goals are left undefined and emerge from the process.
section_name: Exploring Dilemmas in Multi-actor Governance Processes

section_content: Secondly, experiments should involve all necessary stakeholders to ensure that innovations actually become successful (cf. Raven et al., 2007, Kivisaari et al., 2004). However, on the other hand the experiments should be risky, daring and based on unconventional thoughts and therefore more or less shielded from the existing regimes of stakeholders (cf. Caniëls and Romijn, 2008, Ieromonachou et al., 2004, Brown et al., 2003). How can experiments be governed in safe, yet sufficiently hostile environments however remains a challenge (Elzen et al., 2004, p. 293). And more importantly, what are the characteristics of these environments? How to prevent free-riding and the desire for transportation industry (largescale) diffusion to other parts of the transport system. For example, a procedure like conceptcertification may have a negative effect on the possibilities to disseminate the results. This dilemma gains added complexity as Brown et al. (2003, p. 312) experiments gain from a sense of urgency in the form of failure, surprises, risk taking, public and media attention, and adverse events.  Third, inherent tensions may exist between the experimental goals at the operational level (i. e. first order learning) and experimental goals with regard to transition management perspective (i. e. second order learning). For example project time (first order goal) versus project quality (second order learning): the design recommendations claim that project managers should consult and incorporate as many different stakeholders as possible in the design and execution of the experiments as possible (de Bruijn and ten Heuvelhof, 2008, pp. 36-42), but also reach unanimous agreement on the various learning objectives of the experiment (Dunn, 1994; Miser and Quade, 1985) and the desire to keep experiments as simple as possible (Caniëls and Romijn, 2006, p. 11 ) is a continuous trade-off against time and resources that needs to be consciously managed.
section_name: 

section_content: What are project managers to do when their innovative experiment is about to fail and the only way to rescue it is to move outside the originally agreed upon design scope? How should project management navigate between the attitude necessary for learning -openness to the idea of reassessing the problem definition driving the experiment, its objectives, approaches and tools used in its execution (Brown et al., 2003, p. 295 ) -and performance goals and creeping interests that grow as experiments gradually reach a more mature level? Do you go for a slightly converted version of the previous experiment that promises to deliver something within a reasonable timeframe or opt for a new project and breadth of support? The larger the number of actors, the more difficult it is to create a (renewed) shared vision. Reaching shared goals, objectives, strategies, tools, interpretations of data and other key dimensions requires long negotiations and will inevitably cause long delay. What guidelines can be provided when dealing with conflicts between local experiment and regime management? Furthermore whose task is it to determine whether an innovation does have potential and could contribute to an overall transition? Who should decide on this and how is this to be decided?
section_name: 

section_content: The practical dilemmas and trade-offs that were raised are but a few of those possible that managers in charge of experiments in the face on a daily basis. These dilemmas demand continuous attention and conscious decision making processes from those that are engaged in the management of these experiments as well as those that monitor the goals of transition management experiments. However, this requires acknowledgement from the transportation field and efforts from managers in charge of experiments that seek to contribute to transitions. The impact of these necessary requirements that can be associated with the continuous balancing and trading off of various project goals has only been implied but not been seriously addressed until recently, and has been found to apply to the transport system (Avelino, 2009). 

As was experienced in a study of the TRANSUMO research program, transition management can be a powerful concept, but practical barriers can easily influence the attainment of the secondorder goals of transition management (Voß et al., 2009). Many of the dilemmas outlined in this paper were identified in the Dutch TRANSUMO research program (Avelino, 2009, p. 383). Although the goals of the research program are ambitious in nature, daily reality in the form of project management sees a definitive emphasis on first-order goals, rather than second order learning goals. It could be argued that mapping and identifying the dilemmas that were highlighted here could provide scholars more insight in multi-actor governance issues in transition management. Most literature that deals with methods and practical guidelines only displays 'roadmaps' or provides elements that should be included in the experimental setups but fail to point out the problematic choices that managers face. These issues have so far been largely neglected in practice and managers in charge of transport experiments in the Netherlands could find additional information on dealing with potential dilemmas that need their continuous attention helpful. This would allow them to view transition management and the second-order goals in a more 'instrumental manner' (Avelino, 2009, p. 383), which in turn might heighten the chance of experiments conducted in the transportation industry contributing to a more sustainable future. 

For those engaged in setting up sustainable transport policies the knowledge provided is perhaps not new, but a powerful reminder that existing institutional arrangements and the way in which policies are set up are in themselves powerful constraints that might (un)consciously restrain or even defeat the goals these policies seek. The 'mundane' administrative and bureaucratic tools of accountability, representation that are present in large-scale sustainability programs can provide important hurdles for professionals engaged in the management of experiments. Furthermore, more efforts could and should be undertaken to support these managers and to train and educate these managers how to deal with the dilemmas and tradeoffs that were identified. This means going beyond prescriptive 'to-do' lists but teaching managers how they can (and/or should) deal with these dilemmas and tradeoffs is of paramount importance if policy makers want to increase the chance that transitions are successfully 'managed'.
section_name: Conclusions