Agile risk management for multi‐cloud software development

IET SoftwareVolume 13, Issue 3 p. 172-181 ArticleOpen Access Agile risk management for multi-cloud software development Victor Muntés-Mulero, Corresponding Author Victor Muntés-Mulero Victor.Muntes@ca.com Strategic Research CA Technologies, Strategic Research, Plaça de la Pau s/n, Edifici 2 Planta 4, Cornellà de Llobregat, 08940 SpainSearch for more papers by this authorOscar Ripolles, Oscar Ripolles Strategic Research CA Technologies, Strategic Research, Plaça de la Pau s/n, Edifici 2 Planta 4, Cornellà de Llobregat, 08940 SpainSearch for more papers by this authorSmrati Gupta, Smrati Gupta Strategic Research CA Technologies, AgileOps, 3965 Freedom Circle, Santa Clara, CA, 95054 USASearch for more papers by this authorJacek Dominiak, Jacek Dominiak AgileOps CA Technologies, Strategic Research, ul. Woloska 5, 02-675, Warszaw, PolandSearch for more papers by this authorEric Willeke, Eric Willeke Strategic Research CA Technologies, AgileOps, 3965 Freedom Circle, Santa Clara, CA, 95054 USASearch for more papers by this authorPeter Matthews, Peter Matthews Strategic Research CA Technologies, Strategic Research, Riding Court Road Datchet, Berkshire, GB, SL39LL UKSearch for more papers by this authorBalázs Somosköi, Balázs Somosköi Lufthansa Systems GmbH & Co. KG, Siemensdamm 62, D-13627 Berlin, GermanySearch for more papers by this author Victor Muntés-Mulero, Corresponding Author Victor Muntés-Mulero Victor.Muntes@ca.com Strategic Research CA Technologies, Strategic Research, Plaça de la Pau s/n, Edifici 2 Planta 4, Cornellà de Llobregat, 08940 SpainSearch for more papers by this authorOscar Ripolles, Oscar Ripolles Strategic Research CA Technologies, Strategic Research, Plaça de la Pau s/n, Edifici 2 Planta 4, Cornellà de Llobregat, 08940 SpainSearch for more papers by this authorSmrati Gupta, Smrati Gupta Strategic Research CA Technologies, AgileOps, 3965 Freedom Circle, Santa Clara, CA, 95054 USASearch for more papers by this authorJacek Dominiak, Jacek Dominiak AgileOps CA Technologies, Strategic Research, ul. Woloska 5, 02-675, Warszaw, PolandSearch for more papers by this authorEric Willeke, Eric Willeke Strategic Research CA Technologies, AgileOps, 3965 Freedom Circle, Santa Clara, CA, 95054 USASearch for more papers by this authorPeter Matthews, Peter Matthews Strategic Research CA Technologies, Strategic Research, Riding Court Road Datchet, Berkshire, GB, SL39LL UKSearch for more papers by this authorBalázs Somosköi, Balázs Somosköi Lufthansa Systems GmbH & Co. KG, Siemensdamm 62, D-13627 Berlin, GermanySearch for more papers by this author First published: 01 June 2019 https://doi.org/10.1049/iet-sen.2018.5295Citations: 8AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Industry in all sectors is experiencing a profound digital transformation that puts software at the core of their businesses. To react to continuously changing user requirements and dynamic markets, companies need to build robust workflows that allow them to increase their agility in order to remain competitive. This increasingly rapid transformation, especially in domains such as Internet of things or cloud computing, poses significant challenges to guarantee high-quality software, since dynamism and agile short-term planning reduce the ability to detect and manage risks. In this study, the authors describe the main challenges related to managing risk in agile software development, building on the experience of more than 20 agile coaches operating continuously for 15 years with hundreds of teams in industries in all sectors. They also propose a framework to manage risks that consider those challenges and supports collaboration, agility, and continuous development. An implementation of that framework is then described in a tool that handles risks and mitigation actions associated with the development of multi-cloud applications. The methodology and the tool have been validated by a team of evaluators that were asked to consider its use in developing an urban smart mobility service and an airline flight scheduling system. 1 Introduction Organisations in all industries recognise that they are continuing their evolution into technology and data companies [1] and that their business models are being partially or fully transformed by software. One of the main drivers of this transformation has been the adoption of paradigms such as Internet of things (IoT) or cloud computing. Cloud computing allows companies to create new business models that scale depending on the demand, to migrate their operations to the cloud, and to create new service models such as software as a service (SaaS) [2]. Large technology companies acknowledge the importance of SaaS and cloud computing [https://www.cio.com.au/article/619978/ca-technologies-realigns-saas-y-ux-driven-try-before-you-buy-market/.], and users also express their preference for a subscription-based model since customers become the focus of the companies instead of the product or the transaction [https://www.forbes.com/sites/kimberlywhitler/2016/01/17/a-new-business-trend-shifting-from-a-service-model-to-a-subscription-based-model/#1517bc704a5f.]. Gartner estimates that by 2022 the main barrier for IoT adoption will still be security [3]. North Bridge indicated in their latest Future of Cloud Computing Survey [4] that security was still the top concern and inhibitor of cloud adoption. Furthermore, Article 25 in the recent general data protection regulation (GDPR) [5] discusses data protection by design and by default, underlining that considering privacy from the beginning is essential to address privacy successfully. Additionally, there is an increased need to create trustworthy systems. Just as an example, Yan et al. [6] published a survey where they describe the different dimensions of trust for the IoT systems. Their work concludes that risk management is an essential piece to guarantee trustworthiness. Privacy or security-by-design can only be achieved if risk management is performed from the beginning of the software development cycle. Continuous and agile risk-management processes are of great importance in IoT and cloud computing since these environments have a complex and distributed network of services that increase the attack surface. In the context of multi-cloud applications, whose components are deployed over different infrastructures provided by different cloud service providers (CSPs), risk related to accountability, assurance, agility, or even financial aspects become even more challenging. Risk analysis can also guide the selection of CSPs, and several authors have proposed methodologies to consider risks in addition to the quality of service or cost [7], especially in the context of multi-cloud applications [8]. Therefore, any component, whether running on premises, remotely hosted by a CSP or offered as a service (by a CSP, a device in an IoT etc.), is subject to risk analysis considerations. Poor risk management allied to a reactive strategy usually forces companies to continuously re-factor their application architectures to improve overall software quality and security, incurring in technical debt and high re-implementation costs [9]. The demands of software driven businesses and the need for fast innovation are forcing organisations to replace traditional software development methods such as Waterfall for alternatives that must be agile and support continuous development and delivery models in an aggressively changing market. The principles that drive DevOps and Agile methodologies, with a focus on transparency and collaboration between teams, can help minimise the risks in the application and the inconsistencies in the way risks are managed. Nevertheless, the Agile Manifesto (agilemanifesto.org) suggests that Agile teams focus on delivering the simplest code that supports the needs of their customers. In their attempt to adhere to this bottom-up principle, Agile teams take an incremental and iterative approach that focuses on delivering individual software capabilities, usually derived from functional requirements (FRs), diminishing the relevance of system-wide quality attributes and other aspects related to non-FRs (NFRs), which, in general, are not well managed in agile methodologies [10, 11]. This negatively affects code quality and the quality of the overall software architecture. As a consequence, elicitation and management of risks connected to NFRs in Agile become increasingly important to mitigate the negative effect of relegating NFRs to second-class citizens. To our knowledge, only some initial work has been done to conceptualise agile risk management techniques, and effective control of NFR-related risks in agile methodologies is still to be discussed. This paper aims at describing challenges that deserve more attention and then proposing and validating a solution. This paper is organised as follows. Section 2 analyses how NFRs are currently managed, and the main obstacles related specifically to risk management. This analysis will help us describe in Section 3 unsolved problems that pose challenges in agile risk management. As a solution to these challenges, in Section 4 we propose a framework to support a collaborative and agile risk analysis in environments that require continuous software delivery, in order to detect risks and generate sets of mitigation actions. Section 5 describes how this framework and associated tool set have been implemented for the development of multi-cloud applications. This tool set was validated by a team of evaluators whose feedback is analysed in Section 6. Finally, Section 7 draws some conclusions and outlines future lines of work. 2 Related work This section discusses relevant work related to risk-management methods for software development, NFRs management in Agile methodologies, continuous software development, and risk management in a distributed agile development (DAD). 2.1 Traditional risk management for software development Implementing risk-management processes is essential, especially for complex, high-risk projects [12]. The business domain can also affect the need for risk analysis. In this sense, domains that are less volatile such as supply chain software will not change so quickly, while other businesses will be continuously looking at their customers and reacting to changes and modifying the requirements accordingly. In general, previous work focuses on the classical schedule, budget, and scope risk analysis. When using risk management techniques for software projects managed through waterfall methodologies, risks are thoroughly analysed at the beginning of the project but when requirements change during the project lifetime risk analysis may become obsolete [13]. Waterfall models assume requirements to be clearly defined in advance in the design stage and, in general, to remain fairly immutable, which tends to be unrealistic for projects in a rapidly changing market. Although there are change management mechanisms in waterfall processes that permit for requirements to change, risk management is seldom involved, and the overall risk analysis is not updated. There are a number of risk assessment methodologies that include quantitative metrics such as the probability of occurrence or the effort to implement control measures or even its cost. However, others consider qualitative metrics, e.g. an appraisal of the project staff's motivation. There are many quantitative risk methodologies and tools such as RiskWatch [RiskWatch: https://www.riskwatch.com, accessed 09 May 2017] or information security risk analysis method (ISRAM) [14] and there are many qualitative risk methodologies such as operationally critical threat, asset, and vulnerability evaluation (OCTAVE) [15], Coras [16], or one of the most commonly known in the hosted software arena spoofing identity, tampering, repudiation, information disclosure, denial of service, and elevation of privilege (STRIDE) [17]. Open web application security project (OWASP) [18] is an open standard trying to define the risk aware software development. It aims at making software security visible so that organisations are able to make informed decisions. DREAD [19] is a successor of STRIDE and provides another approach specialised in the multi-stack type of applications. Pasha et al. [20] performed recently a detailed and thorough analysis of different risk management approaches, both for small- and large-scale software systems. In addition to the state of the art, the authors also proposed a methodology for risk mitigation that was very complete but lacks agility. 2.2 Agile risk management for software development In general, it is accepted that managing risks and other NFRs is ill-defined in agile approaches [21]. Different lightweight NFRs methodologies for agile processes have been presented such as [22] or [23], where authors combined both FRs and NFRs in one framework and computed a risk-based requirements implementation sequence. Recently, Medeiros et al. [11] propose an approach to specify requirements based on design practises targeted to the developer. It is worth highlighting the work by Moran [24, 25], where the author explicitly tackles issues related to risk management for agile software development. Moran proposes a risk-modified Kanban board and a user story map. To our knowledge, this is the only research work that could be framed in the research challenges described in this paper. Some products such as codeBeamer [Risk Management in codeBeamer: https://intland.com/video/test-and-quality-assurance-management-videos/risk-management-in-codebeamer/.], which provide scaled agile capabilities, also include a mechanism to assign risks to both functional and NFRs. However, their methods to define and manage risks are very similar to traditional risk-management methods. Most of the traditional risk-management methodologies described above focus on the assessment of risks at a singular stage in time. However, these techniques do not address one of the most prominent challenges in today's adaptive, pivot-oriented world: the challenge of the continuously evolving risk profile. Managing continuous changes in software development has been studied from different perspectives. Practises such as 'release early, release often' have been promoted and adopted in open source software development [26] and they prove to benefit software quality and consistency [27]. While continuous software integration [28] may be the most well-known practise in industry related to continuous software development, the increasing focus on security and privacy has also generated work on continuous security and regulatory compliance. Fitzgerald and Stol [29] publish a roadmap and agenda for continuous software engineering. Ameller et al. [30] propose to re-plan the current release every time an activity triggers the need for an updated release plan. Initially, agile methods were devised to fit small projects with co-located developers in contexts, where safety and security were not critical [31]. However, the use of these methods has been extended to large projects with distributed development teams [32] and safety-critical systems [33]. In this last paper, Fitzgerald et al. discuss regulated scrum, an adaption of Scrum to support compliance in regulated environments such as medical devices, railway, or aviation. They propose adding new ceremonies, artefacts and roles to allow compliance to be assessed at the end of each sprint. DAD approach is increasingly adopted by more and more software companies. The idea of DAD is to combine the quality and speed benefits of agile with the cost benefits of distributed software development (DSD). This combination generates significant risks, considering the contradicting nature of agile and DSD [34]. Shrivastava and Rathod [35] present a risk-management framework for DAD, studying risks related to the software development life cycle, project management, group awareness etc. However, this work is focused on analysing risk factors that represent a threat to the successful completion of a software development project, rather than risk threatening NFRs. In [36], Aslam et al. performed a systematic literature review on risks and control mechanisms for DSD, which could be extrapolated for a system to support DAD. Finally, it is worth noting that, in general, security tends to be considered lower in priority, sometimes unintentionally, when considered as one of the NFRs of an application. Continuous security [37] aims at putting security as a high-priority concept through all phases of the software development process. Authors detect nine building blocks for continuous application software security including, for example, employee training and awareness, creating a security software group or giving NFRs the same importance as FRs. As an example, they suggest creating user stories to capture NFRs. However, this does not provide a complete solution to challenges in Section 3 since, for example, collaboration is not expressly addressed. 3 Challenges in risk management for agile software development In this section, we present the main challenges for agile risk management related to software development. Before describing the challenges, it is worth discussing how risks are managed in Agile. The first wave of Agile adoption in many companies involved structuring software development through Scrum teams. When it comes to application risk management, most companies cannot afford to have risk-management experts in each team. A frequently used solution is to choose to have staff responsible for detecting the most important risks in a centralised way. The use of framework methodologies to scale Agile such as scaled agile framework (SAFe) [Scaled Agile Framework: http://www.scaledagileframework.com.], marginally improves this situation. In this case, organisations structure software development by creating teams of teams that plan and synchronise their work through programme increments (PIs). When planning a new PI (typically every 10 weeks), all the actors participating in the software development process attend the meeting. PI planning usually starts with a vision and a prioritised list of new features of the product, which are then decomposed into user stories. This is defined by product owners and created by the different Agile teams, which are in charge of drafting plans but also of analysing risks and impediments. In fact, user stories replace traditional FRs, and describe the intended system behaviour. Consequently, risk analysis is done related to expected features and to these user stories. Owing to this, NFRs, which are usually not represented through these user stories, tend to be unintentionally diminished in terms of importance [37] and commonly ignored during the risk analysis. During PI planning risks are identified by each team, and then aggregated into a programme risk sheet that is reviewed in a plenary session. The group then discusses and categorises programme risks and impediments. After PI planning, once risks have been classified, and actions and owners are established, all teams are assumed to be aware of these potential risks and impediments detected during the meeting and they are expected to act accordingly throughout the sprints in that PI. Additionally, the programme leadership team will typically track those risks that are not resolved to ensure the right coordination occurs. Unfortunately, the mapping between risks and user stories do not usually happen and, since PIs are guided by customer requirements, the risks detected remain as a relegated part of the development process. Risk management in an agile environment must be integrated into the process, present in PI planning, in sprint planning, and in any other ceremony. It must be light but continuous activity. The challenges included in this section are derived from the analysis of the related work and they are also based on CA Technologies Rally active coaching. Our conclusions build on top of more than 20 agile coaches operating continuously for 15 years with the ever-increasing scale of engagement. Authors of this paper accumulated more than 10 years of experience coaching companies to help them adopt Agile, between 2008 and 2017. We have directly interacted with more than 1000 individual contributors. This includes a total reach of more than 8000 people on hundreds of development teams spanning across six continents. Our conclusions are based on face-to-face coaching work in at least nine countries and on the interaction with companies spanning all kinds of industries, from video game to health care to aeronautics to government, to give some examples. As a result of this analysis, several challenges have been identified that need to be tackled to integrate risk analysis, and in particular risk analysis related to NFRs, in agile software development. 3.1 Traditional risk analysis practises for software development do not easily translate to Agile Traditionally, risk analysis was performed 'at design time' in the waterfall method, where design, development, and operations are sequential and discrete worlds. These approaches usually rely on a risk management expert and it becomes impractical for small and medium enterprises in terms of cost and finding the proper resources. It also becomes impractical for companies working toward the adoption of more agile software development methodologies. There exist some proposals for agile risk management [24] but they do not take into account existing risk and threat analysis techniques such as STRIDE [17] or DREAD [19], or other approaches for managing security risks such as OCTAVE [15], which are more focused on NFRs. 3.2 Analysis of risks should be continuous System and architectures evolve continuously. In Agile, risk management is often neglected as part of a backlog or sprint plan, which leads to a view that it is not important until the later stages of a development project. In SAFe, PI planning sessions are useful to explore new risks that are foreseeable at the time of planning. However, a risk analysis every 3 months may not be sufficient when risks are detected by any actor at any time. This exposes the general problem related to agile methodologies failing to effectively address risks and other NFRs in a structured manner. There is a need for risk analysis methodologies that are adapted to agile contexts but still achieve the level of analysis and detail provided by traditional risk assessment and mitigation techniques, in particular, related to NFRs. Fitzgerald and Stol [29] illustrate how lean thinking can be applied to continuous software engineering. However, they do not explicitly tackle challenges related to continuous risk management. Research into enabling continuous risk management can provide mechanisms and tools that resolve a challenge of implementing continuous analysis of risks. It is clear that a balance has to be struck between managing risk during the development process and not overloading or slowing the momentum of an agile methodology. 3.3 Teams do not have sufficient expertise on risk analysis Agile teams are usually cross-functional, optimised for communication, and delivery of value. Although this could facilitate the work of specialised staff across several teams to analyse risks, allocating resources with expertise on risk assessment in each team becomes impractical. It is not possible to have a risk analysis expert in each agile team. This encourages some level of centralisation but at the same time it requires transparency, and for agile/Scrum team members a higher capacity to participate and contribute in risk analysis and propose mitigation strategies when necessary. Consequently, lack of expertise is an important issue for agile self-managed teams but also for small or medium enterprises that cannot afford risk experts in each team. Detecting the most prominent risks and deciding the best mitigation actions may be a difficult task. Similarly, deciding the level of likelihood and impact of a particular risk may also be very subjective and, therefore, difficult to assess and measure. In addition, domain-specific risk analysis further aggravates the situation as the level of expertise becomes more demanding. While training is an essential aspect [37], lack of expertise may always be present and needs to be mitigated. 3.4 Tools to manage risk in Agile do not foster collaboration To improve the control of risk in Agile, tools that allow better transparency and enable collaboration of all stakeholders involved in the process are crucial. In this sense, creating new tools that can be easily embedded in other common agile tools to manage software development is very important. Current tools to analyse and manage risks are quite limited. These are usually implemented through excel spreadsheets that are shared during PI planning but do not allow for further collaboration beyond this face-to-face meeting. In general, there is a general lack of collaborative tools to engage all the stakeholders potentially involved in analysing risks in a transparent way, during PI planning and afterwards. Common generic tools such as Kanban-style boards are increasingly used by software industry for scheduling work, representing user stories, features etc. In this sense, some works suggest the use of risk-modified Kanbans [24, 25]. The detection of new risks or planning mitigation actions should immediately propagate information and even trigger warning or actions through these software development management tools. There is an added challenge to handling risk in a collaborative way. Collective inter-team code ownership makes it difficult to control risks related to a particular component. In Agile, multiple teams frequently modify the code associated with a single component. Consequently, collective ownership makes it more difficult to control potential risks related to a particular component. User stories will impact several components and many of them will typically need to modify the same shared component. Efficient tools do not exist to facilitate inter-team risk analysis. Beyond these four challenges, it is also worth noting that there is also an intrinsic obstacle to improve risk management related to the organisation culture. Cultural changes of any type require time and care to be successful. The challenge, in this case, is to develop a cultural change methodology that places risk management as a central and critical part of an agile methodology. Cultural change of this type is often top down, implemented by management mandating risk management as an integral part of every sprint. This has the potential to lead to varying levels of adoption by the development staff. 4 New framework for Agile risk management In this section, we propose a new agile risk analysis framework to facilitate the creation of tools for agile risk management. This framework addresses the four challenges (C1–C4) described in Section 3. It is a framework that facilitates translating traditional risk analysis practises for software development to agile software development contexts, allowing for continuous risk analysis, and permitting the main stakeholders, from the agile team members to any other level in SAFe, to collaborate together. This framework facilitates the control of risk associated with the assets of a system. The materialisation of an asset in each system may depend on the type of system and the main focus of the risk analysis. For instance, components in an application architecture of a software system being developed may be the assets to be analysed. However, in a different situation, risks to be analysed may be associated with user stories or product features. We conceive risk management as a continuous activity where risks may be subject to consideration and evaluation in the different stages of the software development life cycle (C2). Furthermore, the analysis assessment process is devised in a simple and visual way that simplifies the collaboration in a multi-disciplinary team of application development stakeholders (C4), frequently referred to as the DevOps team. We consider recommendations, constraints, and rules to guide and support the whole team and minimise the impact of having self-managed teams without adequate risk-management capabilities (C3). Finally, to address the agility challenge (C1) but also to foster continuity and collaboration, the Kanban process board philosophy is adopted to produce a risk assessment process board. Our framework uses a pull system in the style of Kanban, where the status of each asset with respect to a predefined risk analysis methodology is expressed through the different columns in the Kanban board. This makes our framework agnostic to any specific risk analysis methodology. A tool compliant with this framework links a Kanban-style representation with a particular risk analysis methodology, mapping each of the steps of that particular methodology to one of the columns in the Kanban. This naturally provides a solution to most of our challenges (C1, C2, and C4), as it provides a mechanism to include traditional risk analysis methodologies into an agile-ready tool such as the Kanban. Kanban-style boards are used in continuous deployment environments, fostering collaboration among team member