Analytic Hierarchy Process Based Prioritisation and Taxonomy of Success Factors for Scaling Agile Methods in Global Software Development

IET SoftwareVolume 14, Issue 4 p. 389-401 Research ArticleOpen Access Analytic Hierarchy Process Based Prioritisation and Taxonomy of Success Factors for Scaling Agile Methods in Global Software Development Mohammad Shameem, Corresponding Author shameem.ism@gmail.com Department of Computer Applications, Madanapalle Institute of Technology & Science, Madanapalle, AP, IndiaSearch for more papers by this authorArif Ali Khan, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of ChinaSearch for more papers by this authorMd. Gulzarul Hasan, Department of Mathematics, Madanapalle Institute of Technology & Science, Madanapalle, AP, IndiaSearch for more papers by this authorMuhammad Azeem Akbar, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of ChinaSearch for more papers by this author Mohammad Shameem, Corresponding Author shameem.ism@gmail.com Department of Computer Applications, Madanapalle Institute of Technology & Science, Madanapalle, AP, IndiaSearch for more papers by this authorArif Ali Khan, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of ChinaSearch for more papers by this authorMd. Gulzarul Hasan, Department of Mathematics, Madanapalle Institute of Technology & Science, Madanapalle, AP, IndiaSearch for more papers by this authorMuhammad Azeem Akbar, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, People's Republic of ChinaSearch for more papers by this author First published: 01 August 2020 https://doi.org/10.1049/iet-sen.2019.0196Citations: 8AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat Abstract Global software development (GSD) organisations are currently adopting agile frameworks in order to efficiently develop a software product. The main objective of this study is to identify the success factors (SFs), which could possibly have a positive impact on scaling agile practices in a GSD environment and develop their taxonomy based on their prioritisation using the analytic hierarchy process (AHP) approach. This study is conducted in four stages: problem identification and goal of the study (1), identification of SFs and their categorisations (2), validation of the SFs using questionnaire survey (3), and application of AHP to prioritise the SFs and develop the taxonomy of the SFs and their respective categories (4). The results of this study indicated that ‘technology’ is the most significant category as compared to the other categories of the SFs. Similarly, rich technological infrastructure is identified as a most important factor. Based on the findings of this study, authors can conclude that the contribution of this study is not only limited to development of the taxonomy of the SFs, but also their proper prioritisation by introducing AHP approach, which assists software organisations to scale agile methods effectively in the GSD environment. 1 Introduction The majority of software development organisations adopted the concept of global software development (GSD) in order to get various economic and financial benefits [1, 2]. In the GSD projects, the development teams operate under the core three distributed dimensions: geographical, cultural, and temporal distances, which make the development activities more complex and bring several challenges, especially when the agile methodologies are implemented [3]. Currently, in software development organisations, agile methodologies have gained considerable attention that provides opportunities to manage the change in system development activities [4]. Agile software development is a lightweight software development approach based on incremental and iterative development processes that support frequent, continuous deployment practices [5]. In agile methods, development and testing activities are performed concurrently with the customer collaboration, unlike the traditional approach [5]. The frequent change in system requirements motivates software development organisations to adopt agile manifesto that can assist them in delivering a faster and high-quality product that satisfies the requirements of the client's organisation [4]. Agile methodologies differ from the traditional software development approaches (i.e. Waterfall Model) in such a way that they produce less project documentation, fast project delivery, accommodate requirements changing, increasing customer satisfaction, improve product quality, and provide transparency to the customers [4]. There are various agile software development frameworks and models that are based on the agile manifesto, e.g. Dynamic System Development Approach (DSDA), Extreme Programming (XP), Scrum, Adaptive Software Development (ASD), and Feature-Driven Approach (FDA) [4, 6, 7]. Nowadays, agile development methods have demonstrated improvements in developing frequent testable and deployable solutions, decreasing time-to-market, and increasing flexibility for change during the project life cycle [8]. The flexibility for change during the project life cycle has motivated most of the software organisations to implement the agile framework in a GSD environment in order to get benefits such as low-development cost and early delivery of the product [9, 10]. Agile frameworks are applied in GSD settings through communication and subsequent knowledge exchange between globally outsourced distributed teams. Despite the various benefits of using agile software development (ASD) in a GSD environment involves multiple challenges, including coordination, communication, change management, cultural conflicts, temporal distances, etc. [9, 11]. The agile methodology was initially proposed for collocated, small, and single-team projects. The collocated projects as best suited for the agile development [9]. According to Paasivaara [12], agile methods and a GSD environment work on different principles and this could be the main cause for these challenges, which increases the complexities of scaling agile software development activities in the GSD environment. Moreover, Shameem et al. [13] conducted an empirical study using a questionnaire survey to investigate the challenges and prioritise them based on their relative importance for scaling agile development in the distributed environment. In their study, the results highlighted that lack of organisational commitments, difficulties in face-to-face communication, information sharing, and managing the requirements are the most important challenges on which GSD organisations need to focus on implementing agile methodologies. The available literature has given limited attention to investigate the significance of the agile framework in the global software development environment and particularly the success factors (SFs) that could positively impact the scaling process [9, 12]. The SFs represent the key areas that need to be focused in order to scale agile methods in the GSD environment [4, 8]. Various research studies are conducted and discussed the key motivators for adopting the agile approaches in the domain of large scale and collocated software development [4, 8, 11], but there is research gap of identifying the SFs of agile development in the domain of GSD environment [6, 12]. However, in the modern software development era, most of the organisations are practicing global software development [12, 14], therefore it is a need of study that could highlight the significance of agile development SFs in the GSD environment. SFs are the motivators that give an understanding of agile methods in the GSD context. In this study, we have investigated the SFs discussed in the agile and GSD literature. The identified factors were further categorised and prioritise using the AHP method. The AHP technique has been proposed by Saaty [15] to manage the complex decision-making problem. AHP is a well-known multi-criteria decision-making approach that is previously used in different other studies for decision-making problems [16, 17]. Hence, we have followed the step-by-step instructions of the AHP approach in order to evaluate the relative significance and criticality of the identified SFs. Following are the research questions develop to address the given research gap: RQ1: What are the SFs discussed in the available literature and recommended by the experts for scaling agile method in GSD. RQ2: What would be the priority of the identified factors based on the AHP analysis? The remaining of the paper is organised as follows: the motivation and research gap of the study is discussed in Section 2. The background of the study is discussed in Section 3. In Section 4, the analytic hierarchy process (AHP) method is briefly presented. The research design is discussed in Section 5 and the results and discussions of the study are reported in Section 6. The limitations and implications of the study are discussed in Sections 7 and 8, respectively. Finally, the conclusions and future directions of the study are presented in Section 9. 2 Research gap and motivation Currently, the majority of the organisations are adopting agile methodologies for developing frequent testable and deployable solutions, decreasing time-to-market, and increasing flexibility for change during the project life cycle [12]. Agile development is a lightweight software development process that provides the opportunities to manage the dynamic behaviour of system development requirements [5]. The injection of regular feedback loops and informal communication and stakeholder engagement among customers, managers, and the project team also enhance the quality of projects undertaken in agile environments [6]. The fundamental philosophy of agile methods is derived from the agile manifesto created by several system design professionals who have created distinct frameworks based on comparable concepts focusing on teamwork, individual imports and their interactions, early product delivery, customer cooperation and efficient reactions to change demands [18, 19]. The majority of the organisations have followed the concept of GSD in order to achieve business benefits [20]. It is also significant that the GSD team members have complete understanding, knowledge, and awareness of agile development in the GSD environment. According to Niazi et al. [21], most of the client and vendor organisations are following the phenomena of GSD for developing a quality product in low development cost. The implementation of agile methods in the GSD environment is different as compared with their implementation in the collocated environment (single site development). Various challenges have been reported for scaling agile development in the GSD environment. Shrivastava and Rathod [9] conducted a study and identified 43 risk factors in the distributed agile development. Layman et al. [22] reported that the three distributed dimensions (geographical distance, cultural distance, and temporal distance) brings communication challenges among the team members, which decreases the direct communication opportunities and increases the conflicts among the team members dispersed across the distributed environment. Shameem et al. [13] conducted an empirical study and identified 22 challenges for scaling agile development. They have prioritised the identified challenges using AHP. Moreover, they have discussed the need for a study that can highlight the SFs for successful scaling the agile methods. This study is conducted with the motivation of our previous study Shameem et al. [13], to explore the SFs that can positively impact the agile software development methodologies in the context of GSD. In summary, few research studies have been done to discuss the scaling practices of agile methods in the GSD [9, 12, 22]. To the best of our knowledge, no study has been conducted, where the SFs are categorised and prioritised based on their relative importance. In a very recent study, Kalenda et al. [23] conducted a systematic literature review (SLR) and identified the challenges, SFs, and practices to address them. However, they have not discussed the relative importance of all the factors identified for implementing agile development in the large-scale software organisations. Therefore, this research gap is addressed in this study by discussing the SFs for scaling agile methods in a globally distributed environment. Before implementing the agile practices, the development of a framework based on the SFs and their rankings could be imperative for the successful utilisation of agile practices in the GSD organisations. Therefore, the findings of this study will provide a taxonomy of the SFs that could help the software organisations to revise their management strategies for evaluation and improvement for scaling agile practices in the GSD environment. In addition, the knowledge about the identified SFs will significantly contribute to developing a generic and robust model that could assist GSD outsourcing organisations to focus more on the SFs to achieve the best process outcomes with limited human resources, capital, technological tools, and fast product delivery-to-market. 3 Background study The number of studies has reported SFs in different contexts of the globally distributed software development. Layman et al. [22] considered communication between the team members as the most importance factor in the distributed extreme programming (XP) development and highlighted that effective communication between the distributed team make effective coordination between them and control the agile software development activities in the GSD environment which have also been supported by different researchers [11, 24]. The factors that need to be required for effective communication practices which were considered as SFs for scaling agile development. Similarly, Herbsleb and Mockus [1] suggested that team members often use different communication technologies to communicate and exchange project related activities between the distributed team members. The poor technological infrastructure between the distributed sites can cause project delay, security issues, and information sharing problems. Dikert et al. [4] conducted an SLR study and reported the significance of various SFs in the domain of large size organisations including continuous communication, continuous learning, trust, continuous customer involvement, development transparency, organisational support, management commitment, agile training, and rich communication infrastructure between the development sites. These factors have also been cited by the authors [8, 25-27]. Additionally, in their study, Laanti [7] presented a framework for implementing agile methods in large distributed projects. The framework was based on five levels which was assessed by conducting pilot study in large scale agile project to assess the operational validity of the framework. They have considered the practitioners coaching and training as the two important factors for successful implementation of agile practices in the large scale projects. The customer involvement in globally distributed agile projects make the development activities more effective and efficient in several tasks, i.e. writing user stories, discussing product features, prioritising the client requirements and the regular feedback and engagement among the managers, and the project team to enhance the quality of projects undertaken in agile environments [14, 28]. Hoda et al. [24] highlighted the importance of customer involvement in the development activities and their impact on real-life agile projects. They have conducted a qualitative study and collected data from 30 practitioners working in the domain of agile projects. The data was analysed using the grounded theory approach, and the reported results demonstrate that the customer involvement is one of the most important factors in Globally Distributed Agile Software Development (GDASD). It will bring the agile team and the customers in close communication and coordination [2]. Recently, a case study was conducted by Paasivaara [12] with the objective to scale agile practices in a large-scale organisation ‘Comptel’. This case study highlighted the significance of agile development in the distributed organisations and discussed the importance of using a common framework for scaling agile methods in a globally distributed environment. This study suggests that the key issues with agile development are not related to lack of models and standard, but rather a lack of effective strategies to successfully scale the agile methods in real-world GSD environment. Kalenda et al. [23] conducted a literature review and reported the importance of practices, barriers, and SFs for implementing agile development in the GSD context. In their findings, they have identified several factors, i.e. management support, prior experience, prior training, technological infrastructure, effective communications as the most critical SFs which positively impact on scaling programs of agile development in the large-scale distributed software development organisation. Finally, we have extracted a total 22 factors which could positively impact on scaling agile practices in the GSD environment (Table 1). We have further classified the identified SFs into the five core categories of the framework developed by Ramasubbu [30]. Ramasubu [30] proposed a framework to categorise the key issues of software process improvement implementation in the GSD environment. He has classified the identified issues into the five core categories: management, coordination, methodology, human resource management, and technology factors. Due to the similar nature of both studies, hence we have also adopted the concepts of Ramasubbu [30] framework and classified the identified SFs in the five above mentioned categories. The categorisation process was informally done by all the authors of the paper based on their research experience and understanding of the SFs. The complete list of the factors along their categories is provided in Table 1. The same categorisation technique has been previously used in different other studies [2, 30, 31]. Table 1. Identified SFs and their source of adoption Main categories of factors SFs References management (C1) Continuous organisational support (SF1) [1, 4] Conducting the prior training (SF2) Organisational commitments (SF3) Strong Client–Vendor relationships (SF4) Improving inter-personal relationships (SF5) technology (C2) continuous knowledge sharing (SF6) [4, 8, 14] continuous learning environment (SF7) overseas site's response (SF8) rich technological infrastructure (SF9) continuous information sharing (SF10) human resource (C3) motivation (SF11) [12, 24] skilled human resources (SF12) self-organising teams (SF13) strong relationship between team members (SF14) encouraging project visibility (SF15) coordination (C4) 3C (communication, coordination, and control) (SF16) [7, 29] effective requirements analysis (SF17) frequent customer involvement (SF18) methodology (C5) short iteration (SF19) [4, 8] team size (SF20) effective retrospective (SF21) coaching for scaling agile development (SF22) 4 AHP method AHP is a technique for complex and multi-criteria decision making (MCDM) problems, that are used to determine the relative importance between the set of multiple criteria. AHP was initially proposed by Saaty [15], and has been used in the different other domains for solving complex decision-making problems [32-34]. AHP uses the hierarchical structure of the problem to handle both qualitative and quantitative criteria [35, 36]. AHP method is based on three core phases: a hierarchical structural model of the problem; comparative judgment of alternative, criteria; and synthesis of the results. We have briefly discussed the MCDM AHP procedure in the following sections: Creation of the hierarchical structure of the complex decision problem. In the first step, a complicated decision-making problem is presented in a hierarchical structure in which the problem breaks down into a hierarchy of interrelated decision elements (criteria, decision alternatives). A hierarchical tree of the objective, criteria, and sub-criteria is structured (as shown in Fig. 1) using the AHP approach. All objectives of the problem are presented at the top level of hierarchy and criteria at the middle level. The sub-criteria are presented at the lower level of the hierarchy [32]. Fig. 1Open in figure viewerPowerPoint Hierarchical structure of the problem Table 2. Linguistic variables used for getting responses from the experts Verbal judgment or preferences Numerical rating equally preferred (EP) 1 moderately preferred (MP) 3 strongly preferred (SP) 5 very strongly preferred (VSP) 7 extremely preferred (EXP) 9 intermediate values 2, 4, 6, 8 Table 3. Relationship between size of matrix and randomly generated consistency index (RI) n 1 2 3 4 5 6 7 8 9 10 RI 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49 Calculating the priority weight of all criteria at each level of hierarchy with the help of pair-wise comparison. Once the problem has been decomposed and presented in a hierarchical structure, the prioritisation starts in the second step in order to find the relative importance of the criteria within each level and finish at the lower level. In each level hierarchy, each criterion is pairwise compared with all other criteria [35]. In AHP, the preferences of the decision-makers are quantifies using a standard 9-point AHP scale (as shown in Table 2) and the possible preferences of the decision-makers are as follows: 1 = ‘Equally preferred (EP)’, 3 = ‘Moderately preferred (MP)’, 5 = ‘Strongly preferred (SP)’, 7 = ‘Very strongly preferred (VSP)’, 9 = ‘Extremely preferred (EXP)’, and 2, 4, 6, and 8 shows the values in between. In MCDM problems AHP approach, the pairwise comparison matrices are used. Suppose we have n different criteria having the weights , respectively. The results of the pairwise comparison on the n criteria are presented in an () evaluation matrix (as shown (1)) in which every element is the quotient of weight to the respective criteria as shown in the matrix B. (1) Calculation the consistency of the pairwise comparison matrix. At the last step, the validity of pairwise comparison must be evaluated, which is tested using the consistency ratio and consistency index , which can be calculated using (2) and (3), respectively, as suggested by Saaty [15]. (2)where = largest eigenvalue of matrix of order n. w: local priority weight of the criteria. x: column sum of the pair-wise comparison matrix. (3)where RI is the value of randomly generated consistency index for different size (n) of the matrix as given in Table 3. The accepted value of CR changes with the size of the pair-wise comparison matrix, i.e. for a 3 × 3 matrix, it is 0.05, for a 4 × 4 matrix, 0.08 and for all the matrices, of n > = 5, it is 0.1. If the calculated CR value is equal to, or less than the specified value, this indicates that the evaluation within the matrix is acceptable or it indicates that the evaluation in the pair-wise comparison matrix is acceptable or it is very close to the ideal values. If the final consistency ratio exceeds this value, the procedure has to be therefore reviewed, reconsidered, and improved until the acceptable value of CR is achieved. According to Wang and Yang [37], the purpose of measuring CR is to illustrate the consistency of decision-makers as well as the consistency of the overall hierarchy of the problem. In various studies, researchers have followed the AHP method to prioritise the factors relevant in their studies [34, 38-41]. For example, Jovanović et al. [39] used the AHP method for ranking the manufacturing sectors in Serbia for an energy management system. Kabra et al. [34] used the AHP method to prioritise the coordination challenges in the humanitarian supply chain management. Shameem et al. [13] conducted an empirical study and identified 22 challenges for scaling agile development. They have prioritised the identified challenges using AHP. Moslem and Duleba [41] conducted an AHP based study for evaluating passengers demand to improvement the public transport in Mersin, the city of Turkey. In their study, they have identified few frequently alluded factors and used the analytic hierarchy process that has been applied based on the questionnaires that have been used to collect data from the experts regarding the hierarchy of quality factors, and as evaluators. In this study, we have briefly discussed the AHP method to prioritise the categories and their respective SFs for scaling agile development in the GSD environment in Section 5.4. 5 Research design The goal of this study was to identify the SFs of scaling agile methods in GSD and prioritise them based on their relative importance. The research method to achieve these objectives is carried out in the following four stages: Stage 1: Problem identification and goal of the study. Stage 2: Identifying the SFs which could positively impact agile methods in the GSD environment. Stage 3: Validation of the SFs using the questionnaire survey conducted with the agile practitioners. Stage 4: Prioritising the SFs using the AHP method. All the stages of our research methodology are briefly discussed in the following sections and shown in Fig. 2. Fig. 2Open in figure viewerPowerPoint Schematic diagram of research process 5.1 Stage 1: problem identification and goal of the study The problem identification and objectives of this study are as briefly discussed in Sections 1 and 2. 5.2 Stage 2: identification of the SFs from the literature review The SFs in a globally distributed agile development were identified through a literature review as discussed in Section 3. The identified SFs were categorised in five different categories, i.e. Management, Technology, Human resource, Coordination, Methodology based on the discussions with the experts. 5.3 Stage 3: validation of the SFs using questionnaire survey 5.3.1 Questionnaire development for validating the SFs In this stage, the SFs and their categorisation identified from the literature were validated using the approach of the survey questionnaire. An on-line survey instrument was developed to collect responses from the industry practitioners and researchers. In each category of the factors, the open-ended questions were also added by the authors to identify the additional factors which were not identified during the review process. The authors followed the questionnaire-based survey method because it offers a number of advantages and some of these advantages are mentioned below: Using the questionnaire survey, it is easy to collect data from the respondents located at various geographically distributed locations. The survey can be administered easily, quickly and effectively. The on-line survey is cost-effective. 5.3.2 Pilot evaluation of the questionnaire A pilot evaluation of the questionnaire was conducted in order to assess the content, possible weaknesses, and key variables of the questionnaire. In our study, the questionnaire was evaluated by experts of agile development and GSD projects including the empirical software engineering professors from the City University of Hong Kong, COMSATS University, Islamabad, Pakistan, King Fahd University of Petroleum and Minerals Saudi Arabia, and Indian Institute of Technology (Indian School of Mines), Dhanbad. The responses obtained in the pilot study were tested based on the following points: Consistency among the respondents was checked in the responses. To check any question is skipped by the respondents. To check the SFs and their respective category. The questionnaire was further assessed on issues that could have a negative impact on the response rate. Most of the suggestions and comments were given by respondents of the pilot study were based on the rephrasing of the SFs and to improve the response rate of the survey. Two experts suggested moving SF (i.e. the strong client–vendor relationship) to the management category rather than given in the coordination category of the SFs. However, all the suggestions were related to improving the quality of the instrument, hence, all the suggested changes were incorporated in the questionnaire, and the final version of the survey instrument was developed. A sample of the survey questionnaire is provided in Appendix 1. 5.3.3 Data sources and data collection As the objectives of this research study were to identify and validate the SFs which could positive impact on scaling program of agile development in the GSD. Therefore, it was important to approach diverse range of agile practitioners working in the domain of globally distributed agile development. The target population were approached by sending the invitation to the various software professionals, and software development com