A 20‐year mapping of Bayesian belief networks in software project management

IET SoftwareVolume 16, Issue 1 p. 14-28 REVIEWOpen Access A 20-year mapping of Bayesian belief networks in software project management Abraham L. R. de Sousa, Corresponding Author Abraham L. R. de Sousa lincoln.rabelo@ufopa.edu.br orcid.org/0000-0003-3365-568X Institute of Engineering and Geosciences, Federal University of Western Pará (Ufopa), Belém, Pará, Brazil Institute of Exact and Natural Sciences, Federal University of Pará (UFPA), Belém, Pará, Brazil Correspondence Abraham L. R. de Sousa, Institute of Engineering and Geosciences, Federal University of Western Pará (Ufopa), Belém, Pará, Brazil. Email: lincoln.rabelo@ufopa.edu.brSearch for more papers by this authorCleidson R. B. de Souza, Cleidson R. B. de Souza Institute of Exact and Natural Sciences, Federal University of Pará (UFPA), Belém, Pará, BrazilSearch for more papers by this authorRodrigo Q. Reis, Rodrigo Q. Reis Institute of Exact and Natural Sciences, Federal University of Pará (UFPA), Belém, Pará, BrazilSearch for more papers by this author Abraham L. R. de Sousa, Corresponding Author Abraham L. R. de Sousa lincoln.rabelo@ufopa.edu.br orcid.org/0000-0003-3365-568X Institute of Engineering and Geosciences, Federal University of Western Pará (Ufopa), Belém, Pará, Brazil Institute of Exact and Natural Sciences, Federal University of Pará (UFPA), Belém, Pará, Brazil Correspondence Abraham L. R. de Sousa, Institute of Engineering and Geosciences, Federal University of Western Pará (Ufopa), Belém, Pará, Brazil. Email: lincoln.rabelo@ufopa.edu.brSearch for more papers by this authorCleidson R. B. de Souza, Cleidson R. B. de Souza Institute of Exact and Natural Sciences, Federal University of Pará (UFPA), Belém, Pará, BrazilSearch for more papers by this authorRodrigo Q. Reis, Rodrigo Q. Reis Institute of Exact and Natural Sciences, Federal University of Pará (UFPA), Belém, Pará, BrazilSearch for more papers by this author First published: 09 November 2021 https://doi.org/10.1049/sfw2.12043AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract In recent years, more software engineering researchers have focussed on methods, techniques, tools, and processes to support software project management (SPM) addressing quality, cost, and time constraints. In this direction, the use of Bayesian belief networks (BBNs) has gained attention for providing a powerful mechanism for cause–effect analysis with both qualitative and quantitative data to support decision making under uncertainty. This work aims to provide an overview of the first 20 years of research in BBNs applied to SPM, and by so doing it contributes to the structuring of this research topic and the identification of new research opportunities. We conducted a systematic mapping (SM) study on the use of BBN as a decision-support tool for SPM issues. We analysed over 109 relevant publications, from 1999 to 2018 (i.e., 20 years), to understand the motivations for using Bayesian networks (why), the problem domain and model scope addressed by researchers (what), the stage of the life cycle in which BBNs are being used (when), the venues of the publications (where), and the tools used to model the Bayesian networks (how). We draw the following conclusions from the results of our SM: (1) The application of Bayesian networks in SPM has been an active research topic since 1999. (2) Prediction and planning are the most common purposes in 60% of selected papers. (3) Software quality (55%) is the problem domain most investigated. (4) Most of the surveyed studies embrace the project (47%) and product (37%) scopes. And finally, (5) the development phase (86%) is when Bayesian networks have been more used. As future work, we highlight the need to further investigate methodologies that enable the use of BBNs in real software development contexts. 1 INTRODUCTION Bayesian belief networks (BBNs) are a modelling technique for causal relationships that are based on Bayesian inference. BBNs have gained attention since they offer a powerful mechanism to conduct cause–effect analysis with both qualitative and quantitative features and, as a consequence, to support decision making under uncertainty. In this work, we consider that uncertainty is an inherent feature of the decision-making process. There are many uncertainties in software project management (SPM), for example, (a) uncertainties in estimating project size, schedule, cost, quality; (b) uncertainties regarding the impact of changes in requirements; and (c) uncertainties about the impact of new technology adoption. Neil and Fenton [1] and Johnson [2] were the first ones to propose BBNs for modelling software development project uncertainties. Although BBNs have been emerging as a useful mechanism to evaluate and analyse software process uncertainties, their usage and evolution are not covered comprehensively by previous studies. Therefore, this work aims to provide a comprehensive view of the growing adoption of BBNs to support SPM, and by so doing, it contributes to the structuring of this research topic and the identification of new research opportunities. To achieve such a goal, we have performed a systematic mapping (SM) study covering 20 years (i.e., from 1999 to 2018) of BBN usage as a decision-support tool in SPM issues. According to Petersen, Vakkalanka and Kuzniarz [3], a SM study, also called a scoping study, is designed to give an overview of a research area and allows one to discover research gaps and trends. We analysed over 109 relevant publications to find answers to our research questions that includes an understanding about the motivations for using Bayesian networks (why), the problem domain and model scope addressed by researchers using BBNs (what), the stage of the life cycle in which BBNs are used (when), the venue in which the publication was reported (where), and the tools used to model the Bayesian networks (how). Our results show that Bayesian networks in SPM have become an active research topic, especially for the purpose of prediction and planning of issues related to software quality. The project and product scopes are the most explored by the surveyed studies. Similarly, the development phase is when Bayesian networks have been more used. And, finally, we conclude that more studies need to be carried out using Bayesian networks in real software development contexts. The remainder of this work is structured as follows: The following section briefly describes the concept of BBNs, its features, and potential. The reader familiar with these concepts may skip this section. Section 3 describes the methods adopted in our SM, including our research questions, search strategy, study selection criteria, and data extraction methods. Section 4 discusses the results answering each one of our research questions. The threats to validity are presented in Section 5. And finally, the conclusions of our work are presented in Section 6. 2 BACKGROUND: BBNs A BBN, or simply Bayesian network or BBN, is a model suited to representing knowledge of a complex domain where this knowledge is uncertain, ambiguous, or incomplete. It is especially useful in situations involving reasoning and decision making under uncertainty. Charniak [4] recommends the use of BBNs to model situations in which causality is relevant, but our understanding of what is happening is incomplete, uncertain and, therefore, needs probabilistic treatment. Graphically, a BBN is an acyclic-directed graph based on the notion of cause–effect relationships. Each node represents a variable of a specific domain, while the arcs indicate the existence of probabilistic dependence between the connected nodes. According to Mendes et al. [5] and Kjaerulff [6], a BBN uses two components to represent knowledge. The first component represents the qualitative structure of a BBN, depicted by the directed acyclic graph's nodes and arcs. The nodes represent the relevant variables (e.g., factors) in the modelled domain, and each directed arc depicts dependence between the factors. The second component represents the quantitative structure of a BBN and associates a conditional probability distribution (CPD) to each node. A parent node's CPD (Figure 1, nodes A and B) describes the relative probability of each state. Meanwhile, a child node's CPD (Figure 1, node C) represents the relative probability of each conditional state on every combination of states of its parents. So, the relative likelihood of C being equal to medium probability, with A and B being both low is equal to 0.2. Each row in a CPD represents a conditional probability distribution, and, therefore, the sum of its values must be equal to 1. FIGURE 1Open in figure viewerPowerPoint Illustrative Bayesian belief network and its conditional probability distribution BBNs are based on Bayes' rule (also known as Bayes' theorem) and, therefore, are considered as a formal means of dealing with inaccurate data and expert judgement. Bayes rule allows the calculation of unknown probabilities from known conditional probabilities. In practical terms, it gives the decision-maker the possibility of updating one's subjective belief when new facts arrive. Bayes' rule is an immediate consequence of the total probability theorem that allows the calculation of the posterior probability: P ( A | B ) = P ( B | A ) P ( A ) P ( B ) (1)wherein P (A) and P (B) are a priori probabilities of A and B. P (B | A) and P (A | B) are a posteriori probabilities of conditional B to A (probability of B given A) and conditional A to B (probability of A given B), respectively. Thus, Bayes' rule shows how it is possible to change a priori probabilities, from new evidence, and obtain a posteriori probability. The following is an example of using Bayes rule, adapted from Pena [7], in the context of software development and related to the possibility of finalising a project in the allocated time. 2.1 An example of Bayesian networks in project management Suppose that 75% of the projects developed by a company were finished on schedule. A specific technique of time estimation used by the company provides 80% accuracy in projects finished on schedule and 20% of projects that end up late. Suppose now that the company's project manager applied the estimation technique and (s)he found that the project will end on time. How likely is this project to be completed on schedule? Now, let us apply Bayesian reasoning to this problem. The project may end on time (inside) or not (outside). These mutually exclusive alternatives are in Table 1. A priori probability represents the project's chance of finishing on time, before any forecast. Let us suppose we know that 75% of completed projects are on time, the a priori probability of completion on time is 75% (0.75), and late completion is 25% (0.25). TABLE 1. A priori probability Inside Outside a priori 0.75 0.25 Note: Source: Adapted from [7]. The next step, presented in Table 2, considers the result of the estimation technique to calculate the a posteriori probability. We know that according to the estimation technique adopted by the company, if the project finishes on time, the conditional probability of a correct result is 80% (0.80) and 20% (0.2) for late projects. Thus, we can obtain the joint probability by multiplying a priori probability with the conditional probability. TABLE 2. Joint and Conditional probabilities Inside Outside a priori 0.75 0.25 Conditional 0.8 0.2 Joint 0.75 × 0.8 = 0.6 0.25 × 0.1 = 0.05 Note: Source: Adapted from [7]. Note that the a priori probabilities have a sum equal to 1. However, this does not occur for joint probability. Thus, it is necessary to normalise the joint probability so that its sum becomes 1, and, therefore, we will have the a posteriori probability. For normalisation, we must divide each joint probability by its sum. In this case, the a posteriori probability that the project will finish on time is 92% (see Table 3). TABLE 3. Joint and Conditional probabilities Inside Outside a priori 0.75 0.25 Conditional 0.8 0.2 Joint 0.75 × 0.8 = 0.6 0.25 × 0.2 = 0.05 Normalisation (0.6 + 0.05 = 0.65) a posteriori 0.6/0.65 = 0.92 0.05/0.65 = 0.08 Note: Source: Adapted from [7]. 2.2 Advantages of Bayesian networks The literature [8, 9] presents several advantages of Bayesian networks, including (1) Ease of understanding and interpretation: all parameters of a Bayesian network have formal semantics defined and are grounded by Bayes' theorem, avoiding ambiguities. The graphical representation of Bayesian networks simplifies the modelling and understanding of causal relationships among variables, even in complex domains. (2) Treatment of lost data: BBNs support qualitative and quantitative data. In the absence of data, a network can be constructed from the knowledge of domain experts or by using probabilistic distributions. When there is data, but it is incomplete, a BBN can learn the relationship between its variables or the state of its variables through algorithms. (3) Dealing with causal relationships: BBNs allow the analysis of problems using cause–effect reasoning, based on probabilistic conditional statements. Because BBNs have causal and probabilistic semantics, it is possible to combine data a priori with current knowledge of the problem domain being modelled. (4) Two-way (bidirectional) inferences: from a model and a set of defined inputs, it is possible to observe the outputs produced by a BNN; or through an output configuration given by an expert, it is possible to observe the desired input. (5) Tool support: There are many tools available that support the modelling and simulation of Bayesian networks. These tools implement algorithms for data processing and for Bayesian inference that makes their application efficient and practical. Despite the advantages presented above, Bayesian networks also have limitations. For instance, it is not recommended to use Bayesian networks for the treatment of problems in which the cause–effect relationship between domain variables changes frequently. In this case, each change in the relationships between the variables implies updating the CPD values of the respective variables and restarting the model validation process. 3 BAYESIAN NETWORKS IN PROJECT MANAGEMENT OF SOFTWARE DEVELOPMENT: A SYSTEMATIC MAPPING According to Petersen, Vakkalanka and Kuzniarz [3], SM studies or scoping studies are designed to give an overview of a research area by classifying and counting contributions in relation to the categories of that classification. It involves searching the literature to find what topics have been covered in the literature and where publications about these topics have appeared. SM studies are concerned with structuring a research area. Their research questions are general as they aim to discover research gaps and trends. SM reviews a broader topic and classifies the primary research papers in that specific domain. The research questions for such a study are of quite a high level and include issues such as which sub-topics have been addressed and what sub-topics have sufficient empirical studies to support a more detailed systematic review. SM studies use the same basic methodology as systematic literature reviews (SLRs) but aim to identify all research related to a topic rather than answering questions about the relative merits of competing technologies that conventional SLRs address [3, 10]; in other words, a standard SLR is driven by a very specific research question [11], such as ‘Are algorithmic cost models more accurate than expert judgement-based estimates?’ Kitchenham et al. [11] highlight additional differences between SMs and SLRs for those interested in this topic. In this work, we consider that uncertainty is an inherent feature of the decision-making process and BBNs play an important role as a decision-support tool. Based on this line of reasoning, we believe that conducting a review of the use of Bayesian networks in decision support for SPM issues can contribute to the structuring of this research topic and enable the identification of new research opportunities. 3.1 Research questions According to the guidelines presented by Petersen, Vakkalanka and Kuzniarz [3] and Kitchenham and Charters [12], the first step in the development of a mapping study is the definition of the research questions. Our primary goal is to structure the research related to the adoption of BBNs towards decision making during SPM. Therefore, we used as a baseline the 5W2H information gathering approach, which aims to collect answers to the following questions: Who, What, When, Where, Why and How. However, we decided to suppress some of the 5W2H typical questions: (a) we suppressed the ‘How Much’ field due to the lack of information on the majority of the retrieved studies and (b) we also disregarded the ‘Who’ field because this study focusses on the project manager's viewpoint, and this would be the standard answer to this question. Table 4 presents the questions in a 5W2H way. TABLE 4. Research questions Why Why was a BBN adopted? What What was the problem domain? What was the model scope? When When did the investigate problem happen during the software development life cycle? How How was the BBN modelled? Where Where and when was the study published? Abbreviation: BBN, Bayesian belief networks. Below, we discuss each research question: RQ1. Why each selected study adopted BBNs as a modelling technique? We set up this question into two perspectives. RQ1.1 What was the general purpose of the study? How did the authors of the study justify using BBNs? What are the most common BBN features pointed out in the literature? In general, we want to explore the reasons and motivations for BBNs being used in the selected papers. And RQ1.2 What was the study's specific purpose? We adopted the classification of purposes presented by Zhang, Kitchenham and Pfahl [13] adapted from Kellner, Madachy and Raffo [14]. Table 5 presents our definition for each purpose used in this work: TABLE 5. Specific purposes descriptions Specific purpose Description Prediction and planning Forecasting project planning parameters Control and operational management Tracking and monitoring software project parameters Process improvement Identifying and analysing the best practice to enhance an existing software process Understanding Promoting enhanced insight into a problem Process investigation Evaluating and analysing software project management issues Technology adoption Evaluating the impact of new practices and technologies Communication Modelling and simulating communication issues Training and learning Learning the impacts of common decisions Risk management Characterising, identifying, analysing project risks Tradeoff analysis and optimisation Seeking a better condition or the impact of decisions RQ2. What was the problem domain investigated? Our goal in answering this question is to deepen our BBN knowledge in the context of SPM, investigating which management domains are tackled by the selected papers. We want to find out the project management questions and issues in which the BBNs were tested as a decision-support tool. We categorised the domains of project management according to the knowledge areas defined in the PMBoK [15]: Project Integration Management. Includes the processes and activities to identify, define, combine, unify, and coordinate the various processes and project management activities. Project Scope Management. Includes the processes required to ensure that the project includes all the work required, and only the work required, to complete the project successfully. Project Schedule Management. Includes the processes required to ensure the timely completion of the project. Project Cost Management. Includes the processes required to ensure that the project can be completed within the approved budget. Project Quality Management. Includes the processes for incorporating the organisation's quality policy regarding planning, managing, and controlling project and product quality requirements to meet stakeholders' expectations. Project Resource Management. Includes the processes to identify, acquire, and manage the resources needed for the successful completion of the project. Project Communications Management. Includes the processes required to ensure timely and appropriate planning, collection, creation, distribution, storage, retrieval, management, control, monitoring, and ultimate disposition of project information. Project Risk Management. Includes the processes of conducting risk management planning, identification, analysis, response planning, response implementation, and monitoring the risks of a project. Project Procurement Management. Includes the processes necessary to purchase or acquire products, services, or results needed from outside the project team. Project Stakeholder Management. Includes the processes required to develop appropriate management strategies for effectively engaging stakeholders in project decisions and execution. RQ3. What was the model scope? The model scope depicts the boundaries of the model according to the predominant data type (i.e., variables) used to model each BBN. We categorise the model scope as the following: Process: when variables were related to a specific stage of the life cycle (e.g., requirements elicitation, coding, and testing). Project: when variables were from a project (e.g., effort, risk, productivity, cost, resources, delay, and quality). Product: when data were gathered from product metrics (e.g., lines of code, number of defects, and cohesion). Multi-project: when data were retrieved from distributed software development projects (e.g., communication overhead, distributed resources). RQ4. When did the problem happen? In other words, what was the software life cycle phase the study focussed on? We want to highlight which stage of the software life cycle (e.g., development or evolution) the study is addressing so that we can find out which stage receives more attention from researchers. It is important to distinguish this question from RQ3. For instance, a paper might use BBNs in the scope of a product (RQ3) but focus on the maintenance (RQ4) phase. RQ5. How was the BBN modelled? Our goal with this question is to identify the Bayesian network modelling and execution tools used by the authors of the studies. The availability of tools is a requirement for conducting studies, as it helps researchers focus on the problem under investigation, eliminating the effort required to develop a new tool. Knowing which ones exist and which ones are the most popular can help other researchers in conducting new studies. RQ6. Where and when was each study published? Finally, we want to record where the paper was published (i.e., conference or journal) and its publication year. We believe that this data can help us map the evolution of research using BBNs in the context of software development project management. In the following sections, the search strategy is presented, which was initially conducted by the first author and then reviewed by the second author. 3.2 Paper search strategy We adopted the PICO criteria (population, intervention, comparison, and outcome) suggested by Kitchenham and Charter [12] to identify keywords and formulate search strings from the research topic. The following steps describe the search strategy adopted during the review: Step 1: Identify keywords and synonyms for the terms used in the search query. The terms adopted are in English. Thus, we used the following terms: Population: Publications regarding project management issues in the software development process. Related keywords: project management, software development, software process, and software engineering. Intervention: Publications that apply Bayesian networks. Related keywords: Bayesian network, decision network, and BBN. Comparison: No empirical comparison is made. Outcome: Set of papers related to the use of Bayesian networks to support project management issues in the software development process. No measurable outcome is considered. Related keywords: project management. Step 2: Search string elaboration. We need to elaborate a search string through the combination of the keywords with the Boolean OR when it is necessary to arrange alternative words and synonyms and the Boolean AND to join significant keywords. The search string enables the use of advanced features provided by digital library search engines. (‘Bayesian network’ OR ‘decision network’ OR ‘Bayesian belief network’) AND (‘project management’ OR ‘software process’ OR ‘software engineering’ OR ‘software development’). Step 3: Search string refinement. Due to the distinct functions and features available in different search engines, the search strings for the digital libraries were similar but not the same. It was necessary to adjust the search strings to fit the rules of each search engine. Moreover, we observed that the inclusion of some common terms of the project management domain (e.g., cost, risk, estimation etc.) resulted in the retrieval of a larger number of relevant papers from some digital libraries. The adjustment and inclusion of the added terms resulted in the following search string: (‘Bayesian network’ OR ‘decision network’ OR ‘Bayesian belief network’) AND (‘project management’ OR ‘software process’ OR ‘software engineering’ OR ‘software development’ OR ‘software quality’ OR ‘effort estimation’ OR ‘cost estimation’ OR ‘risk management’ OR ‘defect prediction’). We performed a two-stage strategy for searching. In the first phase, we piloted the search strings into a year chosen randomly (i.e., 2006), trying to refine the result to eliminate irrelevant hits. To assess the result of the search strings, we compared the result with a small sample of studies we already knew (e.g., [16, 17]). At the second stage, after the pilot's results were satisfied, we separated search strings for searching within titles, abstracts, and keywords. We had not specified a lower date limit due to the lack of prior information about the very first paper that meets the search criteria. After the first research attempts, we identified the oldest paper in 1999. Then, we specified 1999 to start the research and 2018 as the final year, since we carried out this study throughout 2019. We searched for studies in some major digital libraries with a relevant bibliographic collection of computer science. The list of sources includes Science Direct (www.sciencedirect.com), the IEEE Computer Society Digital Library (ieeexplore.ieee.org), Scopus (www.scopus.com), and the ACM Digital Library (dl.acm.org). Table 6 presents the search string applied in each digital library and the respective search results. TABLE 6. Search string and digital libraries Digital library Search string Search results ScienceDirect (‘Bayesian network’ OR ‘decision network’ OR ‘bayesian belief network’) AND (‘Software process’ OR ‘Software Engineering’ OR ‘Project Management’ OR ‘Software Development’) 27 ACM ((‘decision network’ ‘bayesian network’ ‘bayesian belief network’) AND ( ‘project management’ ‘software process’ ‘software engineering’ ‘software development’ ‘software quality’ ‘effort estimation’ ‘cost estimation’ ‘risk management’ ‘defect prediction’)) OR acmdlTitle:((‘decision network’ ‘bayesian network’ ‘bayesian belief network’) AND (‘project management’ ‘software process’ ‘software engineering’ ‘software development’ ‘software quality’ ‘effort estimation’ ‘cost estimation’ ‘risk management’ ‘defect prediction’)) 177 IEEE ((((‘Abstract’: ‘Bayesian Network’ OR ‘decision network’) AND (p_Abstract: ‘project management’ OR ‘software process’ OR ‘software development’)) OR ((p_Title: ‘Bayesian Network’ OR ‘decision network’) AND (p_Title: ‘project management’ OR ‘software process’ OR ‘software development’)))) 101 Scopus TITLE-ABS-KEY((‘Bayesian network’ OR ‘decision network’ OR ‘Bayesian belief network’) AND (‘project management’ OR ‘software process’ OR ‘software engineering’ OR ‘software development’)) AND DOCTYPE(ar) AND PUBYEAR<2019 AND (LIMIT-TO(SUBJAREA, ‘COMP’)) 113 We used a reference management tool to remove duplicates and manage the considerable number of references that resulted from the search queries. 3.3 Study selection criteria The adopted inclusion criterion establishes that all the papers should apply BBN to support project management activities in the context of software development. From the candidate studies retrieved from digital libraries, we reviewed the titles, abstracts, and keywords of each paper, generating an initial selection. When these information fields were not enough for a decision about inclusion or exclusion, we extended the review to the body of the document and its references. We adopted the following exclusion criteria: Papers that