# Software Project Risk Identification Based on Scrum Artifacts
## 4 Experimental Evaluation of the Risk Identification Framework
### Results of the Framework Adaptation Stage in Experimental Projects  
All three projects underwent the framework adaptation stage, where the project teams were provided with the proposed structure of artifacts, attributes, and risks. Teams performed the refinement of the elements of the structure, their associations and threshold values based on the expertise of the team. Table 4 summarizes the adaptation results, illustrating the utilization of artifacts, attributes, and risks for each project, compared to the proposed initial structure.  
All projects used Jira to export project execution data to the Excel file and mapped the data from Excel to corresponding attribute values in the framework application.  
Project 1 was experimented upon during the initiation and execution phases over one year. Consequently, the team opted to focus on artifacts, attributes, and risks relevant to the initial stages of the project. This decision led to the exclusion of several artifacts, including AR01.2 (Product Goal), AR01.3 (Epic Discovery), AR01.4 (Design Discovery), AR01.6 (Product Roadmap), AR02.2 (Epic Implementation), and AR03.2 (Product Release Plan). The team determined that the characteristics of these artifacts could be effectively represented through attributes associated with other related artifacts in their project. For instance, attributes such as AT042 (Number of Epics) and AT046 (Number of Initial Tasks), which are tracked within AR01.1 (Product Backlog), were deemed sufficient in this case for identifying risks such as R006 (Poor prioritization and unclear vision) and R015 (Poor communication). Despite a selective focus on artifacts, the adoption rate of risk categories was relatively high, as seven out of the nine risk categories were utilized. Excluded categories were RC5  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Quantitative Indicator** & **Project 1** & **Project 2** & **Project 3** \\ \hline Artifacts used & 55\% & 73\% & 82\% \\ Attributes used & 89\% & 93\% & 70\% \\ Risk categories used & 78\% & 78\% & 89\% \\ Risks used & 69\% & 83\% & 86\% \\ Risk-attribute associations used & 48\% & 82\% & 87\% \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results of the adaptation stage in experimental projects.  
(Design), RC8 (Human), and several risks from categories RC6 (Coding) and RC7 (Testing), based on the same rationale as the exclusion of artifacts.  
Project 2 and Project 3 were introduced with the application of the proposed framework during the later phases of their lifecycles. At the time of the framework application, both projects were mature, characterized by continuous delivery and well-established operational practices. Due to the absence of anticipated specific risks, these projects have opted to monitor a broader set of artifacts than those of Project 1. The adoption of risk categories was also significantly higher compared to Project 1, with Project 2 and Project 3 showing adoption rates of 78% and 89%, respectively. On the other hand, Project 3 adopted the smallest number of attributes due to the decision to exclude unsupported attributes--those for which data were not gathered within their project management environment, e.g., AT041 (Number of tasks without epic), AT031 (Number of open versions). Although no specific risks were anticipated, the teams aimed to monitor a diverse array of potential risks to ensure that projects remain on schedule, as reflected in the high percentage of adopted risk-attribute associations.