## 4 Materials and Methods  
To identify AESS activities, artifacts, and roles found in the literature, we conducted an extensive literature review covering conference and journal papers indexed in four bibliographic databases (i.e., ACM DL, IEEE Xplore, Scopus, and Web of Science). The search queries included a combination of keywords: agile, lean, secure, security, software, development, engineering, method, and management. The queries were used to search titles, abstracts, and keywords of bibliographic records published since 2000. Papers describing or testing a secure software development method compatible with agile methods were retained in the pool of relevant papers. Additionally, the snowball method [52] was employed to widen the reach of the literature review. During this process, we screened the references of initially identified papers for potentially relevant papers. Further, we backward-snowballed the papers with Google Scholar (i.e., we searched for papers citing the identified papers). The papers in the pool of relevant papers were carefully examined to identify the activities, artifacts, and roles they describe. The original names of activities, artifacts, and roles were preserved at this point, even though they may not be indicative of their types. For example, generic security user stories refer to an activity rather than an artifact, as the name suggests in [38]. One researcher extracted the data, and another researcher double-checked the extraction. To make sure that only activities, artifacts, and roles directly contributing to the security of developed software are included in the study, two researchers independently classified them as either related or non-related to security. The researchers disagreed in 11.3 percent of cases. The [PERSON]'s kappa (\(\kappa=0.72\), \(p<0.001\)) suggests substantial agreement between the researcher's assessments [53]. Classification inconsistencies were solved with consensus before moving on to the next step. To determine similarities among the identified activities, artifacts, and roles, similarity matrices were employed. Similarity matrices are \(n\times n\) tables of binary or multi-value scores indicating the likeness between two data points (e.g., activities). They can be used for reducing data in larger qualitative datasets [54]. This enabled us to consolidate activities, artifacts, and roles. The consolidation was performed in three steps. First, two researchers independently created separate similarity matrices for activities, artifacts, and roles. Similarities were scored on a 3-point scale: 0.0 (not similar), 0.5 (partially similar), or 1.0 (very similar). The cross-evaluations of identified activities, artifacts, and roles (e.g., comparing all identified activities with each other) resulted in 3419 assessments by each researcher. The researchers assigned different similarity scores in 1.6 percent of cases. The [PERSON]'s kappa (\(\kappa=0.96,p<0.001\)) suggests very strong (almost perfect) agreement between the researchers' assessments [53]. Such a strong agreement may be attributed to a large number of obvious dissimilarities among activities, artifacts, and roles. Second, similarity matrices produced by each researcher were consolidated into final similarity matrices by calculating the means of individual scores for each pair of security elements. Similarities in the final similarity matrices were therefore scored on a 5-point scale: 0.00 (not similar), 0.25, 0.50, 0.75, 1.00 (very similar). To consolidate activities, artifacts, and roles, we employed [PERSON]'s hierarchical clustering. Since [PERSON]'s method requires distance matrices, we transformed the final similarity matrices into distance matrices by calculating \(x_{distance}=1-x_{similarity}\). The final number of clusters, their naming, and brief descriptions were determined in a focus group of three researchers. The number of clusters for each type of security elements was determined by minimizing the number of clusters while retaining the similarity of intra-cluster security elements.