## 4 Discussion of Results
They are presented here separately in two figures: Figure 2 shows how priorities changed over time for six features not yet delivered and Figure 3 presents how priorities changed over time for seven features that were already delivered to customers as a part of subsequently released new product versions. Moreover, in Figures 2 and 3 only a subset of features is presented for legibility reasons, as the number of features was a few times higher. As we can see, even though the prioritisation was dynamically updated, we could find a situation where two (or more) features were having the same relative priority (FN7 in Figure 3 and F2 in Figure 2). In such a situation, the engineering team consulted with the Product Manager which feature should be selected for implementation. In this case feature F2 was chosen. An additional noteworthy observation on the feature roadmap dynamics is the fact that features prioritised on top of the list once at a certain point of time may be depriitized, and not implemented at all. This is the result of the dynamics in the sales process; e.g., the window of opportunity for the sale may disappear (see features F3, F5, F6 in Figure 2). Figure 2: Product roadmap: planned features. When we examine the data to understand variability for selected features, the following complementary view may be presented (see Figures 4 and 5 below). Note that T1-T8 reflects the moment in time, when observation was collected. Next we have collected all TD Items (TDIs) which were identified, gathered, and described by the development team prepared for implementation. This exercise was done during refinement sessions held every sprint. Each item (see: Table 2) was categorised to understand its type and influence over DevOps activities. Furthermore its priority (relative position) reflected the decision time, and status indicates whether TDI was already refactored, and if there was a valid reason not to implement the prioritised TDI, it was explicitly mentioned (\"too big\": could not have been prioritised over features, \"blocked\":  
Figure 4: Changes in prioritisation per feature: features not yet delivered. Figure 3: Product roadmap: implemented features. in one case TDI awaited other implementations, might be either feature-based or different TDI). As the order of implementation of TDIs was analysed by us in this paper, for each TDI not following the prioritisation (refactored before higher priority ones), we found out that \"context\" was the reason for such a decision. It means that development in the given area was pending, and the scrum team used a common practice called _continuous refactoring_, addressing smaller-size issues while implementing other product backlog items. Additionally \"context\" also meant TDI refactorings stemming from broadened product definition (DevOps perspective), they were identified and implemented in parallel with currently implemented functionality. We should keep in mind that the roadmap of improvements should reflect dynamics of the feature prioritisation. However, in our case study these changes were negligible against the conclusions made. The reason for this is the fact that the majority of technical debt items actually helped in future development of many features concurrently. In order to answer the question whether execution followed the plan we assessed the moments when a given TDI was selected for refactoring. Only then could we determine whether the decision adhered to the prioritisation. Throughout the period of the case study these TDIs which raised concerns (prioritised low and executed before others) were never prioritised high. Hence the simplified view of the TDIs prioritisation dynamics was presented, and the analysis focused on the reason why several lower priority TDIs were addressed. Therefore, even though the top most items were addressed (TDI1-5, TDI7-8), there were a few TDIs addressed which were prioritised significantly lower on the list. The reason for that was continuous refactoring effort, a practice well aligned with agile development pace.