# FRAMUX-EV: A Framework for Evaluating User Experience in Agile Software Development
## 6 Validating the Framework
### Qualitative Results
#### 6.2.2 Qualitative Results: UX Artifacts  
In addition to the UX artifacts presented in the survey, 15 additional artifacts were suggested for consideration. However, almost all the suggestions were specifically related to UX design methods rather than UX artifacts (e.g., user scenarios, customer journey, or user flows). The suggestions that could be considered artifacts (e.g., documents, guidelines, and repositories) were the \"repository of UX tools\" (P5, P20), \"KPI guidelines for stakeholders\" (P17), and \"content repositories\" (P4). \begin{table}
\begin{tabular}{c c} \hline \hline
**Participant** & **Additional Comments** \\ \hline P2 & “It would be necessary to clarify the methodology you are thinking about for agile software development and what kind of teams you are going to have. The methods will eventually depend on that”. \\ P3 & “(Evaluation methods) must be used at the right time, always proportional to the problem to be addressed”. \\ P4 & “RITE is complex because you are improving from one opinion/experience, without considering more user observations to make a total improvement”. \\ P11 & “Each method is related to a stage and what you want to evaluate, not all of them work with everything”. \\ P12 & “A method based on guiding and supporting proposals is more effective because it helps to mitigate disagreements among the users”. \\ P15 & “I find this list quite interesting and complete. They are essential methods with which it is possible to conduct a complete evaluation of the usability and therefore of the user experience”. \\ P17 & “Evaluating mockups does not let you know what users do, but what they could do. They are useful for providing input to the development team”. \\ P19 & “Complicated question, as it depends on the type of system/flow/design you need to evaluate the method to use, but the user testing, evaluation of mockups or prototypes, or user interviews are effective and easy to estimate in time as a task”. \\ P23 & “I believe that the methods that work best are those that make the user interact directly with the implemented design”. \\ P24 & “(The evaluation methods mentioned) provides an excellent roadmap”. \\ P25 & “I believe that an appropriate combination of some of these can provide valuable information for the evaluation”. \\ P32 & “I consider that the most complex thing to do is the heuristic evaluations because of the time involved and that they do not go with any agile methodology”. \\ P34 & “Of all that exists, we use very little, which is bad for the final product”. \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Additional comments on UX evaluation methods mentioned by practitioners. The majority of the respondents seemed satisfied with the artifacts presented, indicating that they would not eliminate any of the artifacts proposed (24 of 34 participants). \"UX personas\" was the most frequently mentioned artifact for elimination, with 4 out of 34 respondents suggesting its removal (P2, P5, P19, P20), followed by the \"UX design system\", with three responses (P9, P19, P26). Although few comments were obtained, there were different opinions regarding the proposal. For instance, P15 and P20 highlighted that it is a \"good initiative\" and \"It is an interesting proposal\", respectively. P24 and P32 highlighted the complexity of the integration and learning of new artifacts in an agile context. Table 10 presents additional comments related to each UX artifact and future actions to be considered. In summary, there were generally no negative comments regarding the proposed UX artifacts. However, it is necessary to better specify the description of these artifacts to avoid